Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Friday, September 2, 2011

Reconciling File Times Between Unix and Windows

I did some enhancements for AnyBackup not too long ago that required comparison of hash keys generated using (in part) files' last modified time. I discovered an oddity that, despite years of being on the platform, I'd never known about Windows. File meta data has a resolution of 2 seconds. Don't believe me? Take a closer look. What this means is that the modified time (in seconds since the epoch) can never be odd, it's impossible.

It also means that when you copy a file from Linux (which tracks meta times accurately to 1/100 of a second) to Windows, the time is rounded up or down accordingly. The oddness that ensures is that when you look at the Windows file copy it'll (sometimes) show a one second difference as compared to the Linux copy. (It all depends on rounding.)

My gut reaction was to just divide the times by 100 and remove the two least significant digits from play, but that lowers precision and doesn't quite guarantee that you'll avoid the problem entirely. (Imagine your Unix modified time is 1699999999, in Windows this will become 1700000000 -- oh the imprecision!)

When you get the modified time of a Linux file (say through a Samba share) it'll invariably have two digits to the right of the decimal place. (At least when doing so via something like Python, not from a Windows property box.) If you convert it to an inegert to remove these the number will be rounded up or down accordingly. Instead I decided to do something like the following:

  1. Round down (regardless of the two digits to the right of the decimal)
  2. Convert to integer
  3. Check if the number is even (modulus 2)
  4. If it is even, add 1
So going back to our initial example, say your Linux file comes back with a modified time of 1699999999.42:
  1. 1699999999.00 (Round down)
  2. 1699999999 (Convert to int)
  3. Not even (1699999999 % 2 = 1)
  4. 1700000000 (Add one)
  5. Voila, it matches the new Windows copy
(Yes, the conversion to an integer isn't really necessary, but we're dealing with whole numbers already anyway, so why not?)

The above steps ensure that you'll end up with a Windows compatible view of the modified time. So what does this look like in Python code? See below:


 mtime = int(math.floor(os.path.getmtime(fileLocation)))  
 if mtime%2:  
   mtime += 1  

Monday, February 14, 2011

CIFS share mount command hanging

I ran into an issue today when attempting to mount a samba share via cifs on a linux client where the mount command was taking two or three minutes to complete. Here's what the fix was in my situation, apparently cifs will first attempt to make a connection via port 445, in my iptables config I have that port dropped, eventually cifs falls back to port 139, however, if you specify port=139 in the mount options, it will go to the correct port the first time around and complete immediately.

Sunday, February 13, 2011

Restore drives that have been erroneously marked as failed in a Raid 5/6 array

Transferred from my old blog.
Scenario:

You have a raid5/6 array (/dev/md0) in which one or more drives have been marked as failed. For instance, I had a motherboard problem in my server recently which would cause my esata controller to spontaneously reset ports and knock 3-4 drives off my array at a time, putting the array in a failed state. All is lost, yes? No! Since in my scenario the array immediately fails upon having more than two drives disappear (this is a raid6 array), no data has changed on the actual file system, you can use the following command to force reassemble the drive. If possible, mdadm will up the event count on the “failed” drive(s) and clear the faulty flag for it. NOTE: Be careful with this, if a given drive was knocked out of your array before a modification to your array (i.e. writing to the array or an array reshape), this can cause massive, non-recoverable data corruption. Only do this if you are SURE that your array contents has not been changed/written to between the time these one or more drives were removed from your array and now.

If you have filled out mdadm.conf with your array and corresponding drives:

mdadm -Af /dev/md0

If you do not have mdadm.conf filled out and rely on mdadm auto assembling your array upon start up, use the following:

mdadm -Af /dev/md0

Where devices is replaced by the drives that make up your array, example:

mdadm -Af /dev/md0 /dev/sd[b-d] /dev/sd[h-o]

Followers