Backin’ Up is Hard to Do

I hate computers.

All I wanted to do was backup my laptop. I’d been doing ad hoc backups for years (a batch file that zips and copies all the important directories) but that system was clearly insufficient. Replacing my ad hoc system with something real was way more complicated than it should have been.

If you remember just one thing from this post, remember 0x8078002A.

And remember the importance of demanding software that just works.

Still annoyed after all these years (update February 2014)

I just bought a 4 TB backup drive from Costco. It doesn’t work with Windows 7’s image backup system and with this particular drive there is no way to make it work. I returned it. Sorry Costco, it’s Microsoft’s fault, but I still had to take it back. Apparently only Western Digital’s large drives can be made to work with Windows 7’s image backup. This problem has been fixed in Windows 8, but Microsoft needs to understand that people aren’t going to upgrade to Windows 8 in order to get backups to work – they’re just going to be annoyed, or switch to OSX.

More details and other updates below.

Network Attached Storage is Not Allowed Storage?

I initially tried buying a 2 TB backup NAS setup but it had a driver bug which made my laptop slow. That’s been fixed, but Windows Backup doesn’t support network drives unless you’re running a super-duper version of Windows. This is not entirely obvious, but I found the information in Windows Help:

image

It’s disappointing that Windows Professional’s Windows Backup doesn’t support network drives, and it’s bizarre that Windows doesn’t have an Upgrade Now button in the backup UI. Whether it’s backup limitations or Remote Desktop restrictions, Microsoft doesn’t seem to understand that taking features away from the low-end versions doesn’t generate revenue unless you also encourage people to upgrade.

USB FTW?

I then tried a 3 TB Western Digital USB drive. And it eventually worked.

I initially told Windows to backup using the default settings and then waited. For hours. I didn’t time it but I did run a bunch of errands and then go for dinner and it was still going. And then it failed:

image

Wow. Failing is annoying, but failing after about six hours? That’s really horrible. Luckily Windows Backup gave me a clear and informative error message that told me exactly what to do in order to resolve my problem. And yes, I do commute primarily through sarcasm.

What Windows Backup really gave me was the dialog above with the More information button. If I clicked that I got this dialog:

image

And if I clicked Show Details I got a bit more information:

image

Notice the hexadecimal error code near the bottom. Then if did a web search on 0x8078002A and clicked on the correct link:

This problem is caused by an issue with the Windows 7 Backup and Restore utility when using drives with 4K [sic] Logical sector sizes, which are found on WD 2.5 TB and larger external drives. This is different from an “Advanced Format” (AFD) drive which use 4K [sic] physical sector sizes. AFD drives should not cause this problem.

My eyes are glazing over. Feast your eyes on the chart listing support for various sector size: http://support.microsoft.com/default.aspx?scid=kb;en-US;982018

If your drive has a logical sector size of 512 bytes then it is supported as long as it has a physical sector size of 512 bytes or a physical sector size of 4 KiB and a compatible operating system which is Windows 7 with SP1 or MS KB 982018 did I mention that my eyes are glazing over.

Unfortunately my drive, apparently, has 4KiB logical sectors which are not supported, at least not for the image backup stage of Windows backup.

4-KiB sectors are becoming common for a few reasons. One is that the larger sectors are slightly more efficient, but another reason is that the Master Boot Record partitioning system (the old way of formatting drives) can only handle four billion (technically 2^32) sectors. With 512-byte sectors that sets a limit of 2 TiB (2.2 TB). Drives larger than that either need to use 4-KiB sectors (letting them go up to 16 TiB) or they need to use the GUID partition table (GPT partitioning scheme) which allows more sectors.

Set a course for the 512 sector

Western Digital recognized that this is a problem and they let consumers reformat their drives to use 512-byte logical sectors and the GPT partitioning scheme. This will make it incompatible with Windows XP but I don’t care about that. So, as recommended, I ran the Western Digital Quick Formatter tool. But before getting to that tool, feast your eyes on this warning:

image

Think about that. I’m doing all of this so that I can backup my data, and this warning is basically saying “before backing up your data please backup your data.” Yes I know they are talking about the data on the backup drive, but still. It’s funny. I’m laughing inside.

The default is not the default

The Quick Formatter tool was pretty straightforward, but it does play some linguistic games with the word ‘default’. To be clear, this drive came from the factory formatted in XP Compatible mode – the factory default for this drive is actually XP Compatible! The factory default is not “Factory Default”. How confusing.

image

Later versions of their formatting tool have changed (and moved) the choices. The correct choice if you want 512-byte sectors is “Most Compatible”.

image

Early summary

If you’re getting impatient let me just say that the reformatted drive worked. I got the backup I wanted, finally, and I’ll get another backup every Sunday. But I have a few more things to complain about. And get off my lawn.

What about Seagate?

Western Digital has an option to format their big drives with 512-byte sectors, but Seagate does not, and I see no sign that other drive manufacturers offer such an option. When my wife bought a 4 TB Seagate backup drive from Costco she failed to verify that it could support 512-byte sectors (I don’t know what she was thinking) and it can’t so we had to return it. Seagate supplies their Disc Wizard software that lets you choose between MBR and GPT partitioning schemes, but in both cases it uses 4-KiB sectors.

Elevation for everything

I have my computer set up to require a password in order to get administrator access. I think this is the only sensible way to set up a computer. It means I can hand it to friends and relatives without worrying that they will do something crazy that will compromise it. Normally the need to type in a password for elevation is not a problem, but when setting up backups, and diagnosing problems with backups, it is absurd. Feast your eyes on the backup control window:

image

Every shield icon is something that requires me to type my password. Setting up a backup, backing up now, retrying a failed backup, seeing details of a backup – type your password please. I did all of these operations multiple times and I got a lot of typing practice. If I could launch the backup UI as administrator and then control all of its functions without further passwords then that would be fine, but that does not work. There are actually four more shield icons in the backup control panel:

image

If (almost) every option in your UI requires elevation then maybe you should allow your UI to be elevated!

Backup usage (update October 18, 2013)

I’ve had to use my backups a few times to restore individual files, and they have worked. However one time I found that the backup had mysteriously stopped a few months prior – the drive had been unplugged. Windows didn’t bother to notify me that the backup drive was inaccessible – but luckily the file I needed was still available. However a backup system that stops without loudly notifying the user? Not acceptable.

More recently I used the image backup to do a hard drive upgrade – see this blog post for more details.

Handling out-of-space (update October 22, 2013)

A related problem with Windows Backup is it handles out-of-space conditions poorly. To make Windows Backup useful for non-experts (like my parents) it needs to keep running, which means it needs to have an option to automatically age-out old backups. Instead Windows Backup seems to halt when the backup drive fills up. That means that its implicit policy is that old files are more important than new files.

Windows Backup does have an option for controlling how many system image backups are retained, but the UI is incomprehensible to me:

The UI says that the current setting will use a maximum of 419.15 GB for backup history. One might expect that this is the maximum space that it will use for system images. Instead it appears to be the maximum space that it will use for system images not including the latest system image. So, the total space used for system images with this setting is roughly twice as much. I think. And they probably mean GiB not GB which adds another 7.5% of distortion.

It’s also confusing that the UI contains two similarly sized numbers with opposite meanings – one is the amount of space that will be used by a setting, and the other is the amount of space that will be saved. That’s like saying “Item A will cost $10 but item B will save $9.95 – which would you like?”

Hexadecimal error codes are not okay

This is not a good user experience. Backing up is important, it can be easy, it should be easy, but right now it is a complete disaster.

  1. Microsoft backup needs to support writing disc images to drives with 4 KiB logical sectors. 4 KiB sectors showed up in 2010 and are mostly supported by Windows 7. The only missing support I have found is in Windows backup’s image backup, which is exactly where the support is needed most, because backups are when huge drives are most needed. @MicrosoftHelps just said no. Windows 7 will be supported for years to come, and soon 2 TB or smaller external drives will be rare beasts, so Windows 7’s image backup is going to gradually become useless!
  2. Windows backup should fail immediately if the sector size is not supported. Waiting six hours to fail is ridiculous. I don’t care if the first six hours of backup (file-by-file copies presumably) have looser requirements than the image backup – consider the user experience and make it better.
  3. Windows backup should not require separate elevation for each individual task. Not every Windows customer runs Windows in low-security mode. Designs like this are why users hate UAC.
  4. Windows backup should not give hidden hexadecimal error codes for common errors. If backup drives for sale at Costco and OfficeMax don’t work with Windows Backup then the error message needs to be front-and-center with a link to the fix-it article.
  5. Selecting what files to backup could be way clearer.
  6. Western Digital needs to make the incompatibility more visible. I carefully read the packaging and the manual and it says the drive is compatible with Windows 7. It says that reformatting is needed for Mac OS, but there is no mention of reformatting being needed for Windows 7. Presumably their expectation is that customers will use Western Digital’s backup program, but if that is required it should be stated in large bold letters.
  7. Seagate and other drive manufacturers should make compatibility with Windows backup more of a priority. Whether they do this by pressuring Microsoft to better support 4 KiB sectors or by allowing formatting with 512-byte sectors I don’t care.

Sure, if I’d used Western Digital’s backup program I would have avoided this problem, but I didn’t want to. I just wanted a big drive that I could backup my data to using the software of my choice.

Everybody should back up. Therefore backing up needs to be easy.

About these ads

About brucedawson

I'm a programmer, working for Google, focusing on optimization and reliability. Nothing's more fun than making code run 10x faster. Unless it's eliminating large numbers of bugs. I also unicycle. And play (ice) hockey. And juggle.
This entry was posted in Computers and Internet and tagged . Bookmark the permalink.

17 Responses to Backin’ Up is Hard to Do

  1. barrkel says:

    Have you tried a restore? I’m sure you know that it’s not a backup if restore doesn’t work, especially a restore after the main system has been obliterated. I recall testing restore in XP’s Backup, and it not working. Maybe it’s fixed now; maybe it will work for you. All I know is that I don’t use it any more. I use CrashPlan, with one instance running on my Nexenta NAS and another on my desktop. CrashPlan can back up to any other machine also running CrashPlan (using it as a peer to peer backup this way is free AIUI), as well as CrashPlan’s cloud storage (their main way of making money). So I have both local and remote backup; the former for performance, the latter for peace of mind. And it works.

    • brucedawson says:

      I have tried a restore. I’ve tried the image backup and restore in order to upgrade to a larger hard drive, twice, and it worked quite well. After the restore you just resize the partition to the new drive’s full size and suddenly you have a lot more space.

      I’ve also done some restores of individual files. I found the process to be fussy, but ultimately successful.

      I also use SyncToy for all of my photos, videos, and music. This keeps it synchronized to another machine, and it also gives me a chance to find out about unplanned changes.

      Finally, I copy all of my photos to an external drive and take them to work occasionally.

  2. billco says:

    This is one area that continues to annoy me with Windows 7. The backup feature is such an afterthought. For years, MS bragged about shadow-copy functionality being ahead of the competition, but it’s 2012 and you still can’t find a reliable backup solution anywhere. Meanwhile, the Mac has had Time Machine for a while now. I even have it backing up to my NAS, which admittedly required some hacking to make my Linux box impersonate an Apple-branded “Time Capsule”, but hey, it works and has saved my hide a few times already. Why, with our giant installed user base and overpriced OS, can we not have something as elegant on the Windows side ?

    And barrkel is bang on the money: restoring your backup is not guaranteed. If your PC is a plain-jane, single disk, single optical drive job, it typically works fine, but as soon as you delve into multiple disks, RAID, SSDs or anything that deviates from the 99% majority, expect to lose a day messing with the bootloader. Perhaps the most infuriating quirk stems from drivers that like to renumber disks, so the stripped-down rescue environment sees one thing, your actual OS sees a different sequence, and you get the exciting job of restoring over and over, trying every permutation of drivers and loading sequences (!), until things click into place. Even the startup repair tool can’t figure it out half the time. The last time this happened, I ended up restoring to a virtual machine just so I could pull my files out and start over with a clean OS.

    I’ve pretty much given up on Windows Backup. I store my user profile on a NAS which does daily snapshots, remote replication and all that fun stuff. If my OS gets trashed, I simply pop in a USB drive with my slipstreamed Win7 installer; an hour later I’m back in VS or Eclipse where I left off. If I have a brain fart and accidentally wipe out an important directory, I dig it out of yesterday’s backup – or the dailies going back three years. While my homebrew solution doesn’t have a nice GUI like the Mac, it gets the job done.

    My weathered words of advice to you are: keep your batch files! They have far fewer points of failure than Windows Backup. If you want to get clever, use Robocopy or rsync to do differential backups, and if there’s stuff you cannot afford to lose, get one of them fancy Amazon-S3 cloud sync utilities. As long as your system relies on regular file copies, you won’t ever be stranded with an non-restorable backup image.

    • brucedawson says:

      Sage advice. I’ll retain the batch files.

    • Yuugi says:

      I’d actually be very, very interested in hearing more specific details about how you set up your backup system.

      I’m currently making do with Windows Backup (which I am not happy with), but moving the profiles to a decent NAS seems like a much more sensible thing to do. Which I honestly have no idea on how to go about on Windows 7/8/8.1 (five machines total).

  3. Rob Allen says:

    I recently purchased an WD My Passport @ 1TB for backups of a linux box and ran into similar issues until I reformatted the drive. We’re at a weird in-between state with drive size now and sector size isn’t something the average consumer should have to know or care about.

  4. djchew says:

    Your basically my hero.
    I agree with all of your statements about usability and user interface. It seems like most companies just don’t get it. Make it logical and easy to use without removing functionality.

    • djchew says:

      Also, Windows Update uses the same types of hidden hexadecimal error codes for common errors. I have had a few instances of issues with that where even Microsoft could not figure it out. The answer from Microsoft Support was ‘format and reinstall. we have no idea what the error means’. That was after over a month of back and forth and no less than 5 support ‘engineers’.

    • brucedawson says:

      Thanks! I could also ramble on about problems with the restore process. I needed to restore my Windows Live Photo Gallery database after I damaged it doing some experiments. Windows Backup said it had restored my files, but then said “except for these ones…” The list of files that it had failed to restore was all of them. Apparently those files can’t be restored in place. I had to start again, find the six-level deep directory, restore to a different location, and then manually copy the files over. This is a fail on multiple levels: it shouldn’t claim success when all of the files failed, it should give an option to retry with tweaked settings instead of starting over, and if it is going to prevent me from restoring in place it should explain why, since otherwise I’m just going to do it anyway.

      Sigh…

  5. yuhong says:

    “This will make it incompatible with Windows XP (that’s the part I’m confused by)”
    XP do not support more than 2^32 logical sectors, which with 512 byte sectors is 2TB.

  6. yuhong says:

    And Win8 do have official support for 4k logical sectors: http://support.microsoft.com/kb/2510009

  7. Marius says:

    Honest question…..why would you format the drive with the WD utility instead of the OS one ?

  8. Pingback: Windows Slowdown, Investigated, Identified, and Now Fixed | Random ASCII

  9. Pingback: Basics of Digital Photo Organization | Random ASCII

  10. Pingback: Hard Drive Upgrade | Random ASCII

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s