AnyBackup 0.9.6 Released

Hot on the heels of 0.9.5, I've released 0.9.6 to address some immediate deficiencies resulting from the major overhaul of the underlying application.

• File cache layer between SQLite and AnyBackup
• This addresses the lag the can occur when browsing an indexed drive, especially when you have > 100k files across all indexed drives
• Improved search dialog making use of the new SQLite backend
• Search by drive
• Specify search type: Contains, Ends with, Starts With, or pass in a SQL string (i.e. %.mp3)
• Specify file type (directory or file)
• Search dialog is persisted so your input is saved (unless a drive is added or removed -- then the dialog is destroyed and recreated)
• Clicking on a result file no longer automatically displays it in the content pane, instead this has been moved to a popup menu option
• Changed remote indexing port to not conflict with standard XVNC port

AnyBackup 0.9.6 -- As always please note that these are all items I own or have bought in one form or another -- don't sue me. :)

The new search dialog in 0.9.6

The new 'Show In Content Panel' option

AnyBackup 0.9.5 Released

I've made a lot of changes since 0.9.4 in November last year. I put 0.9.5 up for download tonight, enjoy!

Major changes:

• SQLite
• Super fast
• Less memory usage
• Comparable disk usage
• More flexible
• Externally accessible
• Many minor bug fixes
• Complete / consistent icon pack
• Backup sets
• Thread crash detection
• Logging to homedir so may work on UAC enabled machines
• Super strict UAC settings seem to deny access to network drives, so if you use network drives like me you still might have issues

http://code.google.com/p/anybackup/downloads/list

Not that I've heard from too many people on the app before, but as always, if you hit issues, don't hesitate to let me know or file an issue on the Google code page.

AnyBackup 0.9 Released

I've released AnyBackup 0.9 today. The GUI has been overhauled, a few key features have been added, and many pain points have been sped up significantly.


The major changes are that I've switched the GUI up to use AUI, it's a lot prettier and easier to get around in. I've tweaked remote indexing and switched to Pyro for sending remote python objects. This cut the remote indexing time in half more or less. I've also added the ability to select which directories you want to backup up from content drives.

Yes, I'm aware the screenshot below says 0.8, I forgot to update this before building the version and it's such a minor issue I saw no reason to rebuild/upload for it.


Download at: http://code.google.com/p/anybackup/downloads/list


Changes:

 Issue 43  - Update GUI to use aui
 Issue 44  - Search result file click broken
 Issue 45  - Add status text to splash screen
 Issue 46  - File view area not showing whole directory path
 Issue 47  - Switch remote indexing to use Pyro
 Issue 48  - Avoid using deepcopy in threaded actions
 Issue 50  - File icon type not always correctly displayed.
 Issue 51  - Remote index function not updating drive space information
 Issue 25  - Allow addition of folders only
AnyBackup 0.9

AnyBackup 0.6 Released

I've released AnyBackup 0.6 today!

Changes:

• Resize-able elements
• File sizes are now displayed (in MB) alongside the file names in the browser and result tables
• Result page selector is only cleared at the appropriate times
• Sticky backup mode keeps track of pending write directories to cluster files appropriately
• Windows exe version now comes in an easy to use installer package

For the few people who've discovered it, enjoy! I've been using it regularly for my own backups and it accurately backs up my greyhole pool.

I've had discussions with a friend about how a branch would be approached for linux, it sounds doable, but right now I don't have the interest since my primary concern is Windows where I use the application. If anyone would like to take a crack at it I can explain my ideas. I certainly wouldn't mind any additional python developers helping maintain / improve AnyBackup.

AnyBackup 0.6

Download at http://code.google.com/p/anybackup/downloads/list

AnyBackup Beta 2 -- Now with 100% more Python!

Following up on Beta 1 I've now finished Beta 2 of AnyBackup! What's different? Well, I ported the entire project to Python instead of Perl. Not because I think that there's any great functional benefit to one over the other, but rather because it's something I needed to pick up for work anyway. The main benefit to this release is that pretty much all the standing issues have been resolved. See below. I'll be looking into setting up a google code page in the near future for AnyBackup since it seems like a much cleaner approach than just updating my blog + mediafire account all the time.

Note: I work on this program in my spare time primarily to solve my own backup needs, I release it for others to use since I figure others may have similar backup needs that AnyBackup can fulfill. That said, this is beta software (and even if it weren't) I can make no guarantees that it will always 100% back up all your data and that no data loss will occur. There can be bugs that I don't know about / haven't hit. So basically, buyer beware, use at your own risk, I can't be held responsible for any issues that arise. If you do hit issues, don't hesitate to report them!

Changes:

• Now written in Python instead of Perl (see explanation above)
• File comparisons now made against file sizes (in KB) and directory paths in addition to file names
• See issues below for more details
• File object revised to store directory root and directory path separately
• This allowed me to get rid of some ugly regular expression hacks going on in the backup and indexing methods
• When backing up files or restoring files AnyBackup now creates a pending size change (the total of file size deletes and additions) and makes sure that any additional adds will fit, this keeps things from running into an issue where a drive will run out of space and copies fail
• Icon added to show whether a drive is currently connected or not (+ and green for connected, x and red for disconnected)
• Drive background color in the drive list now changes colors based on free space (>15% green, >10% yellow, <10% red)
• Drive free/total space added to drive list (in GB)

Known Issues:

• If a backup volume is full but there is new content to backup that prefers this full volume (due to the above backup logic based on parent folders), it will still choose this full volume as the destination and then throw an error that there isn't enough space on the destination volume.
• Follow up -- my solution for now is preferring a drive based on the above logic, but it creates a pending write total and checks to see if all the files we want to add will fit, if not it will instead grab the most free drive and back the files up there, in the next release I'll add a property file which will allow you to turn this sticky files feature on or off. (For now it's on -- in the next release you can turn it off and it will always place files on the most free drive at the time of choosing)
• For some reason when running a new backup, AnyBackup will often leave a few old files (that is, files that are no longer found on your content drives -- meaning they've been deleted / renamed / written to) -- this normally has no negative repercussions, but I'll figure out what's causing this eventually. 
• Some people are having issues launching the exe packaged version of AnyBackup, I'm not sure what's causing the issue, I've been using Cava Packager to create the exe's + installers and it runs perfectly on my test machines. I'll put up an archive of the raw perl later for those that have issues -- the downside to using the raw perl is that you'll need to have a valid perl install with all the packages I use.
• Since this is now in python being packaged with py2exe, I'm assuming this will no longer be an issue.
• File comparisons are made only against file names, so if you change/write to a file or have duplicate file names in multiple directories this can lead to inconsistencies such as only backing up one file instead of multiple or not backing up the updated version of a file
• I augmented this naive comparison, it now checks the file sizes and directory names in addition to file names. The only possible complication I see at this point is that the file size comparison is done in KB and not bytes, I'm not sure if this will cause issues (I'm thinking not) Feel free to let me know if this impacts small files with minute changes, though! (This is the only situation I really see it being an issue.) If I get reports I'll look into changing this.
• Minor issue with the most free drive logic, it needs to be updated to incorporate the new pendingwrite total otherwise it may not be grabbing the drive with the most free space, but rather than drive with the most free space before any operations have taken place (i.e. if we have 10gb pending to a drive which has 8gb more free than any other volume, it should no longer be considered the most free drive, but right now, it will be) 
• The CLI interface for the app, backupWrapper.py isn't guaranteed to work yet and all features are definitely not built out (primarily restore definitely does not work, I backup will not remove old files from backup volumes). What does work I haven't tested, so buyer beware when using it in this release. In the next release I will finish building out the features for it and also package it as an exe alongside the GUI interface.

 I'm only going to add one new screen shot to show off the (very small) UI changes to the main window. Despite the complete port of the app to Python, it's still using wxWidgets which is pretty much identical across languages.

Download:
(As before, this code is released under the GPL license!)

• AnyBackup Beta 2 Python Source 
• AnyBackup Beta 2 Win32

Update: Google Code project created! You can find it here.

AnyBackup Beta 1

I've been working on this side project for a little while now, it's called AnyBackup (as the title may have indicated). It's written in perl with wxWidgets as the GUI toolkit. Despite being written in perl, it is by no means platform independent, it's very much windows only for the time being. I built this program because there just doesn't seem to be a good solution out there for backing up multiple volumes to multiple volumes, especially to dynamic drives -- that is, drives that may or may not be connected and may or may not retain the same drive letter, a very real concern when your backup drives are being hooked up via USB! AnyBackup identifies your drives via a combination of the volume name (a volume must be named before being added, AnyBackup will refuse to add unnamed volumes) and the volume serial number. The bottom line, it's fairly stable and mostly works, I have not had it eat any data, and I've use it to keep my content drives in sync with my backup drives, I've even used it to restore some missing data. There are a few areas which need more intelligent handling as you can still get into a few situations with full drives that would require manual intervention to resolve.

 Features

• Backup any number / sized volumes to any number / sized volumes
• Restore missing files or backup new files / delete old files
• Search through files on any of your content and backup drives
• An easy to navigate gui interface for navigating the indexed content of your drives
• Backup volumes don't all have to be connected at once
• The idea here is that if you're connecting your backup drives via usb, you can connect them one at a time, AnyBackup will prompt you for the specific volume it needs (this is ideal if you're using a USB/eSATA dock to connect your backup drives)
• Allows you to specify a list of file extensions you wish to backup (so if you only wanted to backup mp3's, for instance, you could go to  Edit -> Edit Lists (or Ctrl+E) and select valid extensions from the drop down, and add "mp3" to the list. To index all files you could add ".*" to the list of valid extensions
• Allows you to specify regular expressions for files to avoid indexing
• Will semi-intelligently backup content and attempt to cluster content together
• By this I mean if you had a folder for your favorite band, say, The Beatles, and you have sub folders for all the albums (i.e. The White Album, Abbey Road, etc) which then contain audio files, when backing up each audio file it will look to see if a backup volume already contains its parent folder (i.e. The White Album) or its parent's parent (The Beatles) and if a backup volume already contains either, it will choose that backup volume as its destination.

Known Issues

• If a backup volume is full but there is new content to backup that prefers this full volume (due to the above backup logic based on parent folders), it will still choose this full volume as the destination and then throw an error that there isn't enough space on the destination volume.
• The workaround is obvious, move files off the full volume to a less full volume and then refresh both the backup volumes so AnyBackup will be aware of the changes.
• I'm leaning towards a solution that involves taking the parent folder and moving it to a different volume that has sufficient space and then copying the new content there too (obviously after performing a check that the new destination drive has enough space for the parent folder size + the new content size)
• For some reason when running a new backup, AnyBackup will often leave a few old files (that is, files that are no longer found on your content drives -- meaning they've been deleted / renamed / written to) -- this normally has no negative repercussions, but I'll figure out what's causing this eventually. 
• If you have multiple network drives mounted to the same share but at different levels (i.e. X: is mounted to \\server\share\dir1 and Z: is mounted to \\server\share\dir2) AnyBackup has no way of differentiating between the two as they'll have the same volume name and volume serial number, so it will refuse to add X when Z is already added in the above example. There are no plans to change this, my suggestion is to create separate shares so that, at the very least, the volume name will change between the two.
• Some people are having issues launching the exe packaged version of AnyBackup, I'm not sure what's causing the issue, I've been using Cava Packager to create the exe's + installers and it runs perfectly on my test machines. I'll put up an archive of the raw perl later for those that have issues -- the downside to using the raw perl is that you'll need to have a valid perl install with all the packages I use.
• AnyBackup will create folders on backup drives even if there are no valid files inside it -- so basically any folders present on the content drives will make it to one of the backup volumes, even if they have no valid file types, I actually prefer this, but I may add an option to toggle the behavior later.

Enough talk, pictures now!


Without further adieu, I give you the download information.

Download:
AnyBackup Beta 1 -- Win32 Packaged EXE 
AnyBackup Beta 1 -- Perl Source 
(Please note, when using the Perl Source above, you'll need to install several additional perl packages: Win32::API, Win32::DriveInfo, Cava::Packager -- this needs to be done via the Cava Packager program, and finally wxPerl -- this can take a while to install!)

Also note,  I have not had a chance to add the GPL license to the above source, but please note that ALL code posted here is released under the GPL license! The GPL license can be viewed  here for those that are interested!