ddar is a free de-duplicating archiver for Unix. Save space, bandwidth and time by storing duplicate regions of data only once. Use ddar to:
- Back up local data to a remote server, each time saving space, upload bandwidth and time by only transferring data not already present on the remote server. This is what cloud backup services like Tarsnap allow you to do already. Now you can use your own storage and internal bandwidth to achieve the same thing.
- Back up remote data to a local disk, downloading and storing only changed data each time (the inverse of the above).
- Back up local data to a local external disk, each time saving time and space by only writing data that was not present on the disk already. I have six gzipped tarballs of around 50 GiB each of my laptop stored on my external disk, but all together they are only using 64 GiB of storage.
- Efficiently store any data that has redundancy. ddar will exploit redundancy across different files stored at any time.
Found this: http://www.synctus.com/blog/2011/03/a-tarsnap-alike-for-private-storage yesterday. Very Nice
Details: http://www.synctus.com/ddar/
Discussion: http://news.ycombinator.com/item?id=2327130
Comparison with rdiff-backup: http://www.synctus.com/ddar/rdiff-backup.html (including a, very, short example python program to extract from the archive directly)
Update: original web server off line. Archive of original http://web.archive.org/web/20131209161307/http://www.synctus.com/ddar/
Converted to wiki at https://github.com/basak/ddar/wiki
It was inspired by tarsnap and designed to be a similar, but local and faster, backup system. There is no encryption and no need for it. Use an encrypted filesystem if you want it.
Thing I expect to use it for:
- archiving disc images (virtual machines and dd copies of small laptop)
- saving big files on vfat removable storage.
- backing up :-)
