http://dragoman.org/tym Regards Tomas
No good deed goes unpunished ;) Very nicely coded script, but it's a bit dense. I'm good at bash and can survive in rsync, but could you provide a description of what it actually does so I don't have to spend a long time analysing the code? Does it keep multiple versions like the name implies? Will it survive directory names with embedded blanks (in the parameters)? At first glance, this looks like a problem, but I may have missed something. Why trap so many signals? If something goes wrong, do I have to kill -9 to stop it? Does one of those keep it running when you logoff? I don't see a nohup in the script. I've seen some methods that use hard links to make subsequent backups smaller. I haven't quite figured out how that works, but it doesn't look like you use it. What does your script do if the destination runs out of space or isn't mounted? TIA Joe> http://dragoman.org/tym > > Regards > Tomas > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >
These are comments to several previous posting. Regards Tomas 1. Limitations tym is a wrap: the hard work is done by rsync, with whatever limitations it might have, in particular for "non-atomic". Each backup is a new directory; "--backup" is not used as no renaming is required. 2. Time machine implemented inside rsync As one can observe from tym, it should be straightforward to implement a time machines inside rsync: rsync ... --tm-source --tm-dest [--tm-machine] [--tm-log] The correct parameters should be automatically included. Finer aspect such as atomicity could be better considered. If it is appropriate is a different matter. 3. Cron It can run properly run from cron as it is demonized. 4. Remote machine - As long as tym is concerned, it is client-side. - The communication is with ssh and rsync. - No filesystem mounting is required. - Unattended execution requires ~./ssh keys to avoid password prompting from ssh and rsync. 5. Finding the latest backup In function MakeDest, locally (or remote with ssh), the line ; else ls $DestDir | tail -1 6. Creating new directory Created by bash (not rsync) in function MakeDest, locally (or remote with ssh), the line ; then mkdir $DestDir The tree below by rsync. 7. Automatic space recovery I will put this in the wish list. - Calculate the needed space. - Check the maximum allocated space (it could be less than total available). - Delete as necessary the oldest version with appropriate heuristics. tym is "pull". 8. rsync parameters See the function Sync, the concerned parameters are: rsync -azhq --delete --partial For the first run. "--link-dest" is not present: the variable "$LinkDest" is empty. "--hard-links" is not used and there is for and against reasons. From man rsync: "... finding multiply-linked files is expensive." "Without this option, hard-linked files in the transfer are treated as though they were separate files." 9. Wish list - More comments in the code - Improve man page for tym - Autoamtic space recovery - tymkill: command for soft kill with SIGTERM. - tymold: command to delete oldest backups
Correction, tym is "push": rsync does all the remote manipulation.
The worse case might look like this:
- Run program foo-run.com
- Log data foo-log.com
- Source data foo-source1.com to foo-sourceN.com
- Destination data foo-destination.com
For this, one might use a syntax similar to scp syntax. For example running the
command from foo-run.com:
scp foo1 at foo-source1:/bar1 ... fooN at foo-sourceN.com:/barN fooD at
foo-destination.com:/barD
It would have to be extended for the logs. Depending on the use case, one should
see how much should be generalize.
At present in tym, there are two possibilities:
- One machine: run, log, source and destination
- Two machines:
1. Run, log and source
2. Destination
Parameters in tym:
readonly SourceList=/foo/bar-source # local source
files/directories
readonly DestDir=/foo/bar-dest # backup directory, local or
remote
readonly Machine=foo at example.com # defaults to local machine
Regards
Tomas
ARG! http://lmgtfy.com/?q=bloom+filter K S Braunsdorf wrote:> Just google.com or ddg.gg "bloom filter" and you'd be more happy. > > --ksbLassi Tuura wrote:> Hey, > >> I still have no clue what a bloom filter is?? ;-) Cluesticks anyone? > > www.google.com/search?q=bloom+filter > > L.