[Tornado-users] Tornado /home nearly full: CORRECTION

[Johan Raber]

Dear users,

I have been made aware of a problem in my suggested approach to moving
files from the home directory: The "du" command will likely give different
results depending on the filesystem type you move your data to and is
therefore not a reliable measure of success for the data transfer.

An alternative, more rigorous, approach is to do two consecutive rsyncs à la:
$ rsync -a /from/dir/ /destination/dir/
$ rsync -a /from/dir/ /destination/dir/
which will, the second time it is run, check to see that the files have been
transferred and make sure the directories are sync'ed.

A systematic (healthy) paranoia check can then be performed as follows:
$ cd /from/dir

$ find . -type f -print0|xargs -0 stat --printf "%s %n\n" | sort >~/ORIG

$ cd /destination/dir/

$ find . -type f -print0|xargs -0 stat --printf "%s %n\n" | sort >~/COPY

$ diff ~/ORIG ~/COPY

There should be no output from diff if all is well.

After that you may check the md5 checksum of a set of random files (as many as your
paranoia level suggest weighted by how many files you transferred).

My apologies if you feel mislead by my previous suggestions. They were not
wrong per se apart from the "du" type of "success measure". The above
approach though is a more rigorous methodology, and therefore preferrable.

If you have reason to believe some files have been lost. Open a case and
your data will be restored from the backups.

Best regards,
Johan Raber

-- 
Johan Raber, PhD
Systems expert -- distributed computing
National Supercomputer Center, Linköping University
SE-581 83 LINKÖPING, SWEDEN