[Bi-users] File copying dos and don'ts

[Kent Engström]

We have recently seen bad performance on login node bi3 due to file
copying activity there, and as we have sent out emails about the
retiring of rossby25 and smhid18 (as well as the earlier rossby20,
smhid13 and fouo6 filesystems) it may be beneficial to remind you about
some file copying dos and don'ts:

*) Use a compute node (allocate the whole node using -N1) when copying
large amounts of data between filesystems. Use "interactive" if that
suits you best, or "sbatch" if that makes more sense.

*) If you cannot use a compute node (for example due to the fileystems
not being present there anymore), please use bi1 or bi2 for the copying
instead of the default bi3 login node, to spread the load. If you use
Thinlinc, screen, tmux etc you can get to another login node using
commands like "ssh l1" and "ssh l2".

*) Use "rsync" instead of "cp" or similar to make it easier to continue
an interrupted transfer without a lot of duplicated work.

*) For rsync: Do *NOT* use the "-z" (long form "--compress") when
copying between two local filesystems. It does not make sense in that
situation.

*) For rsync: Do *NOT* use the "-c" (long form "--checksum"). If you
think you need it, read the part in the man page about it. If you still
think you need it, talk to smhi-support at nsc.liu.se before you use it.

*) For rsync: Do *NOT* use the "--append-verify" flag. Talk to
smhi-support at nsc.liu.se if you think you need that.

*) If you need help with copying, please contact
smhi-support at nsc.liu.se.


Best Regards,
-- 
Kent Engström, National Supercomputer Centre
kent at nsc.liu.se, +46 13 28 4444