[Bi-users] Final reminder: Retiring filesystems rossby20, smhid13 and fouo6

[Fredrik Nyström]

Dear Accumulus storage users,

final deadline for copying data from rossby20, smhid13 and fouo6 is 
Friday September 30.

rossby20, smhid13 and fouo6 is still mounted readonly on Bi login nodes.

(rossby20 should be done, ok to unmount from Michael Sahlin today.)

If you need assistance copying data feel free to contact 
NSC <smhi-support at nsc.liu.se>.


Kind regards / Fredrik

On 2022-06-16 15:30, Fredrik Nyström wrote:
> Dear Accumulus storage users,
> 
> 
> this is a reminder about rossby20, smhid13 and fouo6 being retired on 
> June 30 2022. 
> 
> 
> 
> Today we have also updated rsync from 3.1.2 to 3.2.4 on Bi login nodes.
> 
> - Default checksum method is xxhash instead of md5, higher throughput 
>   and less cpu usage. Up to ~3 times faster depending on bandwidth to 
>   storage, <30% gain on Bi due to network and to a certain degree I/O 
>   bottlenecks.
> 
> - Default compression method is zstd instead of zlib. You should not 
>   use compression (-z or --compress) when both source and destination 
>   are local paths, but doing so by mistake and getting zstd will not 
>   incur any performance penalty. zlib on the other hand will cap out 
>   at less than 33MB/s and consume lots of cpu.
>   Note that when using rsync over ssh, both server and client need to 
>   have support for zstd, otherwise zlib will be used and this will most 
>   likely be slower than without compression. You can check for 
>   available hash and compression methods with "rsync --version".
> 
> 
> 
> When you restart rsync after downtime on Tuseday June 12, please do not 
> use rsync with the "-c" option unless you know the implications. 
> 
> Example: Interrupted after copying half of a 100TB file tree with 
> compression ratio 1.5 on Lustre servers. Restarting rsync with "-c", 
> rsync will then verify checksums for all files it has already copied, 
> on both source and destination before it resumes copying files. In this 
> case it means reading 150TB of data over network before resuming.
> 
> 
> Kind regards / Fredrik
> 
> On 2022-04-07 13:30, Fredrik Nyström wrote:
>> Dear Accumulus storage users,
>>
>> on Thursday June 30 2022 we will retire the following filesystems:
>>
>> /nobackup/rossby20 /nobackup/smhid13 /nobackup/fouo6
>>
>> Please copy any data that you wish to keep before this date.
>>
>> We would also appreciate if you from now on try to avoid filling up 
>> these filesystems with more precious data. Temporary data that does
>> not need to be copied is ok. Same goes for rossby25 and smhid18 which
>> will be retired in October 2022.
>>
>> At this time there is plenty of space available on rossby27 and
>> smhid20.
>>
>>
>> If you need assistance copying data from rossby20, smhid13 and fouo6,
>>  feel free to contact NSC <smhi-support at nsc.liu.se>.
>>
>>
>> Kind Regards,
> 

-- 
Fredrik Nyström, National Supercomputer Centre
freny at nsc.liu.se