[Vagnekman-users] Configuration change of the login node

[Per Lundqvist]

On Fri, 19 Nov 2010, Per Lundqvist wrote:

> On Thu, 18 Nov 2010, lars malinowsky wrote:
> 
> > Hello,
> > 
> > as several transfers (rsync/scp) in and out of ekman
> > seem to be hit the 10 minute cpu-time limit, the ekman
> > compute-node k33n41.pdc.kth.se can be used for these
> > transfers - you should be able to log on to it and use
> > it in the same manner as you log on to ekman.pdc.kth.se.
> > 
> > Note that this compute node does not have a 10Gbit
> > network to the outside world, only a 1Gbit network.
> > 
> > Consider this a stop-gap solution, for
> > transfers only, until advised to do differently.
> > 
> > It's unknown whether the ffv-utility works out
> > of the box or not on k33n41.pdc.kth.se.
> 
> ffv now works from k33n41.pdc.kth.se

Forgot to mention: there's a way to circumvent this limit on ffv jobs
by using the option "--no-start-early" (at: "submit", "resume", "auth"
and "afslog").

The default for ffv is to always execute the job in the background
directly by the user, but this will inherit the limits set on the
shell. With --no-start-early the job is instead started from cron. The
side effect of --no-start-early is that the actual file transfer might
be delayed up to 1 minute, but that shouldn't be an issue for large
filesets.

/Per Lundqvist

> 
> In this situation if an ffv job would hit the cpu-time limit the
> underlying rsync session would be killed, which would cause the ffv
> job to be paused. When a job is paused you are notified by mail about
> this. To resume a paused job you have to log in again and type:
> 
>    ffv resume <JOBID>
> 
> I could modify ffv such that it automatically tries to resume the
> rsync file transfer if it is killed for some reason - i.e. not pausing
> the job and not needing any manual intervention. But this is not how
> it works today. It might not be such a good idea to change this
> behaviour either, since most often if a process is killed it is done
> for a reason - it is then very unexpected for the rsync transfer to
> pop up automatically again.
> 
> /Per Lundqvist
> 
> 

-- 
Per Lundqvist

National Supercomputer Centre
Linköping University, Sweden

http://www.nsc.liu.se