[Triolith-users] Disabling SSH compute node access

[Pär Lindfors]

Dear Triolith users,

On Tuesday 2016-06-07 we will disable SSH access to compute nodes on 
Triolith. To access compute nodes we provide a SSH/RSH compatible 
command "jobsh" as a replacement for ssh. You can of course also use 
native Slurm commands like srun.

This change will affect all jobs that are started from 2016-06-07, even 
if they are submitted before that time. jobsh is already available on 
Triolith so you can start using it right away.

This will NOT affect access to Trioliths login nodes. Both SSH and 
ThinLinc will continue working the same as now.

Most common affected use cases, and typical solution:

* ssh from a login node to a compute node where a job is running
- Solution: Use jobsh instead of ssh

* Use ssh in batch scripts to start commands on other nodes in the job
- Solution1: Use jobsh instead of ssh
- Solution2: Modify job script to use srun instead of ssh

* Use of MPI implementations that use ssh to launch processes on other 
nodes, typically when running commercial software that includes its own 
MPI implementation
- Solution1: Pass options to mpirun/mpiexec to enable Slurm support
- Solution2: Pass options to mpirun/mpiexec to use jobsh instead of ssh

Reason for this change:

Jobs running on Triolith are only allowed to use CPUs and memory 
allocated to the job. Slurm enforces this by using Linux kernel cgroups. 
However SSH logins run outside of the cgroups and can use resources that 
are not allocated to a job. This also results in incorrect accounting of 
resources used by the job.
Since the reconfiguration done on 2016-03-01 SSH logins also sees 
another view of certain file systems than what a batch jobs does, which 
breaks certain applications.
We do not believe this will cause any major issues as SSH access to 
compute nodes is already disabled on all other clusters at NSC.

Please do not hesitate to contact support at nsc.liu.se for help or if you 
have questions.

Kind regards,
Pär Lindfors, NSC