SV: [Gimle-users] Behov av job klasserpåGimle

[Per.Unden]

Dear Markus, Klaus, Magnus and others,
I think such a proposal is too extreme, then we will loose too much in
the use of Gimle and I would guess that such a proposal could cut down
the actual usage of Gimle to about 50% of the theoretical maximum. At
extra busy times it would restrain each of our groups rather much since
we would not be able to temporarily use up free slots from another group
who e.g. happen to be away on a conference. In e.g. FoUp we are too few
to guarantee even usage throughout the year. 
Although Markus' idea can serve as one of the parameters to keep in mind
for a fair share system, i.e. when all groups are busy running, then  a
good queueing system should result in something towards that, but I
don't believe in an absolute division of the nodes. I think, like with
the sort of system that I proposed, it will give a fair distribution
according to the number of active users, not by group in FoU. And I
don't think that is too bad, sometimes we have only 1-2 persons active,
other times 4-5, and I don't see a problem that FoUo gets more when FoUp
only have 1-2 users and vice versa, when we are 5, then FoUo or FoUrc
gets less (more like the "normal"). 
Finally, on a philosophical point, I think in an institute and a Res
Dept we have the computing facilities together since we think this is
more cost effective than each group buying their own facilities. So I
think we should continue to cooperate on how to give a fair access for
all users while using the overall facility relatively efficiently.
Best Regards
Per

On Sat, 2009-02-21 at 01:33 +0100, Meier Markus wrote:
> Dear all (this time in English for Robinson)
>  
> many of you wish to have an updated queue sytem on Gimle to secure
> both a fair assess and
> optimal usage of the resource. OK let's try to find one. The problem
> is that the needs of the different
> groups differ substantially. For illustration 2 extremes:
> 1) for FoUp it is important to have quick assess and many short
> restarting jobs with a small number of nodes
> 2) for FoUo it is important to have maybe just one job with many nodes
> (say 31) continously running
> That is what you have seen during the last weeks. As FoUo:s
> application is very heavy (it takes
> quite a long time even with a high number of nodes), we try to keep
> basically one job running
> all the time. In the development and analysis of this job about 5
> people are involved.
>  
> To quarantee the continous work for all of us I see only one solution
> for a queue system. Each group will
> get there own queue with  a maximum number of nodes according to the
> cost share of Gimle:
> FoUrc 13, FoUp 32, FoUo 32 , FoUl 6, FoUa 1
> Every group can decide how to organize the work within their own queue
> (either just one job with many
> nodes or several smaller jobs but never more than the node limit
> together).
> In addition there should be a risk queue that all groups can
> use together in addition to their "own" nodes.
> That would allow to use empty nodes without blocking anybody. Maybe we
> need a few rules also for the risk queue like maximum cpu (e.g. 3
> hours) and a limited number of jobs per group (not per user).
> However there should not be too many rules that it is still possible
> to understand the queue system.
>  
> Does this sounds fair?
>  
> Best wishes
> Markus
>  
>  
>  
> 
> ______________________________________________________________________
> Från: gimle-users-bounces at lists.nsc.liu.se genom Wyser Klaus
> Skickat: to 2009-02-19 13:18
> Till: Undén Per; gimle-users at nsc.liu.se
> Ämne: RE: [Gimle-users] Behov av job klasser påGimle
> 
> 
> Hej Per och alla andra---
> 
> Ditt förslag med olika köer är värt att följa up, men jag skulle gärna
> vilja se en riskkö utöver de som du förslår. Ditt förslag har nämligen
> 2 nackdelar: 1) varje användare kan skicka jobb till alla köer och
> helt enkelt fylla upp maskinen med olika jobb, och inget kan stoppa
> dem. 2) En användare kan inte utföra fler än säg 2 normala jobb även
> om maskinen står tom.
> 
> Med en riskkö kan man lösa det smidigt, eftersom jobb som går på
> riskkö stannar om nån annan behöver resursen. De startas igen (från
> början) när det åter blir tid över för dem. Man kan skicka obegränsad
> med riskjobb, och därmed utnyttja maskinen bättre när den inte används
> för annat.
> 
> Sist men inte minst: riskjobb är perfekt för att lägga upp långa
> körningar över helgen eller semester där användare inte kan nås och
> ombes ta bort sina långa tunga jobb.
> 
> /Klaus
> 
> 
> > -----Original Message-----
> > From: gimle-users-bounces at lists.nsc.liu.se [mailto:gimle-users-
> > bounces at lists.nsc.liu.se] On Behalf Of Per.Unden
> > Sent: Thursday, February 19, 2009 12:30 PM
> > To: gimle-users at nsc.liu.se
> > Subject: [Gimle-users] Behov av job klasser påGimle
> >
> > Hej allihop,
> > Jag skulle vilja se att det införs ett system med olika köer och
> > begränsningar på Gimle för att det är ett återkommande problem för
> våra
> > användare och vad jag hört även på Rossby.
> > Jag tror att vi får ta nackdelen av ett viss underutnyttjande mot
> att
> > våra forskare inte kan få igenom sina korta utvecklingsjobb eller
> vänta
> > orimligt länge för varje cykel i experimenten. Det leder till sämre
> > produktivitet och otrivsel i allmänhet och arbetstiden är faktiskt
> > dyrare än alla CPU tider tillsammans.
> > Detaljerna får experter utforma och beror på vad som är praktiskt
> > möjligt, men något som:
> >
> > Class  / priority
> >
> > Ultra-long    > 2 days   < 48 nodes    on special requests /
> weekends ?
> > Low prio     < 2 days   < 48 nodes     max 1 per user
> > Normal      < 1 day    < 24 nodes    max 2 per user
> > High        < 1 hour   < 48 nodes    max 1 per user
> > Very short  < 10 mins  < 24 nodes    max 2 per user
> >
> > Detta är ett exempel och man bör arbeta ut vad gränserna bör vara
> > utifrån användarerfarenhet.
> >
> > Mvh
> > Per Unden
> >
> >
> > _______________________________________________
> > Gimle-users mailing list
> > Gimle-users at lists.nsc.liu.se
> > http://www.nsc.liu.se/mailman/listinfo/gimle-users
> 
> _______________________________________________
> Gimle-users mailing list
> Gimle-users at lists.nsc.liu.se
> http://www.nsc.liu.se/mailman/listinfo/gimle-users
> 
> 
> 
>