SV: [Gimle-users] Behov av job klasserpåGimle

[Colin.Jones]

Hi All

Without wanting to get into the details of this, I do agree with Per.
There should be no need to lock certain numbers of nodes to various
users. This makes for inefficiency. Rather a sensible queuing system and
respect for each others needs should be enough.

Colin


On Mon, 2009-02-23 at 08:37 +0100, Per.Unden wrote:
> Dear Markus, Klaus, Magnus and others,
> I think such a proposal is too extreme, then we will loose too much in
> the use of Gimle and I would guess that such a proposal could cut down
> the actual usage of Gimle to about 50% of the theoretical maximum. At
> extra busy times it would restrain each of our groups rather much since
> we would not be able to temporarily use up free slots from another group
> who e.g. happen to be away on a conference. In e.g. FoUp we are too few
> to guarantee even usage throughout the year. 
> Although Markus' idea can serve as one of the parameters to keep in mind
> for a fair share system, i.e. when all groups are busy running, then  a
> good queueing system should result in something towards that, but I
> don't believe in an absolute division of the nodes. I think, like with
> the sort of system that I proposed, it will give a fair distribution
> according to the number of active users, not by group in FoU. And I
> don't think that is too bad, sometimes we have only 1-2 persons active,
> other times 4-5, and I don't see a problem that FoUo gets more when FoUp
> only have 1-2 users and vice versa, when we are 5, then FoUo or FoUrc
> gets less (more like the "normal"). 
> Finally, on a philosophical point, I think in an institute and a Res
> Dept we have the computing facilities together since we think this is
> more cost effective than each group buying their own facilities. So I
> think we should continue to cooperate on how to give a fair access for
> all users while using the overall facility relatively efficiently.
> Best Regards
> Per
> 
> On Sat, 2009-02-21 at 01:33 +0100, Meier Markus wrote:
> > Dear all (this time in English for Robinson)
> >  
> > many of you wish to have an updated queue sytem on Gimle to secure
> > both a fair assess and
> > optimal usage of the resource. OK let's try to find one. The problem
> > is that the needs of the different
> > groups differ substantially. For illustration 2 extremes:
> > 1) for FoUp it is important to have quick assess and many short
> > restarting jobs with a small number of nodes
> > 2) for FoUo it is important to have maybe just one job with many nodes
> > (say 31) continously running
> > That is what you have seen during the last weeks. As FoUo:s
> > application is very heavy (it takes
> > quite a long time even with a high number of nodes), we try to keep
> > basically one job running
> > all the time. In the development and analysis of this job about 5
> > people are involved.
> >  
> > To quarantee the continous work for all of us I see only one solution
> > for a queue system. Each group will
> > get there own queue with  a maximum number of nodes according to the
> > cost share of Gimle:
> > FoUrc 13, FoUp 32, FoUo 32 , FoUl 6, FoUa 1
> > Every group can decide how to organize the work within their own queue
> > (either just one job with many
> > nodes or several smaller jobs but never more than the node limit
> > together).
> > In addition there should be a risk queue that all groups can
> > use together in addition to their "own" nodes.
> > That would allow to use empty nodes without blocking anybody. Maybe we
> > need a few rules also for the risk queue like maximum cpu (e.g. 3
> > hours) and a limited number of jobs per group (not per user).
> > However there should not be too many rules that it is still possible
> > to understand the queue system.
> >  
> > Does this sounds fair?
> >  
> > Best wishes
> > Markus
> >  
> >  
> >  
> > 
> > ______________________________________________________________________
> > Från: gimle-users-bounces at lists.nsc.liu.se genom Wyser Klaus
> > Skickat: to 2009-02-19 13:18
> > Till: Undén Per; gimle-users at nsc.liu.se
> > Ämne: RE: [Gimle-users] Behov av job klasser påGimle
> > 
> > 
> > Hej Per och alla andra---
> > 
> > Ditt förslag med olika köer är värt att följa up, men jag skulle gärna
> > vilja se en riskkö utöver de som du förslår. Ditt förslag har nämligen
> > 2 nackdelar: 1) varje användare kan skicka jobb till alla köer och
> > helt enkelt fylla upp maskinen med olika jobb, och inget kan stoppa
> > dem. 2) En användare kan inte utföra fler än säg 2 normala jobb även
> > om maskinen står tom.
> > 
> > Med en riskkö kan man lösa det smidigt, eftersom jobb som går på
> > riskkö stannar om nån annan behöver resursen. De startas igen (från
> > början) när det åter blir tid över för dem. Man kan skicka obegränsad
> > med riskjobb, och därmed utnyttja maskinen bättre när den inte används
> > för annat.
> > 
> > Sist men inte minst: riskjobb är perfekt för att lägga upp långa
> > körningar över helgen eller semester där användare inte kan nås och
> > ombes ta bort sina långa tunga jobb.
> > 
> > /Klaus
> > 
> > 
> > > -----Original Message-----
> > > From: gimle-users-bounces at lists.nsc.liu.se [mailto:gimle-users-
> > > bounces at lists.nsc.liu.se] On Behalf Of Per.Unden
> > > Sent: Thursday, February 19, 2009 12:30 PM
> > > To: gimle-users at nsc.liu.se
> > > Subject: [Gimle-users] Behov av job klasser påGimle
> > >
> > > Hej allihop,
> > > Jag skulle vilja se att det införs ett system med olika köer och
> > > begränsningar på Gimle för att det är ett återkommande problem för
> > våra
> > > användare och vad jag hört även på Rossby.
> > > Jag tror att vi får ta nackdelen av ett viss underutnyttjande mot
> > att
> > > våra forskare inte kan få igenom sina korta utvecklingsjobb eller
> > vänta
> > > orimligt länge för varje cykel i experimenten. Det leder till sämre
> > > produktivitet och otrivsel i allmänhet och arbetstiden är faktiskt
> > > dyrare än alla CPU tider tillsammans.
> > > Detaljerna får experter utforma och beror på vad som är praktiskt
> > > möjligt, men något som:
> > >
> > > Class  / priority
> > >
> > > Ultra-long    > 2 days   < 48 nodes    on special requests /
> > weekends ?
> > > Low prio     < 2 days   < 48 nodes     max 1 per user
> > > Normal      < 1 day    < 24 nodes    max 2 per user
> > > High        < 1 hour   < 48 nodes    max 1 per user
> > > Very short  < 10 mins  < 24 nodes    max 2 per user
> > >
> > > Detta är ett exempel och man bör arbeta ut vad gränserna bör vara
> > > utifrån användarerfarenhet.
> > >
> > > Mvh
> > > Per Unden
> > >
> > >
> > > _______________________________________________
> > > Gimle-users mailing list
> > > Gimle-users at lists.nsc.liu.se
> > > http://www.nsc.liu.se/mailman/listinfo/gimle-users
> > 
> > _______________________________________________
> > Gimle-users mailing list
> > Gimle-users at lists.nsc.liu.se
> > http://www.nsc.liu.se/mailman/listinfo/gimle-users
> > 
> > 
> > 
> > 
> _______________________________________________
> Gimle-users mailing list
> Gimle-users at nsc.liu.se
> http://www.nsc.liu.se/mailman/listinfo/gimle-users
>