[Yum-devel] [PATCH 1/3] add 'max_connections' option

[James Antill]

On Fri, 2011-12-23 at 08:05 -0500, Zdenek Pavlas wrote:
> > > 3) Look at max_connections code in MirrorGroup._process_kwargs().
> > >    If you set max_connections=1 and first mirror has a limit of 2,
> > >    then we'd fail.
> > 
> >  No ... if the mirror has a max_connection of N and the configuration
> > has a max_connection of M ... then the max_connection for that mirror
> > is min(N, M).
> 
> Tried to think as a user.. We limit per-host #conn, because that's
> what server admins expect and require.  But IMO it makes little 
> sense to limit per-repo #conn.

 You are saying we shouldn't go over the max-connections the server
admin. gave us to a single host? ... if so, I agree :).

> When I run a 100+ package download on a memory-starved system
> all I need is to limit the total number of child processes
> forked.  
> 
> Using max_connections as a hard global limit should not be very hard to
> implement, as it's tracked already, and I'd just delay starting
> of a new download when the global limit is maxed out.

 If you want to make max_connections _just_ global, and then that be a
hard upper limit on the number of download connections ... that's fine
by me.
 My main concern was that if a repo. config. for max-connections was set
to N and we make N+1 connections for that repo. ... life is not going to
be pleasant.

> This still leaves open the selection of mirrors.  I like the
> idea of selecting the 'top N' mirrors, but we need to guess
> a suitable value somehow, eg global limit divided by number
> of repos we download from..

 This is likely to get "complicated" long term. ... where people will
want to say things like "I don't mind if you do 666 connections, but
don't go past these X,Y,Z mirrors (top N according to MirrorManager, or
whatever) unless you have to."

 The "obvious" first step is "don't leave the first mirror" for each
MirrorManager repo. and "unto max-connections" for everything else. The
next thing people might want is "if I have private mirrors, then use as
many private mirrors as you want". Could be a simple per. reop. config.
too.
 Or a default to just "don't leave the first mirror" ... given that
people will want some wacky things.