[Yum-devel] [UG] openers and keepalive

Ryan Tomayko rtomayko at naeblis.cx
Tue Mar 30 01:00:08 UTC 2004


On Mon, 2004-03-29 at 13:13, Michael Stenner wrote:
> 2) opener-caching broken
> 
>    The opener-caching behavior seems to not quite be working right
>    yet, so I disabled it for now.  It's a very simple and obvious
>    modification.  So, for now, a new opener is built for EVERY fetch.
> 
>    I only explored this enough to find that the problem was related to
>    the caching.  I found it illuminating to dump the contents of
>    opener.handle_errors and opener.handlers before each fetch.  That
>    clearly showed that somehow, support for the right stuff found
>    itself missing.  This only showed up in the test suites where LOTS
>    of stuff was happening.  All my simple tests worked.
> 
>    This is one of the downsides of using global structures, I guess.
>    We should also keep the threadsafety of such things in mind.
> 
>    Ryan, if you pursue this, let me know.  I'll do the same.  That
>    way, we won't duplicate effort.  For now, I'm going to keep working
>    on the progress meter stuff.

Sorry, I must have missed something there. I've got this one. I will try
to look into it tonight. 

> 3) custom openers
>    
>    In talking with a potential user on irc, I was reminded that it
>    would be nice to allow custom openers/handlers.  I figured we
>    should address this at the same time since it's basically the same
>    code.  Here are the reasons I can think of why someone might want
>    to customize things:
> 
>     * introduce support for another protocol or option
>     * REMOVE support for a protocol or option (the user I chatted with
>       wants to REMOVE file:// support for security reasons)
> 
>    Now, the biggest problem I can think of is that there doesn't seem
>    to be a tidy way to ADD handlers to an existing opener, and there
>    is almost certainly no tidy way to REMOVE handlers from one.

:) Yes, very messy if we try to modify cached openers.

>    The simplest thing we could do would be to provide a "opener"
>    option.  Then, one could specify a custom opener for the grabber,
>    or per-fetch, as usual.  If not provided, we could fall back our
>    grabber building/caching policy.  The biggest problem with this is
>    that the caller would be completely responsible for building it
>    with support for all the necessary stuff: byteranges, keepalive,
>    proxies.  Therefore, the presence options like "proxy" and
>    "keepalive" would have no effect if "opener" is used.  Of these,
>    the most annoying is the proxy thing.

I like it. I can add an opener option per your spec tonight while I'm
looking at the broken cache stuff. IMO, this is a good starting point
and doesn't back us into any corners. If we get any clever ideas on how
to make customizing handlers a bit easier, we should be able to roll it
in on top of this approach without much issue.

- Ryan





More information about the Yum-devel mailing list