[Yum] The Future of urlgrabber

[Michael Stenner]

CC-ing to the yum list because I know icon and seth (at least) are
interested in this issue.

On Sun, Oct 12, 2003 at 11:14:11PM -0400, Jeremy Katz wrote:
> On Sat, 2003-10-11 at 15:16, Michael Stenner wrote:
> > Also, there was some discussion of how urlgrabber should be used by
> > other programs (yum, for example).  Should urlgrabber have its own
> > rpm (deb, whatever) and simply be called as a dependency?  Or should
> > it be slurped into a codebase occasionally.  The former requires much
> > more careful backward compatibility and therefore places much tighter
> > constraints on urlgrabber's evolution.  Because urlgrabber is still
> > young and changing fairly rapidly, I lean toward the latter.
> 
> I'd actually argue the other way -- if it's external, *really* make it
> external.  Follow Havoc's suggestions regarding parallel installability
> if needed.  
> 
> Saying just to suck the code in where wanted means that if external
> projects don't pull in versions with bug fixes, you'll still get the
> bugs and be dependent on a third party to do an update.  

Yes.  I agree with you there.

> Also, it's the same as arguing that you should just suck copies of
> every library into every application or at the minimum that you
> should always do static linking.

Slow down there.  Every library into every application?  Saying you
should write _a_ program in python is not the same as saying you
should write EVERY program in python.  By no means would a decision to
write urlgrabber for "slurping" imply that all libraries should be
written that way.

 [ background: Havoc's suggestions are at
   http://ometer.com/parallel.html
   and in this context would amount to naming things urlgrabber2,
   urlgrabber3, etc. on disk when behavior changes ]

We now have three alternatives on the table:
  1) simple external
     + very tidy
     - major constraints on the code (preserve backward compat, etc)
     + apps get automatic bugfixes with urlgrabber upgrade
  2) parallel external
     - not so tidy
     + fewer constraints (change major num when BC breaks)
     + apps get automatic bugfixes with urlgrabber upgrade
  3) slurped internal
     + very tidy
     + no constraints (apps slurp whenevery they want/can)
     - no automatic bugfixes - must slurp new version

I haven't really ruled out (or settled on) any of these yet.  I guess
the design process will have some impact on my opinion.

> My $0.02, for what it's worth.

It's definitely worth something... probably even more that $0.02 :)

> Also, I'd be interested at looking over any new design stuff as I am
> looking at moving anaconda over to using urlgrabber for the next
> release.  I'm tired of dealing with urllib and love the idea of actually
> being able to use a consistent module for that stuff ;)

Yes, I understand.  Maybe I'll set up a little mailing list for
discussion [if it's more than just you and me :)].

					-Michael
-- 
  Michael Stenner                       Office Phone: 919-660-2513
  Duke University, Dept. of Physics       mstenner at phy.duke.edu
  Box 90305, Durham N.C. 27708-0305