[Yum] Re: Questions about headers ...

Robert G. Brown rgb at phy.duke.edu
Wed Sep 24 17:50:31 UTC 2003


On Wed, 24 Sep 2003, C.Lee Taylor wrote:

>     But that is the case,
> du -ch freshrpms/header* os/header* updates/header*
> 16K     freshrpms/header.info
> 1.9M    freshrpms/headers
> 88K     os/header.info
> 20K     os/headers
> 12K     updates/header.info
> 1.3M    updates/headers
> 3.3M    total
> 
>     Which does not seem like a lot, but currently takes a good half hour 
> to transfer to a new yum install ...

Ouch!  What are you on, a 56K line?  Let's see, 56 Kbps = 7 KBps or 1 MB
every 150 seconds or so (call it three minutes).  The above totals less
than 10 MB, and so yes, 30 minutes.

I don't see any way around it, though.  You can compress the headers and
send them that way, but you've got to get the headers from whereever
they live onto your system(s).

> >The best way is to create a local repository (usually by mirroring an
> >existing repository).  This costs you a whole lot of bandwidth -- once
> >-- to set up the repository, and then you use LOCAL bandwidth in your
> >LAN to do all the updates and whatever.
> >
>     I do something like that, basicly I backup my /var/cache/yum 
> directory and take it with me to do and install ... looking at setting 
> up and local repo for my computers in my network, but have just not got 
> there yet ...

Try the draft HOWTO -- I'd be very interested in how it works for you as
a set of instructions.  Some people have noted on the list that it is
pretty easy to just convert your cache into a repository, but I think
there are advantages to using rsync (or in a pinch, wget) to set up an
actual mirror.  For one thing rsync automagically will manage
compression for you, and the header data may be quite compressible.
Actual rpm's aren't generally horribly compressible as they contain data
that is already compressed, for the most part.

> >I have a DSL link into my home, and DSL is slow as molasses (384 Kbps
> >inbound, on a good day).
> >
>     ADSL kas just been introduced in South Africa, and only to selected 
> aera's where the local telco believe that can charge large fee's and 
> make their money back quickly ... as for the average company with enough 
> money to get fix lines, they are normally 64Kbps lines, else everybody 
> else is like to have modems, so your 384Kbps seems like heavan here ... 
> sorry to get carried away ...

You have my sympathy.  At 64 kbps and slower, it is a LOT faster to just
put everything on a laptop and carry the laptop around with a crossover
cable.  Or even drive from town to town, in some cases.  A full linux
distribution is several GB in size, which several THOUSAND minutes, at
1440 minutes/day.  It took close to a day to make my original mirror
even over DSL.

> >  I have a bunch of hosts at home to yum install
> >or yum update.  Disk, on the other hand, is absurdly cheap and
> >plentiful.  So I mirror the repository(s) at Duke via rsync onto my home
> >server.  The first time this was immensely painful -- it took something
> >like a day to complete the mirror.  
> >
>     This is what I will end up doing, just got to find the time to 
> understand everything that I will be using, so when something stops 
> working, I at least know where to look ... as for a day to complete, I 
> watch an installation in Botswana take a day just to download the 
> headers ... not kewl ...

Actually, you can do a lot of the experimentation with a very small
repository -- just set up a directory (available via some kind of URL)
and stick a few homemade RPM's into it, run yum-arch on the directory,
and stick a server line into yum.conf for it.  It should "just work"
(i.e. yum list should show the rpm's), and you can play until it works
and you're comfortable.

A full mirror repository is then simply a matter of scale.  If you want
to be able to do e.g. kickstart installs, you'll need to mirror a
suitable RH image, is all.

>     See the ligth in this  and as I said, will get this going in the 
> near furture for my computers on the network, but could procedure be 
> looked at which one could almost packaged /var/cache/yum and put in 
> place on a target computer, this would save quite a bit of bandwidth 
> with a new installation ... also would saved backup disk space (CDR) 
> with the option I aks about a little while ago ( clean oldpackages ) ...

Sure, and that you can do now, but instead of packaging it just scp it.
Or (best) rsync it, compressed.  I'm still trying to envision just where
your bottlenecks lie -- between you and Europe/US?  Between you (server)
and your LAN clients?  Between you (linux consultant) and client LANs in
many cities and towns?

There are going to be different answers for all of these.

To solve the Europe/US bottleneck, rsync a mirror of the best
repository(s) you can find and keep it up to date, and use it locally
over a high BW connection.

To solve the bottleneck inside your LAN, use a better network.  By LAN I
mean on the same network as your server(s) above, with no slow links in
between.  These days switches are cheap, wire is cheap, punchblocks are
cheap, wireless is cheap, and even wireless is plenty fast to run yum.

To solve the bottleneck between client LANs, well, that IS a problem.
Sneakernet (in a modern revision of carrying a repository with you on
calls on a laptop or portable disk or a set of CD-RW's) is often going
to beat waiting two days or more to send a copy over phone lines that
slow.  Hell, the MAIL might beat that.  rsync might be enough to manage
at least slow incremental changes once the client LAN servers have a
basic repository image, though, augmented by the occassional sneakernet
run (or a mailing of CDR's) when a change comes along that is too big to
propagate by phone.

Hope this helps, but it probably won't as much as finding a faster
network would...:-)

   rgb

> 
> Thanks for the quick responce ...
> Mailed
> Lee
> 
> 
> _______________________________________________
> Yum mailing list
> Yum at lists.dulug.duke.edu
> https://lists.dulug.duke.edu/mailman/listinfo/yum
> 

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu






More information about the Yum mailing list