[Yum-devel] [Fwd: Why yum do not continue partial metadata downloads?]

[Seth Vidal]

On Tue, 25 Aug 2009, Hedayat Vatnakhah wrote:

> 
> Hi all,
> Currently yum continues partial downloads when downloading packages, but 
> not for repository meta-data (e.g. primary.sqlite.bz2). It would be 
> really nice if this feature is added to all yum files downloads rather 
> than just RPM packages. It will improve the speed of yum on such 
> situations, specially for people with slow internet connection (or some 
> slow/busy mirrors). Let me bring an example:
> Today, I decided to install a small plymouth theme package. So, I issued 
> a 'yum install plymouth-theme-sp*' command. Yum decided to update its 
> meta-data for Fedora updates repository and started to get its 
> primary.sqlite file (which was 2.5 MB and is relatively small compared 
> to the Fedora repository's pribary.sqlite, where the problem is much 
> bigger). At the first try, it downloaded about 1/3 of the file and then 
> timed out. It started from scratch with another mirror to download the 
> file, and it timed out again after downloading 2.1MB. It started again, 
> and downloaded almost all of the file (around 99% I think) and timed out 
> again! (I'm using a dial-up internet connection at home). Well, that was 
> really frustrating, so I decided to go with 'yum --disablerepo=updates 
> install plymouth-theme-sp*' and fortunately no dependency problems 
> happened. Also, the package I wanted was just around 67KB...
> You can guess how much trouble I had to download Fedora repository's 
> metadata (for this reason I added metadata_expire=-1 for Fedora 
> repository as I don't like to get those files again.).
> It is also one of the good reasons I have for not using PackageKit when 
> a really fast Internet connection is not available, since it don't tell 
> the user how the process of downloading repository metadata is going (it 
> might get 'timed out' for ever!).
> 
> Something a bit unrelated: I found some emails in this mailing list's 
> archives (in 2007) about patches for supporting delta metadata files. 
> This is a topic which I'm really interested in, and I'm thinking about 
> it for awhile (But I don't have any experience in yum development now, 
> and also not much free time). But I want to know about their status. Are 
> they available in Yum code repository? What is remaining for them to 
> become finished and usable? Maybe I could put some time on this issue soon.
>

As I told you on the bug:
https://bugzilla.redhat.com/show_bug.cgi?id=515744

Yum had not continued metadata downloads in the past b/c the metadata was 
not assured to be uniquely named.

createrepo can now create unique names but it is not the default mode
mainly to make sure we don't end up with problems with other depsolvers
and tools using the repodata and making bad assumptions about filenames.

It's probably a  fair idea to make the unique filenames the default though 
it tends to lead to consternation when looking up the files by a human.

Deltametadata got started then stalled out. Right now I believe it would 
require an entire reimplementation to get it working.

-sv