[Yum-devel] [Fwd: Why yum do not continue partial metadata downloads?]

Hedayat Vatankhah hedayat at grad.com
Tue Aug 25 19:02:36 UTC 2009


/*Seth Vidal <skvidal at fedoraproject.org>*/ wrote on ‫سه‌شنبه ۲۵ اوت ۰۹، 
۲۰:۰۱:۵۷‬:
>
>
> On Tue, 25 Aug 2009, Hedayat Vatnakhah wrote:
>
>>
>> Hi all,
>> Currently yum continues partial downloads when downloading packages, 
>> but not for repository meta-data (e.g. primary.sqlite.bz2). It would 
>> be really nice if this feature is added to all yum files downloads 
>> rather than just RPM packages. It will improve the speed of yum on 
>> such situations, specially for people with slow internet connection 
>> (or some slow/busy mirrors). Let me bring an example:
>> Today, I decided to install a small plymouth theme package. So, I 
>> issued a 'yum install plymouth-theme-sp*' command. Yum decided to 
>> update its meta-data for Fedora updates repository and started to get 
>> its primary.sqlite file (which was 2.5 MB and is relatively small 
>> compared to the Fedora repository's pribary.sqlite, where the problem 
>> is much bigger). At the first try, it downloaded about 1/3 of the 
>> file and then timed out. It started from scratch with another mirror 
>> to download the file, and it timed out again after downloading 2.1MB. 
>> It started again, and downloaded almost all of the file (around 99% I 
>> think) and timed out again! (I'm using a dial-up internet connection 
>> at home). Well, that was really frustrating, so I decided to go with 
>> 'yum --disablerepo=updates install plymouth-theme-sp*' and 
>> fortunately no dependency problems happened. Also, the package I 
>> wanted was just around 67KB...
>> You can guess how much trouble I had to download Fedora repository's 
>> metadata (for this reason I added metadata_expire=-1 for Fedora 
>> repository as I don't like to get those files again.).
>> It is also one of the good reasons I have for not using PackageKit 
>> when a really fast Internet connection is not available, since it 
>> don't tell the user how the process of downloading repository 
>> metadata is going (it might get 'timed out' for ever!).
>>
>> Something a bit unrelated: I found some emails in this mailing list's 
>> archives (in 2007) about patches for supporting delta metadata files. 
>> This is a topic which I'm really interested in, and I'm thinking 
>> about it for awhile (But I don't have any experience in yum 
>> development now, and also not much free time). But I want to know 
>> about their status. Are they available in Yum code repository? What 
>> is remaining for them to become finished and usable? Maybe I could 
>> put some time on this issue soon.
>>
>
> As I told you on the bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=515744
>
> Yum had not continued metadata downloads in the past b/c the metadata 
> was not assured to be uniquely named.
>
> createrepo can now create unique names but it is not the default mode
> mainly to make sure we don't end up with problems with other depsolvers
> and tools using the repodata and making bad assumptions about filenames.
>
> It's probably a  fair idea to make the unique filenames the default 
> though it tends to lead to consternation when looking up the files by 
> a human.
Sorry for duplication, the original email were written before that bug 
report. Yes, I think making this default and adding the resume feature 
for downloading metadata files is a reasonable action. IMHO it was a 
design bug that yum do/did not know if the remote database is newer than 
the current one.
Anyway, some tools (at least smart) resume downloading of metadata files 
already, and it would be nice if yum do so as soon as possible.
About file names being not friendly to humans, it is certainly much less 
undesirable than the current situation (and completely unimportant for 
end users); but is it really needed for the unique file names to be so 
complex? Why a simple version based naming, or date based naming doesn't 
work?

>
> Deltametadata got started then stalled out. Right now I believe it 
> would require an entire reimplementation to get it working.
OK, I'll hopefully look at it in future.

Thanks,
Hedayat

>
> -sv
>
>
>
>
> _______________________________________________
> Yum-devel mailing list
> Yum-devel at lists.baseurl.org
> http://lists.baseurl.org/mailman/listinfo/yum-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.baseurl.org/pipermail/yum-devel/attachments/20090825/fc6b4dee/attachment.htm>


More information about the Yum-devel mailing list