[Yum] Re: why doesn't yum cache anything?

[Farkas Levente]

Konstantin Ryabitsev wrote:
> On Sun, 02 Jan 2005 21:59:58 +0100, Farkas Levente <lfarkas at bppiac.hu> wrote:
> 
>>just try out a:
>>time yum -C list mtr
>>it has nothing to do with the network and the speed is almost the same!
> 
> 
> In my experience it isn't, but then again, doing "yum list \*mtr\*"
> only takes 8 seconds on my laptop. If it takes longer on other
> machines, considering mine is certainly not the fastest of the bunch,
> my suspicion is that the slowdown is not dependent on the processor.
> Memory and network would be my suspects.
> 
> 
>>it seems there are two place where yum is very slow:
>>- loading the local metadata (even if using the pickle files)
>>- the dependencie resolution
>>probably it needs to investigate further what is the real reason and how
>>can be solve. if yum be so slow for a long time, there will be someone
>>who create another package using a better/more clever local cache file
>>format and and may be reimplement it in a faster language (may be a
>>better/faster server side metadata format).
[skip]
> Feel free to encourage anyone to come up with a system better than yum
> that doesn't do things the apt way, i.e. doing everything outside of
> RPM and then just making a system call to the binary. I believe anyone
> attempting to re-implement yum will soon run into exactly the same
> problems as those faced by Seth et al. If someone does come up with an
> application that functions speedily and covers the same robust set of
> features that yum provides, then I'm sure Seth will have no problem
> stepping aside and letting the poor bastard who replaces him to take
> all the abuse. In fact, as someone for whom Seth is a direct
> supervisor, I'll downright encourage that development. :)
> 
> However, I believe that until RPM is replaced with an able enough
> alternative, things will not dramatically improve, however much you
> desire them to, and however many times you rewrite yum. Feel free to
> prove me incorrect.

first of all i do NOT like to blame anyone especially not seth since i 
like yum as the only usable tool. but that doesn't mean i can't tell my 
point of view and my critics. in short:
- yum is slow (rather slow) and it can (and i hope will) be faster!
imho it's better to not show any kind of exmaples, but
- even if nothing changed on the system (add/remove) like in case of 
list it takes a long time to get the result.
- even when only package name, version, release, epoc information will 
be enough (which is the most usualy case!!!! ie. yum 
install/erase/update/check-update/list) it load, parse and build 
python's object from all data which is about 95-99% is waste of 
resources (time and memory). this is just my estimation based on that 
the rpm filename is longer than the EVR string and only the pickle file 
is loaded:
-----------------------------------
# ls /mnt/download/mirror/fedora/3/i386/os/Fedora/RPMS/|wc -c
48537
# ls -l /var/cache/yum/os/
total 6252
drwxr-xr-x  2 root root    4096 Dec 20 18:48 headers/
drwxr-xr-x  2 root root    4096 Nov 12 10:06 packages/
-rw-r--r--  1 root root  816171 Nov  4 00:22 primary.xml.gz
-rw-r--r--  1 root root 5532228 Nov 12 10:06 
primary.xml.gz.ce7bdfd5a6066a4f18c498b693faa16597a0733c.pickle
-rw-r--r--  1 root root    1140 Nov  4 00:22 repomd.xml
-----------------------------------
and to show a much slower case when someone use more repos:
---------------------------------------------
# cat /proc/cpuinfo /proc/meminfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Pentium(R) 4 CPU 2.20GHz
stepping        : 4
cpu MHz         : 2222.143
cache size      : 512 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips        : 4374.52

MemTotal:      1035124 kB
MemFree:        108180 kB
Buffers:         98368 kB
Cached:         589540 kB
SwapCached:         12 kB
Active:         687336 kB
Inactive:       185700 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      1035124 kB
LowFree:        108180 kB
SwapTotal:     2008084 kB
SwapFree:      2006912 kB
Dirty:           63892 kB
Writeback:           0 kB
Mapped:         430524 kB
Slab:            26996 kB
Committed_AS:   831304 kB
PageTables:       6096 kB
VmallocTotal:  3088376 kB
VmallocUsed:      3672 kB
VmallocChunk:  3084504 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     4096 kB

# time yum -C list mtr
Setting up Repo:  dag
Setting up Repo:  freshrpms
Setting up Repo:  jpackage16-generic
Setting up Repo:  jpackage16-fc3
Setting up Repo:  updates
Setting up Repo:  Newrpms
Setting up Repo:  addons
Setting up Repo:  os
Setting up Repo:  pre-extras
Reading repository metadata in from local files
dag       : ################################################## 1531/1531
freshrpms : ################################################## 456/456
jpackage16: ################################################## 1277/1277
jpackage16: ################################################## 19/19
updates   : ################################################## 407/407
Newrpms   : ################################################## 374/374
addons    : ################################################## 10/10
os        : ################################################## 2622/2622
pre-extras: ################################################## 630/630
Excluding Packages in global exclude list
Finished
Installed Packages
mtr.i386                                 2:0.65-1.1.fc3.rf      installed

real    0m39.976s
user    0m11.271s
sys     0m1.030s
---------------------------------------------

-- 
   Levente                               "Si vis pacem para bellum!"