[Yum] metadata parser in C

[Tambet Ingo]

Hey,

Attached is a yum metadata parser written in C. It should produce
identical results with the sqlitecache implementation in yum-2.6.1 but
should be quite a bit faster. Here are the numbers parsing FC5 core
metadata (2207 packages):

file      this  old
--------------------
primary   0.75  8.54
filelists 2.55  17.0
other     3.87  19.96

Total     7.17  45.5

I ran the tests 3 times and took the average and deleted *.sqlite after
each run.

To test it, unpack the attached tarball and build it with standard:
python setup.py build
sudo python setup.py install

To make yum use it, rename the original parser and copy the file from
the tarball to it's place:
mv /usr/lib/python2.4/site-pakages/yum/sqlitecache.py /usr/lib/python2.4/site-pakages/yum/sqlitecache.py.old
cp ./sqlitecache.py /usr/lib/python2.4/site-pakages/yum/sqlitecache.py

That's it, the next time yum should use the new and faster parser.

Some notes:

I updated the dbversion because I modified the SQL schema slightly: When
deleting a package from 'packages' table, there are now SQL triggers to
delete related rows from other tables (files, 'prco', filelists,
changelog).

It doesn't work for regular users: The current implementation uses in
memory database for that case, which is not possible with this
implementation. There's no way to return the sqlite db handle from C to
python so the parser currently closes the db and returns the file name
and python part re-opens it. I'm thinking about adding per-user sqlite
caches somewhere in users' home directories for that. Something like
~/.yum/$reponame/$md_filename.xml.gz.sqlite.

The yum logging isn't used. All the output is printed to stdout and
stderr. It's quite easy to fix but ...

Thoughts? Ideas? Comments?

Thanks,
Tambet

-------------- next part --------------
A non-text attachment was scrubbed...
Name: yum-parser.tar.gz
Type: application/x-compressed-tar
Size: 10621 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum/attachments/20060510/a9efc94e/attachment-0001.bin