[Yum-devel] metadata parser in C

Russell Harrison rtlm10 at gmail.com
Wed May 10 13:34:29 UTC 2006


I will say that the biggest complaint I get from people about yum is speed.
I know some people on older machines that are reporting initial parsing
times of over an hour.

I'm assuming (haven't looked at the code) that since his changes require db
changes his parsing algorithm is different from the current python
implementation.  If using C is a concern can some of the same improvements
be made in python?  Even if they're not as dramatic any speed increase for
yum is a big win.

On 5/10/06, seth vidal <skvidal at linux.duke.edu> wrote:
>
> On Wed, 2006-05-10 at 15:45 +0300, Tambet Ingo wrote:
> > email message attachment, "Forwarded message - [Yum] metadata parser
> > in C"
> > > -------- Forwarded Message --------
> > > From: Tambet Ingo <tambet at ximian.com>
> > > Reply-To: Yellowdog Updater, Modified <yum at lists.dulug.duke.edu>
> > > To: Yellowdog Updater, Modified <yum at lists.dulug.duke.edu>
> > > Subject: [Yum] metadata parser in C
> > > Date: Wed, 10 May 2006 14:03:01 +0300
> > >
> > > Hey,
> > >
> > > Attached is a yum metadata parser written in C. It should produce
> > > identical results with the sqlitecache implementation in yum-2.6.1 but
> > > should be quite a bit faster. Here are the numbers parsing FC5 core
> > > metadata (2207 packages):
> > >
>
> Hi Tambet,
> Thanks for fwding this onto this list - it's just a better place for
> the discussion and a number of folks who are developers aren't on
> yum-list b/c it's mostly for user questions.
>
> Also thanks for working on this, it looks cool. I'm going to give you
> some initial thoughts and since I've just jumped out of the shower some
> extended thoughts, too. So take all of it inclusively and we'll see what
> comes out, okay?
>
> Initial thoughts:
> Yay Speed!!!
> Ugh! C.
> Hey, look, someone from ximian contributing, cool.
>
> I like the initial speed results you've posted - those look great but
> all of the parsing being in C concerns me some if only due to
> maintenance issues.  It's not necessarily a show-stopper just a
> show-complicator. :)
>
>
> Post-shower thoughts:
> - If we have the speed up granted by your code then we can rework the
> sqlite db format (especially for filelists) to vastly simplify and speed
> up our all of the 'whatProvides' calls that have to look up files. This
> makes depsolving faster.
> - the metadata parsing code is, for obvious reasons, some of the least
> changed code in yum. So modifying it wouldn't be a massively frequent
> concern
> - this makes all of the metadata importing code obsolete (to some
> extent) which does get rid of a bunch of code that has needed some
> quailty time with an editor. In short it is a parallel thought to a goal
> for the development series.
>
>
> So that's the majority of what I've been thinking of right now. What do
> other folks think?
>
> -sv
>
>
>
> _______________________________________________
> Yum-devel mailing list
> Yum-devel at linux.duke.edu
> https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.baseurl.org/pipermail/yum-devel/attachments/20060510/9fa39288/attachment.htm 


More information about the Yum-devel mailing list