[Yum-devel] yum on an olpc machine (slooooooooooow)

[seth vidal]

On Sun, 2006-12-17 at 16:29 -0500, James Bowes wrote:
> seth vidal wrote:
> > any other ideas of obvious things to look at and/or interesting
> > optimization ideas?
> 
> A few ideas (apologies if they've popped up on the list before):
> 
> * Have a separate thread for each repo (or even md file) when
>    downloading and parsing.

adding threading has a speedup advantage of download but on a low cpu/
low io machine it is just going to kick the crap out of it.


> * Create an optional diff format for the metadata, published with each
>    update to the repository, so clients only have to download and parse
>    changes to the repo.

I was actually curious if there was an sqlite-diff mechanism. All the
content we're pushing in the db is text, if there was an easy way to
produce a diff of them. However, this seems like future optimization
that really only narrows the size of the download. 


> Maybe there is a storage format that would be faster than sqlite for 
> yum's purposes? I have no idea what it would be, however.

It's not just about faster. It's faster, portable and with
relatively-sane bindings. 

The major reason it needs to be some kind of random-access database is
for memory size constraints.

we've already done the "everything in memory" game and that doesn't
scale at all for low-mem systems or for very very very large
repositories.

Would it be possible or reasonable to increase startup-time speed for
yum by importing modules only when they're needed and not at the top of
the script? I know it is definitely considered bad style code-wise but a
yum --version on the olpc box is taking 7s to run. That seems like a
long time to import a module, instantiate a few objects and return one
variable value.

-sv