[Yum-devel] [Fwd: Re: [Dulug] Letter by ESR]

[Michael_E_Brown at Dell.com]

I ran into this yesterday when writing my find_packages_for_my_machine
script. If you use anything resembling a regexp, performance is crap.
Use a provides search that just uses the builtin "like" function in
sqlite, and it runs fine. I'm talking 5 minutes vs 4 seconds for my
case.

Looking through the code, I thought it was funny that you repeated the
select() so many times rather than just caching (in a string or array)
and running regexp over the cache.
--
Michael 

> -----Original Message-----
> From: yum-devel-bounces at linux.duke.edu 
> [mailto:yum-devel-bounces at linux.duke.edu] On Behalf Of seth vidal
> Sent: Thursday, February 22, 2007 4:53 PM
> To: yum-devel at linux.duke.edu
> Subject: [Yum-devel] [Fwd: Re: [Dulug] Letter by ESR]
> 
> 
> this is REALLY worth checking out.
> 
> -sv
> 
> 
> -------- Forwarded Message --------
> From: Patrick Reynolds <reynolds at cs.duke.edu>
> To: seth vidal <skvidal at linux.duke.edu>
> Subject: Re: [Dulug] Letter by ESR
> Date: Thu, 22 Feb 2007 17:41:58 -0500 (EST)
> 
> I've found what's probably the main culprit for yum search 
> performance.
> You're calling select 15000+ times.  Don't.  Even with an 
> index, select is kind of slow.  How about:
>    executeSQL("select * from packages")
>    for each result
>      if regex.match(result.name) || regex.match(result.summary) ....
>        callback(db2package(result))
> Very rough pseudocode, but the idea should be clear.  Call 
> select once.
> 
> I wrote a version with substring (not regex) support, and it 
> runs even faster than apt.  It assumes the presence of the 
> SQLite caches.
> 
> paris:~{1009}$ time ./yum-search foo | wc -l
> 24
> 
> real    0m0.176s
> user    0m0.116s
> sys     0m0.064s
> 
> http://www.cs.duke.edu/~reynolds/projects/yum-search.tar.gz
> 
> --Patrick
> 
> _______________________________________________
> Yum-devel mailing list
> Yum-devel at linux.duke.edu
> https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
>