[Yum-devel] Speed improvements to sqlite filelists
Menno Smits
menno-yum at freshfoo.com
Thu Feb 3 22:08:07 UTC 2005
Hi Gijs,
I'm having a little trouble with this version and I don't have time to
debug right now. I'm getting this output from a clean checkout:
....
Reading repository metadata in from local files
Primary sqlite cache needs updating, reading original meta data
Added 2622 new packages, deleted 0 old in 11.01 seconds
Primary sqlite cache needs updating, reading original meta data
Added 1694 new packages, deleted 0 old in 7.46 seconds
Primary sqlite cache needs updating, reading original meta data
Added 136 new packages, deleted 0 old in 0.44 seconds
Primary sqlite cache needs updating, reading original meta data
Added 649 new packages, deleted 0 old in 4.49 seconds
Primary sqlite cache needs updating, reading original meta data
Added 477 new packages, deleted 0 old in 1.33 seconds
Primary sqlite cache needs updating, reading original meta data
Added 723 new packages, deleted 0 old in 2.41 seconds
Excluding Packages in global exclude list
Finished
Traceback (most recent call last):
File "/usr/bin/yum-dev", line 10, in ?
yummain.main(sys.argv[1:])
File "/home/msmits/projects/yum/yum-sqlite/trunk/yummain.py", line
72, in main
result, resultmsgs = do()
File "/home/msmits/projects/yum/yum-sqlite/trunk/cli.py", line 635,
in doCommands
return self.provides()
File "/home/msmits/projects/yum/yum-sqlite/trunk/cli.py", line 1233,
in provides
matching = self.searchPackageProvides(args,
callback=self.matchcallback)
File "/home/msmits/projects/yum/yum-sqlite/trunk/yum/__init__.py",
line 909, in searchPackageProvides
for filetype in po.returnFileTypes():
File "/home/msmits/projects/yum/yum-sqlite/trunk/yum/sqlitesack.py",
line 42, in returnFileTypes
return YumAvailablePackage.returnFileTypes(self)
File
"/home/msmits/projects/yum/yum-sqlite/trunk/repomd/packageObject.py",
line 299, in returnFileTypes
return self.files.keys()
AttributeError: 'NoneType' object has no attribute 'keys'
These new performance figures sound great. I'm looking forward to trying
this out.
Regards,
Menno
Gijs Hollestelle wrote:
> Hi all,
>
> Today I found some spare time to work on the yum sqlite patches. I've
> implemented the following now for the sqlite filelists caches:
>
> Instead creating a database entry per file I now create a database
> entry per directory in a package and I put the files and filetypes in
> that directory into a single TEXT database field. This reduces cache
> generation times by a factor +/- 4 and diskusage by a factor 2.5
>
> See my subversion repository:
> https://svn.win.tue.nl/viewcvs/yum-sqlite
>
> Greets,
> Gijs
>
> Notes:
> - Currently I'm using |file1|file2|file3| to put them in, but I guess
> this should be changed to something like pickle's dumps, but I havent
> tested that yet.
> - Filetypes can be stored more compactly than |file|file|file|dir|
> maybe something like fffd for this example, because a type can be
> encoded as a single character.
> _______________________________________________
> Yum-devel mailing list
> Yum-devel at linux.duke.edu
> https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
>
> Scanned by the NetBox from NetBox Blue
> (http://netboxblue.com/)
>
Scanned by the NetBox from NetBox Blue
(http://netboxblue.com/)
More information about the Yum-devel
mailing list