[Yum-devel] clean up sooner

[seth vidal]

On Tue, 2006-02-21 at 09:12 -0800, David Lutterkort wrote:

> If you write plain text files where each line corresponds to one record
> as I described in my initial mail, it's pretty easy to deal with teh
> incomplete file. At the very least, you can always read all but the last
> record. With XML, the parser will start complaining at some point; it
> probably depends on the parser how much of the incomplete file you get
> to see before the parser gives up.

the celementree iterparser should be good about it until it explodes at
the end in a fiery crash :)



> Absolutely. I am not saying that you couldn't do all these things with
> sqlite.

okie doke. I thought that's what you meant, sorry.


> Yes, but I never really looked at how fast it is compared to the rest of
> what the program is doing. Do you have some profiling data ?

it does very well for simple non-data-intensive inserts on medium-sizes
of data and the inserts get even better with the newer/better python
modules but I've not done any real benchmarking.

I guess what I've been thinking is that an 800 pkg upgrade is at least
1600 operations. And going through each of those in a text file is
booooooooring and error-prone. 

Whereas just selecting the last N not marked in a certain way is easy
enough.

> Good point; I had thought you wanted one DB for all transactions.
nah - more like what you described - 1 file per.


> I really just wanted to throw the flat file biz out there as another
> idea - after all, I am not doing the actual work to implement this;
> Dennis will know much better which will be the best way to do all this.

quite frankly where it's written to is more or less one method attached
to TsInfo and the rpm callback most likely.

so no matter how it's written it shouldn't be a huge headache to change
or play with.

-sv