[Yum-devel] clean up sooner

[David Lutterkort]

On Mon, 2006-02-20 at 10:48 -0500, seth vidal wrote:
> > I don't think there's anything wrong with using a sqlite DB, but it
> > might be simpler to just use flat files: before actually starting the
> > transaction, write out what needs to be done into a file in a format
> > that is convenient to parse back in. 
> 
> xml. No point in maintaining N file formats just for fun, if we don't
> have to.

It's a new file format either way. Do you think that XML is needed
because the structure of the file will require nontrivial parsing or
that these files might be interesting to other programs ? Also, XML is
not suited for writing out data where the stream could be interrupted at
any moment; if yum crashes half way through writing, the XML is not
well-formed.

> Which means you have to read through the whole file and match them back
> and forth. This is why I thought a little db would be nicer if only b/c
> we could either:
> 1. delete the finished entry
> 2. update the entry to mark its state as complete.
> 3. query for incomplete entries at startup time.

The only difference between your and my approach is when you do the work
you describe above - during the transaction or when yum starts. With
files, you can avoid parsing the file altogether if you maintain a
separate lockfile during the transaction and delete it afterwards.

> > The point here is that recovering a failed transaction is rare, and can
> > be made a little more expensive, and just appending to a file makes
> > writing these logs very cheap.
> 
> sadly I think it is not as rare as we might think.

What percentage of yum transactions do you think fail ? In my
experience, it's certainly less than 10% of the uses.

> I'm curious if writing out an sqlite db is that much less cheap for this
> little an amount of data.

Only profiling can tell us that; though I would assume that there is
some nontrivial startup cost for any sqlite DB.

> > If you use one file per transaction, you could even keep them around as
> > an audit log of what yum did on a given system, and people could simply
> > truncate the logs with an appropriate tmpwatch job. That kinda thing is
> > a little more hairy if you use a DB.
> 
> You can do the same with a sqlite db per transaction, too.

But you have to write special-purpose code to let the user indicate how
much of an audit log to keep - with tmpwatch it's a small configfile,
that gives the user a lot of flexibility in describing how much to keep.

David