[Yum-devel] Test cases status report

[Florian Festi]

Hi!

As James Bowes already asked when we start fixing the problems found in the 
test cases I start here with a small[*] status report:

Problems:
=========

There are 4 test cases with updates from multilib (i386 and x86_64) to just 
one arch (i386 or x86_64 only). Yum keeps the old version of the missing 
arch. This is probably the right thing to do although it is very likely to 
run into file conflicts. In my own tree I made these test shut up. But I'd 
like to do something to handle that situation more graciously later on. May 
be just provide a better error message.

There are several test cases that fail because yum doesn't try to update 
pkgs that matches conflicts.  The other way round (updating the pkg 
containing the conflict) works well - even for conflicts matching provides 
instead of pkg names. This problem should be fixed by a new 
Depsolve.processConflict() implementation which first steps have already 
been discussed.

I just added some test cases to my tree that check obsolete chains. This 
means that there is an obsolete for an installed package and also an 
obsolete for this obsoleting pkg. Right now yum just does one step a time. 
Which isn't that bad as long as you don't run into obsoletes ping pong or 
obsoletes circles. I guess this is a bit bigger thing to address...


Tests needed:
=============

There are some topics left where I don't have test cases yet but I suspect 
more problems or at least deserve their own test cases:

1. Non newest updates: If there are several updates available for one pkg 
interesting things can happen. Especially if there are requirements for the 
middle version. I suspect that there are more complicated scenarios needed 
to actually produce an error. Something like requiring a middle version and 
  having to update further later on comes to my mind.

2. Compat arch scenarios: Fun with i?86. There is bz259021 about installing 
i586 kernels on i686 systems. Even if this case turns out to be of minor 
interest having some test cases can't be wrong.

3. Install only pkgs: Don't know if we hit much trouble in that area but as 
the kernel is an essential component having a few test cases can't hurt.

Conclusion:
===========

For the missing test cases help is appreciated. Writing the tests is really 
easy now. Have a look at what's already there. There's no need to come up 
with an complete 3 or 4 dozen cases test suite. Every bit helps.

Even more welcome are ideas on test cases or scenarios we also should have a 
look at. It is highly unlikely that we can simply start adding combinations 
of the one we already have as we'd end up with literally hundred or even 
thousands of test cases. So looking at the right ones is the way to go.

Florian

[*] Sorry! Didn't get as small as I thought.