[Yum-devel] Building out the test suite

David Malcolm dmalcolm at redhat.com
Wed Feb 16 00:12:25 UTC 2011


Something that's been very successful for the development of Python
itself is having a policy that all patches need to be accompanied by a
test that verifies the correct working of the new code.  This is one of
the things that is done on patch review.  See e.g.:
  http://docs.python.org/devguide/patch.html#preparation
(Historically Python didn't have much of a test suite, but having this
policy has meant that as things have evolved, the code being touched has
had greatly enhanced test coverage).

I'm mostly just lurking here, but I keep watching you review each
other's patches, and every time I see changes that add new API
entrypoints, or optimize things, I have an inner voice tell me "but
where's the test case?"

In particular, if yum's python API is meant to be the preferred API over
the rpm python bindings for accessing packaging information, it ought to
have a test suite that exercises that API, verifying various properties
of it, demonstrating the expected usage (and hopefully ensuring that the
stack is sane when e.g. python or rpm-python change from under you).

Would it be helpful to institute a policy that patches that add new API
entrypoints should contain test cases, and other patches should at least
have a statement that all tests continue to pass?  (Ideally all patches
ought to add new test cases: presumably there's some particular state
that has some pre-existing behavior and you want a different behavior:
set that up, and assert the new behavior happens.  Obviously driving the
API into the given state can be hard, but there are ways to make that
easier).

(I appreciate that I'm mostly just a lurker here)

Sorry, had to get that rant out.  Hope it's helpful.
Dave



More information about the Yum-devel mailing list