[Yum-devel] [UG] Failing mirror tests for ftp
Paul Nasrat
pnasrat at redhat.com
Thu Nov 2 09:22:31 UTC 2006
On Wed, 2006-11-01 at 09:16 -0700, Michael Stenner wrote:
> On Wed, Nov 01, 2006 at 03:17:29PM +0000, Paul Nasrat wrote:
> > Whilst debugging an installer issue we noticed some inconsistencies
> > between ftp/http mirror handling - if you do the following
>
> OK. I'll look into this. I have some ideas if people want to look in
> parallel. Obviously, the error presentation mechanisms of ftp and
> http are very different, so the code needed to fold them into the same
> errors is probably wonky. More than "oh, crap, there's a bug", this
> one grabs as "oh, crap, looks like I need more unit tests" :)
OK I've traced this down to the way urllib.py ftpwrapper is implemented:
http://mail.python.org/pipermail/python-dev/2004-July/046044.html
Copying urllib2.py and uncommenting the line fw.ftp_set_debuglevel(1)
*cmd* 'TYPE I'
*resp* '200 Switching to Binary mode.'
*cmd* 'TYPE A'
*resp* '200 Switching to ASCII mode.'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (127,0,0,1,243,76)'
*cmd* 'NLST foo'
*resp* '150 Here comes the directory listing.'
*resp* '226 Directory send OK.'
*cmd* 'TYPE I'
*resp* '200 Switching to Binary mode.'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (127,0,0,1,226,44)'
*cmd* 'RETR foo'
*resp* '550 Failed to open file.'
*cmd* 'TYPE A'
*resp* '200 Switching to ASCII mode.'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (127,0,0,1,95,251)'
*cmd* 'LIST foo'
*resp* '150 Here comes the directory listing.'
This is within urllib.py, if we patch urllib.py with
--- urllib.py.orig 2006-11-02 08:58:29.000000000 +0000
+++ urllib.py 2006-11-02 08:56:03.000000000 +0000
@@ -787,8 +787,8 @@
# Set transfer mode to ASCII!
self.ftp.voidcmd('TYPE A')
# Try a directory listing
- if file: cmd = 'LIST ' + file
- else: cmd = 'LIST'
+ if file: self.ftp.cwd(file)
+ cmd = 'LIST'
conn = self.ftp.ntransfercmd(cmd)
self.busy = 1
# Pass back both a suitably decorated object and a retrieval length
This takes the same assumption that it's a directory and will fail if
either it's a nonexistent entry or if it's a not readable directory and
you'll get the IOError.
Michael, you're probably more familiar with urllib development than me -
do you think this has any hope of going in, or should we use a custom
wrapper/handler - we already have a copy in byterange.py.
Paul
> -Michael
>
> > diff -u -u -r1.8 base_test_code.py
> > --- base_test_code.py 25 Feb 2005 00:00:36 -0000 1.8
> > +++ base_test_code.py 1 Nov 2006 15:12:12 -0000
> > @@ -19,7 +19,7 @@
> > ref_404 = base_http + 'nonexistent_file'
> > ref_403 = base_http + 'mirror/broken/'
> >
> > -base_mirror_url = base_http + 'mirror/'
> > +base_mirror_url = base_ftp + 'mirror/'
> > good_mirrors = ['m1', 'm2', 'm3']
> > mirror_files = ['test1.txt', 'test2.txt']
> > bad_mirrors = ['broken']
> >
> > and run the test_mirror.py we get the following (note runs clean with
> > default settings and also test_grabber is clean).
> >
> > python test_mirror.py
> > mirror.py tests
> > ActionTests
> > test the effects of a callback-returned action ... ok
> > test default action policy ... ok
> > test the effects of passed-in default_action ... ok
> > test the effects of method-level default_action ... ok
> > BadMirrorTests
> > test that a bad mirror raises URLGrabError ... FAIL
> > BasicTests
> > MirrorGroup.urlgrab ... ok
> > MirrorGroup.urlopen ... ok
> > MirrorGroup.urlread ... ok
> > CallbackTests
> > test that the callback can correctly re-raise the exception ... FAIL
> > test that MG executes the failure callback correctly ... ERROR
> > FailoverTests
> > test that a the MG fails over past a bad mirror ... FAIL
> > SubclassTests
> > MGRandomOrder.urlgrab ... ok
> > MGRandomStart.urlgrab ... ok
> >
> > ===============================================================================
> > ERROR: test that MG executes the failure callback correctly
> > -------------------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "test_mirror.py", line 119, in test_failure_callback
> > self.assertEquals(tricky_list[0][:25],
> > IndexError: list index out of range
> >
> > ===============================================================================
> > FAIL: test that a bad mirror raises URLGrabError
> > -------------------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "test_mirror.py", line 138, in test_simple_grab
> > self.assertRaises(URLGrabError, self.mg.urlgrab, url, filename)
> > File "/home/pauln/scm/duke/urlgrabber/test/munittest.py", line 389, in failUnlessRaises
> > raise self.failureException, excName
> > AssertionError: URLGrabError
> >
> > ===============================================================================
> > FAIL: test that the callback can correctly re-raise the exception
> > -------------------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "test_mirror.py", line 126, in test_callback_reraise
> > self.assertRaises(URLGrabError, self.mg.urlread, 'reference')
> > File "/home/pauln/scm/duke/urlgrabber/test/munittest.py", line 389, in failUnlessRaises
> > raise self.failureException, excName
> > AssertionError: URLGrabError
> >
> > ===============================================================================
> > FAIL: test that a the MG fails over past a bad mirror
> > -------------------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "test_mirror.py", line 161, in test_simple_grab
> > self.assertEqual(len(elist), 1)
> > File "/home/pauln/scm/duke/urlgrabber/test/munittest.py", line 396, in failUnlessEqual
> > raise self.failureException, \
> > AssertionError: 0 != 1
> >
> > -------------------------------------------------------------------------------
> > Ran 13 tests in 0.312s
> >
> > FAILED (failures=3, errors=1)
> >
> >
> > _______________________________________________
> > Yum-devel mailing list
> > Yum-devel at linux.duke.edu
> > https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
>
More information about the Yum-devel
mailing list