[yum-commits] Changes to 'refs/tags/urlgrabber-3-9-0'
skvidal at osuosl.org
skvidal at osuosl.org
Fri Jul 31 20:21:41 UTC 2009
Tag 'urlgrabber-3-9-0' created by Seth Vidal <skvidal at fedoraproject.org> at 2009-07-31 20:19 +0000
urlgrabber 3.9.0
Changes since the dawn of time:
James Antill (2):
Import progress patches from Fedora.
Import fix for ftp ports in old urilib code (probably worthless now, but meh)
Seth Vidal (16):
modify urlgraberror so it has a url attribute and includes the url in all
make it use *args instead of silly if statements
implement PyCurlFileObject. This makes the default and forklifts all the code to
get the content-length/size for ftp pkgs too
implement throttle/bandwidth controls in pycurl
- add range support
- make simple/most proxies work
- make makefile work again without using cvs
- clean up some unused files
when I first started this I hacked something into URLGrabberFileObject - this reverts that hack
add git2cl
- make basic posts work
- make readlines() work for mirrorlists in yum (which probably shouldn't be using it anyway)
update changelog
add a few more things to be cleaned out
changelog + release date touchup
mstenner (113):
Initial revision
created test_urlgrabber.py and started moving tests into it
These are the changes from when ryan was working in a separate repo.
Edited the cvs repo to restructure a little bit. All module python
Added mirror code and associated test code.
[TODO] added not about restructuring import style
Updated MANIFEST.in and README. Created 'maint' directory containing
Added first attempt at reget support. Created test suite for http.
Fixed a typo in mirror.py. Added reget docs and reformatted module-level
Updated TODO.
Added .cvsignore to urlgrabber/ directory
Fixed a stupid typo in the splituser fix (from 5 minutes ago)
added keepalive tests; fixed test_bad_reget_type
added "failure_callback" options to both MirrorGroup and URLGrabber. These
added callback test to test_mirror.py. Restructuring the test code a
making tests a little nicer
updated TODO
converted test_byterange.py to use base_test_code
improved mirror.py docs, added MGRandomStart (which is equivalent to yum's
Put version, date, url, and author information into __init__.py and
updated TODO
Added Id: strings to distributed python files, made grabber.py use the
updated TODO after alpha-huddle
Changed test base_url according to new web page. Added prefix
Made the control of MirrorGroup failover action WAY cooler and more flexible.
Added a little more documentation.
Made tests prettier (again... I do like pretty tests) and created the test
By popular demand, I broke out the unittest coolness into a separate
Added file and ftp tests for reget.
In grabber.py, fixed a bug with urlread when no limit is specified.
Made tests a little cleaner and added "none" test, which is a
OK, I THINK I've made keepalive thread-friendly. The interface is
Rearranged keepalive a little to make it nicer. Also made it deal
added a simple threading test
Updated ChangeLog, TODO. Played with the batchgrabber stuff a little.
Tidied up some of the callback code.
Changed the wait time for the dropped-connection test. The apache
Fixed documentation error in the range docs. Temporarily disabled the
Added "make test" target.
Modified "make ChangeLog" target so that it doesn't print the times. If
updated TODO
Fixed a few bugs related to python 2.3.
fixed minor typo in opener option docs
Replace LICENSE with the LGPL. Added maint/license-notice, which
Tidied up the grabber tests so we're not always skipping those ftp reget
Committing new progress meter code. This new version includes an emulation
fixed documentation typos
Fixed a bug in URLGrabber._retry that prevented failure callbacks from
fixed typo in _make_callback and added testcase
enhanced the failover test to ensure that the bad mirror is tried and fails
Changed the callback syntax. This affects failure_callback use in
Updated callback tests to accomodate the new callback semantics. Also added
Added "release" and "daily" targets to makefile for automated releases.
release 2.9.0
updated ChangeLog
Fixed a typo in the new 'release' target. Reformatted the doc in __init__.py
Added "MANIFEST" to MANIFEST.in. Without this, one couldn't correctly
Added Ryan's progress changes in. That still has some quirks I need
Fixed python 2.4 bug - added .code attribute to returned file objects.
release 2.9.1
updated ChangeLog
slight cleanup - no new code
Added http character encoding patch from Chris Lumens.
release 2.9.2
updated ChangeLog
Fixed minor exception bug.
updated project url
updated README with instructions for building rpms and notes about
release 2.9.3
updated ChangeLog
Added data and instructions for setting up ftp and proxy servers for
Added http_headers and ftp_headers options.
release 2.9.4
updated ChangeLog
Fixed a bug caused by the recent proxy fix for python 2.2. This new
release 2.9.5
updated ChangeLog
Changed development status to Beta an fixed a small build bug. It
release 2.9.6
updated ChangeLog
Added tests to check proxy format. Otherwise, missing proto could
Changed the version checking so SUSE's python "2.3+" version string
Fixed a typo in that last change. Always remember to do "make test" before
started using sys.version_info rather than parsing sys.version
Fixed two namespace-related bugs.
except HTTPError, which basically means all HTTP return codes that are
Added Menno's idea for catching KeyboardInterrupt. This allows the
Fixed a minor formatting bug, helpfully pointed out by the nice folks
Another minor simplification offered by the WTF folks.
Added some slick logging support to urlgrabber. It can be controlled
Updated TODO and preparing for release.
updated ChangeLog
Added a reget progress bar patch from Menno, and fixed
release 2.9.8
updated ChangeLog
Significant improvement to URL parsing. Parsing is now broken out into
Added tests to make sure that the "quote" option works as advertised.
release 2.9.9
updated ChangeLog
Patch from Jeremy Katz to catch read errors after the file has been opened.
Fixed a minor error reporting bug due to changes in python 2.4.
Added patch from James Bowes to make keepalive, byteranges, etc. work
release 2.9.10
updated ChangeLog
release 3.0.0
updated ChangeLog
Added James Bowes' SSL patch to use M2Crypto when available.
release 3.1.0
updated ChangeLog
Minor fix to make byteranges work with some servers. _do_grab now only
Minor doc updates and error handling in grabber.py. Complete rewrite of
lots of changes... improved clarity of cached objects, improved debugging
more debugging code to expose options
rtomayko (48):
Mass commit/merge of changes made for urlgrabber restructuring.. See ChangeLog for details.
Restructured TODO list based on recent email from Michael.
Added item about handling KeyboardInterupt exceptions.
Changed all variables/args named 'range' to rangetup or brange to avoid possible python keyword clash.
Added checkfunc support to urlread.
Brought in fix from yum for user:pass escaping in URLs.
Added reget and batch stuff under the 'maybe' section.
Clean up temp files after testing urlgrab.
First cut at a module to run through some performance tests for grabber.py
Removed speed test from grabber.py. This test has been moved to test/grabberperf.py
Performance tweak to bypass URLGrabberFileObject.read when not using progress_obj or throttle.
Module for testing urlgrabber w/ multiple threads.
Some URLs for batchgrabber.py.
Confirmed FTP connections are being closed properly when using ranges.
urllib2 conditionals were cleaned up a while ago..
Always copy_local when a byte range is specified to urlgrab.
raising RangeError anytime a range is non-satisfiable.
Fixed try/except without an explicit exception. that's a no-no.
Added support for RangeError to URLGrabError handling (errno 9). Also, default is _not_ to retry grabs resulting in RangeError.
raise exceptions for invalid ranges DONE.
Fixed up for new cvs structure. Tried to simplify a bit while I was here.
Fixed up tests to work with new directory layout.
Added mirror tests to main test suite.
Removed ugly code that was suppose to allow a file-object to be passed to urlgrab as the output file.
Using a tempfile for urlgrab test instead of named file in cwd.
Added urlgrab, urlopen, urlread as package level exports.
Grabber tests now call package level urlgrab, urlopen, urlread functions.
Caching urllib2 ProxyHandler and OpenerDirectors instead of creating new ones for each request.
Added proxies to performance tests to ensure that ProxyHandlers are being added to the urllib2 OpenerDirector.
Removed item on reducing urllib2 opener director creation.
Fixed real small typo in mirror doc string.
Modified URLGrabberOptions to use "instance-inheritance" pattern instead of copying all options.
Removed items from TODO list related to options handling as well as the item on reget not working due to how options are copied.
Added opener kwarg/option to allow overriding of urllib2.OpenerDirector.
Added support for specifying verbosity of unit tests when using runtests.py. 'python runtests.py --help' for usage.
Fixed bug with urllib2.OpenerDirector caching.
Added distutils trove classifiers to setup.py for eventual inclusion in PyPI.
opener related items for ALPHA are completed.
Ignoring kdevelop, kate, and anjuta project files.
Updates for Michael's multifile progress bar changes.
Added timeout support.
Fixed AttributeError problem that was pretty common after additions for timeout support.
Added except for socket resource error when reading from urllib2 file object.
Fix for RH bug #140387
Applied urlgrabber-add-text.patch from Terje Rosten
Fixed proftpd test (was raising error when it shouldn't have been)
Fixed RH bug #147124.
Added Python 2.4 to lists of pythons to test..
More information about the Yum-commits
mailing list