[Yum-devel] [PATCH] Stop the python encoding madness, with a big hammer of fire.

David Malcolm dmalcolm at redhat.com
Mon Mar 7 19:28:01 UTC 2011


On Mon, 2011-03-07 at 10:03 -0800, Toshio Kuratomi wrote:
> On Mon, Mar 07, 2011 at 10:56:07AM -0500, seth vidal wrote:
> > On Mon, 2011-03-07 at 10:52 -0500, James Antill wrote:
> > > ---
> > >  yum/misc.py |    7 ++++++-
> > >  1 files changed, 6 insertions(+), 1 deletions(-)
> > > 
> > > diff --git a/yum/misc.py b/yum/misc.py
> > > index 8e81c34..305d4aa 100644
> > > --- a/yum/misc.py
> > > +++ b/yum/misc.py
> > > @@ -977,7 +977,8 @@ def getloginuid():
> > >  # ---------- i18n ----------
> > >  import locale
> > >  import sys
> > > -def setup_locale(override_codecs=True, override_time=False):
> > > +def setup_locale(override_codecs=True, override_time=False,
> > > +                 override_encoding=True):
> > >      # This test needs to be before locale.getpreferredencoding() as that
> > >      # does setlocale(LC_CTYPE, "")
> > >      try:
> > > @@ -995,6 +996,10 @@ def setup_locale(override_codecs=True, override_time=False):
> > >          import codecs
> > >          sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout)
> > >          sys.stdout.errors = 'replace'
> > > +    if override_encoding:
> > > +        # Dear python, please let your 'ascii' default die in a fire. kthxbye
> > > +        reload(sys)
> > > +        sys.setdefaultencoding('utf-8')
> > >  
> > > 
> > >  def get_my_lang_code():
> > 
> > 
> > So, you're just interested in seeing what ways this breaks things?
> > 
> > How about we apply this to rawhide yum first, just for s&g and see what
> > goes KABOOM before applying upstream?


I'm not a yum developer, FWIW, I'm not keen on the proposed patch,
though from the inline comment, it's not clear whether or not the patch
is meant to be taken seriously, or just a vent of frustration.

Here are some alternate approaches:
  - characterize the kinds of bugs that happen, write them up in the
project's developer docs (e.g. the wiki), and write unit tests to cover
them
  - run the unit test suite with this in the environment:
       PYTHONIOENCODING=ascii
so that over-optimisic assumptions about encodings fail even when
stdout/stderr is directly connected to a tty, avoiding the nasty gotcha
when run as a daemon.
  - write a helper function to assert the type of data flowing through
the system (str vs unicode, encoding of str) and fail quickly when a
variable is of the wrong data type.
  - or a subclass of str that enforces the encoding rules, or a custom
class for string data that has the encoding behavior you want.
  - use Toshio's code
etc

FWIW, you may want to review this thread from when my proposal to fix
this got shot down upstream:
  http://thread.gmane.org/gmane.comp.python.devel/109914

(the main python-dev archives seem to be corrupted for that period,
alas)

Hope this is useful
Dave



More information about the Yum-devel mailing list