The “Wat” of Python

2012-01-31 21:19

It is quite likely you are familiar with the Wat talk by Gary Bernhardt. It is sweeping through the Internet, giving some good laugh to pretty much anyone who watches it. Surely it did to me!
The speaker is making fun of Ruby and JavaScript languages (although mostly the latter, really), showing totally unexpected and baffling results of some seemingly trivial operations – like adding two arrays. It turns out that in JavaScript, the result is an empty string. (And the reasons for that provoke even bigger “wat”).

After watching the talk for about five times (it hardly gets old), I started to wonder whether it is only those two languages that exhibit similarly confusing behavior… The answer is of course “No”, and that should be glaringly obvious to anyone who knows at least a bit of C++ ;) But beating on that horse would be way too easy, so I’d rather try something more ambitious.
Hence I ventured forth to search for “wat” in Python 2.x. The journey wasn’t short enough to stop at mere addition operator but nevertheless – and despite me being nowhere near Python expert – I managed to find some candidates rather quickly.

I strove to keep with the original spirit of Gary’s talk, so I only included those quirks that can be easily shown in interactive interpreter. The final result consists of three of them, arranged in the order of increasing puzzlement. They are given without explanation or rationale, hopefully to encourage some thought beyond amusement :)

Behold, then, the Wat of Python!

3. Strange loop

  1. >>> f = lambda: map((yield), range(10))
  2. >>> for x in f(): print x
  3. ...
  4. None

None shall pass!

2. Can’t catch me

  1. >>> try: raise KeyError
  2. ... except ValueError, KeyError:
  3. ...     print "Batman!"
  4. ...
  5. Traceback (most recent call last):
  6.   File "<stdin>", line 1, in <module>
  7. KeyError

It takes much more to summon Batman, you know.

1. The truth is out there

  1. >>> True, False = False, True
  2. >>> True
  3. False

You had probably seen it coming…. Yeah, you certainly had.

If you know any similar quirks of Python, be sure to present them in comments!

Tags: , , ,
Author: Xion, posted under Internet, Programming »


12 comments for post “The “Wat” of Python”.
  1. gryf:
    February 1st, 2012 o 19:12

    1. Well, True and False are just syntactic sugar for integers 1 and 0 which doesn’t exists in Python till version 2.3. You can get them back from builtins, but also you can destroy builtins :)

    >>> (True == 1, False == 1, False == 0, True ==0)
    (True, False, True, False)
    >>> True, False = False, True
    >>> (True == 1, False == 1, False == 0, True ==0)
    (False, True, False, True)
    >>> __builtins__.True == True
    False
    >>> # uuh! surprise!
    ...
    >>> __builtins__.True = 'Fuuuuu…'
    >>>

    Traceback (most recent call last):
    File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
    TypeError: an integer is required
    >>> # oh my… we have broken interpreter

    I assume, that you wanted to use them in case of comparison, like in C languages, but don’t do that, it’s silly (even though Python doesn’t allow to assignment in if statements)

    2. Well, if you had have done your homework…

    >>> try:
    ... raise KeyError('please, read docs carefully!')
    ... except ValueError, KeyError:
    ... print "Batman!"
    ... except KeyError, msg:
    ... print msg
    ...
    'please, read docs carefully!'
    >>> try:
    ... raise KeyError
    ... except (KeyError, ValueError):
    ... print "Batman :)"
    ...
    Batman :)
    >>>

    3. Do you actually wanted to use such construction in real code? :D

  2. dasm:
    February 1st, 2012 o 20:14

    Good work gryf :)
    Xion, I recommend you to read O’Reillys “Learning Python” by Lutz and Ascher. Recently, I read that book (2nd edition) and everything what gryf wrote, is ideally described there.

  3. elGonzales:
    February 1st, 2012 o 23:06

    WAT about Ruby is very silly, because it doesn’t suprise any Ruby user.

  4. Xion:
    February 1st, 2012 o 23:08

    @gryf: I think you have slightly misunderstood :) The logic behind all of those quirks (well, maybe besides True and False not being constants like None) is well known to me, but I still consider them as misbehaviors, or quite nasty traps. It’s great you have dissected them, though – it makes the whole matter informative rather than only amusing :)

    And as for destroying the True and False symbols, you can always restore them by setting to logical value of condition that is actually true:

    1. >>> __builtins__.True = 1==1

    This reminds me how I did infinite loops back in the days :D

  5. gryf:
    February 2nd, 2012 o 12:39

    @Xion, about restoring it – not really – it doesn’t work in 2.6 and above (in Python 3 reassignment bool literals is not possible, since True and False are keywords). Those Python “wat’s” are not so obvious as in Ruby or Javascript. In dynamic languages you have to know how interpreter works in certain situations. Especially the first example with lambda expression and yield statement is not comparable to just simple gotchas like adding an array to an object in Javascript :)

  6. Xion:
    February 2nd, 2012 o 13:57

    @gryf: Weird – it does work in 2.7.2 (tested on both Windows and Linux one), and you easily restore both True and __builtins__.True that way.

    The first example uses lambda only because it’s shorter than def. It’s really about mapping yield over a collection – something you’d thought to be equivalent to:

    1. for x in range(10): yield x

    while in reality yield does not behave like a function AND it can also receive values which ultimately leads to the puzzling None in the first example.

    (The inability to do map((print), range(10)) in Python 2.x is of similar nature, but at least it results in SyntaxError)

  7. gryf:
    February 2nd, 2012 o 20:21

    Weird indeed:

    $ python2.6
    Python 2.6.6 (r266:84292, Feb 25 2011, 10:49:25)
    [GCC 4.4.4] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    'starting up...'
    >>> __builtins__.True = "foo"
    >>> __builtins__.True = 1==1

    Traceback (most recent call last):
    File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
    TypeError: an integer is required
    >>>

    …and 2.7:


    $ python2.7
    Python 2.7.2 (default, Oct 24 2011, 16:38:44)
    [GCC 4.5.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    'starting up...'
    >>> __builtins__.True = "foo"
    >>> __builtins__.True = 1==1

    Traceback (most recent call last):
    File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
    TypeError: an integer is required
    >>>

    On Python 2.5 it works just as you wrote.

    And about yield – you are using it in conjunction with lambda anonymous function, which by its definition should return something. And than you give yield statement. Well, I’m not sure (I have some suspicious though) what actually is happening but mixing those two is a bad idea.

  8. wong_jim:
    February 4th, 2012 o 14:29

    I agree with @gryf in general but I can shed some light on 1) WRT the yield inside a lambda.

    If you’ve read the docs, you’ll come to realize that Python’s lambda is a strange beast that is a relic of Python’s early history. A Python lambda’s body can consist of 1 and only 1 expression which is implicitly wrapped inside a return statement.

    See Python Language Reference section 5.12.

    Now, in Python, a yield is a statement, not an expression, so if you do this:

    lambda: map(yield, range(10))

    The interpreter will complain with a SyntaxError because function params expects expressions, not statements. However, if you wrap it inside a delimiter like what you did (yield), now map() will accept it as a parameter. So what’s going on here? Well, your straw man example got around the Python parser. “yield” is only supposed to be used inside a function definition, so it goes thru the parser in this case because it is inside a lambda. Now, a “yield” yields the value on the RHS expression list. What’s on the RHS of (yield)? There’s nothing, just like and empty “return”. So when you invoke the generator “l”, it always yields None.

    See Python Language Reference section 5.2.10.

    Your second and third example are known problems with the Python 2.x, they are fixed in Python 3.x. Your first example is a little contrived tho, I don’t see how one “fix” it, or have it come up in real life tho. When you need to use () to get around something, that’s usually a code smell.

  9. Xion:
    February 4th, 2012 o 16:43

    Python’s lambdas that allow only for a single expression don’t appear strange to me. It’s limiting, of course, but there are well-known analogies in other languages, including lambdas in Boost.Lambda for C++98, or just every function (anonymous or not) in Haskell. IIRC a shortened syntax for C# anonymous functions and Java 7 syntactic sugar for Runnable/Callable interfaces also permit only one expression with implicit return.

    I digress, though, because in the first example, the lambda is largely irrelevant. The same behavior can be observed when rephrasing to usual function notation:

    1. def f():
    2.     map((yield), range(10))

    Most of your explanation of why this works as it does remains correct, of course. The only thing that is missing is to notice that yield works both ways: not only “returning” a value, but also accepting one, through send method of its enclosing generator (f() here). The second part is important, because it’s that very value which gets mapped over range(10).

    But the value in question is None, as we haven’t sent anything to our generator. And incidentally, map allows for its first parameter to be None rather than a callable, and doesn’t signal any error in such case. (It is equivalent to passing an identity function.) Only combining those two facts we can explain None as a result of loop from the first example. A similar but admittedly much more blatant f would obviously lead to an error (AttributeError, in this case):

    1. def f():
    2.     (yield).x = 42
  10. gryf:
    February 5th, 2012 o 20:59

    As a side note, rephrasing lambda expression as a function in this case would be:

    def f():
    return map((yield), range(10))

    Obviously it will not work – interpreter complains about „’return’ with argument inside generator”. My suspicion here is that lambda is working a bit different than function – most probably it will return first element of generator collection that map will create, and because the yield itself works somehow as a return statement the chain breaks here. What is returned here is value first generator is returned – None. I’m not sure if this is correct unless some analysis of the python code is done.

  11. Brandyn:
    February 23rd, 2012 o 6:25

    Another way of looking at #3

    f = lambda: map((yield), skjdsk)
    >>> for x in f(): print x

    None
    Traceback (most recent call last):
    File “”, line 1, in
    File “”, line 1, in
    NameError: global name ‘skjdsk’ is not defined

    Notice that it prints “None” first, then after the first value it evaluates “skjdsk” which is undefined. Another way of looking at this is
    >>> f = lambda: map((yield), skjdsk)
    >>> l = f()
    >>> print l.next()
    None
    >>> print l.next()
    Traceback (most recent call last):
    File “”, line 1, in
    File “”, line 1, in
    NameError: global name ‘skjdsk’ is not defined

    This shows that the second argument to “map” isn’t even evaluated until after we resume from the yield (i.e., the second call to .next()). The map itself would return the range count still as
    >>> map(None, range(10))
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

    But since in a generator yield must be used to produce additional outputs this is ignored.

  12. BatmanAoD:
    July 18th, 2014 o 23:43

    >>> def foo(a,b=-1): pass

    >>> foo(b=3)
    Traceback (most recent call last):
    File “”, line 1, in
    TypeError: foo() takes at least 1 argument (1 given)

Comments are disabled.
 


© 2017 Karol Kuczmarski "Xion". Layout by Urszulka. Powered by WordPress with QuickLaTeX.com.