The “Wat” of Python

1. Well, True and False are just syntactic sugar for integers 1 and 0 which doesn’t exists in Python till version 2.3. You can get them back from builtins, but also you can destroy builtins :)

>>> (True == 1, False == 1, False == 0, True ==0)
(True, False, True, False)
>>> True, False = False, True
>>> (True == 1, False == 1, False == 0, True ==0)
(False, True, False, True)
>>> __builtins__.True == True
False
>>> # uuh! surprise!
...
>>> __builtins__.True = 'Fuuuuu…'
>>>

Traceback (most recent call last):
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
TypeError: an integer is required
>>> # oh my… we have broken interpreter


I assume, that you wanted to use them in case of comparison, like in C languages, but don’t do that, it’s silly (even though Python doesn’t allow to assignment in if statements)

2. Well, if you had have done your homework…

>>> try:
... raise KeyError('please, read docs carefully!')
... except ValueError, KeyError:
... print "Batman!"
... except KeyError, msg:
... print msg
...
'please, read docs carefully!'
>>> try:
... raise KeyError
... except (KeyError, ValueError):
... print "Batman :)"
...
Batman :)
>>>


3. Do you actually wanted to use such construction in real code? :D

Good work gryf :)
Xion, I recommend you to read O’Reillys “Learning Python” by Lutz and Ascher. Recently, I read that book (2nd edition) and everything what gryf wrote, is ideally described there.

WAT about Ruby is very silly, because it doesn’t suprise any Ruby user.

@gryf: I think you have slightly misunderstood :) The logic behind all of those quirks (well, maybe besides True and False not being constants like None) is well known to me, but I still consider them as misbehaviors, or quite nasty traps. It’s great you have dissected them, though – it makes the whole matter informative rather than only amusing :)

And as for destroying the True and False symbols, you can always restore them by setting to logical value of condition that is actually true:

This reminds me how I did infinite loops back in the days :D

@Xion, about restoring it – not really – it doesn’t work in 2.6 and above (in Python 3 reassignment bool literals is not possible, since True and False are keywords). Those Python “wat’s” are not so obvious as in Ruby or Javascript. In dynamic languages you have to know how interpreter works in certain situations. Especially the first example with lambda expression and yield statement is not comparable to just simple gotchas like adding an array to an object in Javascript :)

@gryf: Weird – it does work in 2.7.2 (tested on both Windows and Linux one), and you easily restore both True and __builtins__.True that way.

The first example uses lambda only because it’s shorter than def. It’s really about mapping yield over a collection – something you’d thought to be equivalent to:

while in reality yield does not behave like a function AND it can also receive values which ultimately leads to the puzzling None in the first example.

(The inability to do map((print), range(10)) in Python 2.x is of similar nature, but at least it results in SyntaxError)

Weird indeed:

$ python2.6
Python 2.6.6 (r266:84292, Feb 25 2011, 10:49:25)
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
'starting up...'
>>> __builtins__.True = "foo"
>>> __builtins__.True = 1==1

Traceback (most recent call last):
File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
TypeError: an integer is required
>>>


…and 2.7:


$ python2.7
Python 2.7.2 (default, Oct 24 2011, 16:38:44)
[GCC 4.5.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
'starting up...'
>>> __builtins__.True = "foo"
>>> __builtins__.True = 1==1

Traceback (most recent call last):
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
TypeError: an integer is required
>>>


On Python 2.5 it works just as you wrote.

And about yield – you are using it in conjunction with lambda anonymous function, which by its definition should return something. And than you give yield statement. Well, I’m not sure (I have some suspicious though) what actually is happening but mixing those two is a bad idea.

I agree with @gryf in general but I can shed some light on 1) WRT the yield inside a lambda.

If you’ve read the docs, you’ll come to realize that Python’s lambda is a strange beast that is a relic of Python’s early history. A Python lambda’s body can consist of 1 and only 1 expression which is implicitly wrapped inside a return statement.

See Python Language Reference section 5.12.

Now, in Python, a yield is a statement, not an expression, so if you do this:

lambda: map(yield, range(10))

The interpreter will complain with a SyntaxError because function params expects expressions, not statements. However, if you wrap it inside a delimiter like what you did (yield), now map() will accept it as a parameter. So what’s going on here? Well, your straw man example got around the Python parser. “yield” is only supposed to be used inside a function definition, so it goes thru the parser in this case because it is inside a lambda. Now, a “yield” yields the value on the RHS expression list. What’s on the RHS of (yield)? There’s nothing, just like and empty “return”. So when you invoke the generator “l”, it always yields None.

See Python Language Reference section 5.2.10.

Your second and third example are known problems with the Python 2.x, they are fixed in Python 3.x. Your first example is a little contrived tho, I don’t see how one “fix” it, or have it come up in real life tho. When you need to use () to get around something, that’s usually a code smell.

Python’s lambdas that allow only for a single expression don’t appear strange to me. It’s limiting, of course, but there are well-known analogies in other languages, including lambdas in Boost.Lambda for C++98, or just every function (anonymous or not) in Haskell. IIRC a shortened syntax for C# anonymous functions and Java 7 syntactic sugar for Runnable/Callable interfaces also permit only one expression with implicit return.

I digress, though, because in the first example, the lambda is largely irrelevant. The same behavior can be observed when rephrasing to usual function notation:

Most of your explanation of why this works as it does remains correct, of course. The only thing that is missing is to notice that yield works both ways: not only “returning” a value, but also accepting one, through send method of its enclosing generator (f() here). The second part is important, because it’s that very value which gets mapped over range(10).

But the value in question is None, as we haven’t sent anything to our generator. And incidentally, map allows for its first parameter to be None rather than a callable, and doesn’t signal any error in such case. (It is equivalent to passing an identity function.) Only combining those two facts we can explain None as a result of loop from the first example. A similar but admittedly much more blatant f would obviously lead to an error (AttributeError, in this case):

As a side note, rephrasing lambda expression as a function in this case would be:

def f():
return map((yield), range(10))


Obviously it will not work – interpreter complains about „’return’ with argument inside generator”. My suspicion here is that lambda is working a bit different than function – most probably it will return first element of generator collection that map will create, and because the yield itself works somehow as a return statement the chain breaks here. What is returned here is value first generator is returned – None. I’m not sure if this is correct unless some analysis of the python code is done.

Another way of looking at #3

f = lambda: map((yield), skjdsk)
>>> for x in f(): print x
…
None
Traceback (most recent call last):
File “”, line 1, in
File “”, line 1, in
NameError: global name ‘skjdsk’ is not defined

Notice that it prints “None” first, then after the first value it evaluates “skjdsk” which is undefined. Another way of looking at this is
>>> f = lambda: map((yield), skjdsk)
>>> l = f()
>>> print l.next()
None
>>> print l.next()
Traceback (most recent call last):
File “”, line 1, in
File “”, line 1, in
NameError: global name ‘skjdsk’ is not defined

This shows that the second argument to “map” isn’t even evaluated until after we resume from the yield (i.e., the second call to .next()). The map itself would return the range count still as
>>> map(None, range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

But since in a generator yield must be used to produce additional outputs this is ignored.

>>> def foo(a,b=-1): pass
…
>>> foo(b=3)
Traceback (most recent call last):
File “”, line 1, in
TypeError: foo() takes at least 1 argument (1 given)