That’s, of course, Python. A language so great that nearly all its flaws people talk about are shared evenly by others of its kin: the highly dynamic, interpreted languages. One would be hard-pressed to find anything that’s not simply a rehashed argument about maintainability, runtime safety or efficiency – concerns that apply equally well to Ruby, Perl and the like. What about anything specifically “pythonic”?…
Well, let’s talk about Python’s exceptions, shall we?
A long and detailed discussion is bound to follow now, isn’t it? There are so many exception classes in Python. Dozens, in fact, and the number has also grown significantly between 2.x and 3.x:
With this broad spectrum to choose from, it’s surely possible to pick a fine-grained exception class for even a highly unusual situation that may happen in your code. And when something goes awry in the standard library, a similarly detailed exception you’ve caught should tell you volumes about the problem by merely looking at the exception class.
Why then, in any typical code, in at least 90% of the cases, the exception you can throw or need to catch is either
Seriously, it’s as if those two were the only useful “abstract” exception types. Because quite frankly, they are: the other 30-odd
Exception subclasses are mostly floating point, I/O or system exceptions. They very nicely cover and distinguish all the various external failures that your code may need to deal with.
But what about the internal ones, of your code or the libraries it uses? Often, they are even more diverse, more important, and also more harmful. Yet you have a grand total of two exception classes to signal them!
Moreover, if the language and the standard modules are any hints, these two are pretty much the only classes you are meant to use. Just look how resourceful a simple
TypeError appears to be:
TypeEr… Well, that’s actually
SyntaxError. Helpful, is it not?
We’ve only covered errors related to function calls, by the way. We could produce a similar list for object creation, and extend it with few other, miscellaneous entries.
The crucial question would be still left unanswered, though: whose type is wrong here, anyway?
Okay, it’s true not every fundamental mishap in Python results in
ValueError. There is also
AttributeError for accessing non-existent attributes, as well as
Speaking of the latter two… Have you noticed how both of those exception refer to essentially the same situation? You botched the argument to
 operator (brackets), and you botched it in effectively the same manner: by taking it from outside of the allowed domain.
But somehow, it matters very much what is expected as the argument’s type. Is it number? Then it’s
IndexError. Something else, usually a string? Then
KeyError it is.
It is not accidental, of course. Indexing an array and performing a dictionary lookup look syntactically the same in Python, and given the typical interpreter’s overhead their internal differences are rarely relevant anymore. These two operations used to be very different, though, and they still very much are in C, C++, Java, etc. Even though it doesn’t have to, Python preserves this bit of computing “history” by having two separate exception classes.
Just why it suddenly tries to be meticulous here, while simultaneously throwing the all-encompassing
TypeError left and right?!
As a side note,
KeyError have a common base called
LookupError. Which makes them absolutely great, a shining example of how an exception hierarchy should be designed.
Too bad it’s also an exceedingly rare example.
The design of Python’s standard exception, as it is shown, leaves much to be desired. You could ask why is it even important, though, and it’s perfectly legitimate question.
In production code of most applications, not much time is spent on error capture and recovery. When an exception happens, there is seldom anything to recover anyway. Most often, the error either causes the whole program to quit, or fails the current transaction (such as HTTP request).
Under these circumstances, it doesn’t matter a whole lot what kind of exception was it exactly. It’s of secondary importance at best, relevant mostly to error reporting: what severity of a log entry to use, or maybe what HTTP status code to return. No one will be splitting hairs if a sensible default is used – like priority
ERROR or code
But sometimes, this stuff matters. When you’re writing tests for your application, you want to verify that in well-defined circumstances, and for specific class of data, the code behaves precisely as intended. For successful runs, the behavior will include its immediate output and any state changes it effects. We verify those by assertions on function results, mocks, or both.
For failures, though, we need to find out what exactly was the failure mode that the fixture triggered. Capturing and inspecting exceptions is the easiest way to accomplish that, but it’s contingent on them having reliable, distinguishing features. When nigh every exception is
TypeError from the standard library, or
ValueError from own and third party code, it all becomes significantly trickier. At worst, it turns into a choice between underspecified and fragile tests, because the optimal solution would require your test suite to speak goddamn English:
Similarly, writing a library intended for external use requires paying attention to all parts of the interface we’re exposing – including exceptions. They should allow to quickly and easily tell all possible error cases apart. Of course, the end user might not care about some or even all of them, but it’s always their call to make. Good library will therefore accommodate for any such decisions.
As the state of affairs regarding exceptions in Python looks so shoddy, what we can do as developers to help with ameliorating it? There is no magic wand to fix the standard library, short of maybe writing PEPs. Migrating to Python 3 also improves things very, very little.
Not everything is beyond our reach, though. The code you have definite control over is obviously the one that you write. Make sure you introduce more diversity into exceptions you raise, instead of surrendering to the malpractice of throwing
ValueError whenever anything at all wrong happens. Domain-specific exceptions are cheap to make, and they can still inherit from the omnipresent
ValueError to maintain compatibility.