As the Python docs state,
with statement is used to encapsulate common patterns of
finally constructs. It is most often associated with managing external resources in order to ensure that they are properly freed, released, closed, disconnected from, or otherwise cleaned up. While at times it is sufficient to just write the
finally block directly, repeated occurrences ask for using this language goodness more consciously, including writing our own context managers for specialized needs.
Those managers – basically a
with-enabled, helper objects – are strikingly similar to small local objects involved in the RAII technique from C++. The acronym expands to Resource Acquisition Is Initialization, further emphasizing the resource management part of this pattern. But let us not be constrained by that notion: the usage space of
with is much wider.
To create our own context manager, we can implement a class with
__exit__ methods. The former is called when
with block is about to be entered, which is typically associated with obtaining an external resource: file handle, database connection, native memory pool, and so on.
Similarly, once we are about to leave the
with block, the
__exit__ method will be invoked. If it was an exception that has kicked us out, relevant information will be passed to
__exit__ as its arguments; we’ll be able to inspect them if necessary.
More details on both of these methods can be of course found in the documentation, which I encourage to peek at.
It is perfectly valid for either of those methods to be a no-op, although there is seldom a point in having only
__enter__, Context manager with just
__exit__, on the other hand, can be quite useful for executing some finishing action that absolutely must be done. If we also have to account for possible errors, custom
with is good alternative to doubly nested
try statements that would have to be used instead.
Let’s look at a practical example. On App Engine, there is a Blobstore service which allows to upload & store big files that are infeasible to keep as text or binary data in database. When it comes to uploading them, most of the relevant logic is already present, so we only have to implement the
The required part here is performing the HTTP
redirect at the end, regardless of whether we successfully did something with the uploaded blob (file) or not. But if “doing something” fails, we would like to delete the blob; otherwise, it will just needlessly take space.
Such behavior can be rather easily contained within a specialized context manager with just the
If we now put the upload handler’s code inside a
with statement with above guard, we will always have a redirect issued, and our blobs will be cleaned up properly in case of error:
Typical uses of
with take advantage of its
as part, which is almost always used to capture some kind of resource handle. A
file object is a canonical example of such practice:
If we want to take advantage of this behavior for our own context managers, we shall return a result from the
__enter__ method. It will then be given a name (specified after
as), and we’ll be able to use it within the
with block, as shown above.
But referencing a resource is by no means the only legitimate use for name declared via
as. It can actually point to any object, no matter what it is or does.
To illustrate this point, I have came up with rather devious semantical construct which I dubbed a “section”. In some way it’s the closest thing to
goto that we can have in Python without resorting to dirty bytecode hacks. It doesn’t support actual jumps (i.e. moving instruction pointer into arbitrary place), but allows to prematurely “bail” out of the
with scope, if desired. Hence, it is analogous to the
break label mechanism from Java, while also serving similar purpose to infamous
Of course, there aren’t many uses for such a thing, and I would prefer not to maintain code that actually employs this trick. But it’s entirely sufficient as an example – albeit somewhat silly. Here’s how it could be implemented:
As you can see, we’re using a special exception as means for “bailing out” of the section. The interesting bits happen mostly inside
__exit__, where we examine the type of exception (if any) and the section it came from (to account for possible nesting). We are then returning a boolean value, indicating whether the exception should be squelched.
__enter__ function is also notable, since it’s quite common to just
return self from it. This way, the context manager itself becomes the
as-object. Not accidentally, this is also the case with the
file type, which is what allows for
open function to be used both normally and in
as clause. For
section, such behavior is not required, but
self is simply the most straightforward solution.
Looking at slightly bigger picture, we can see that
with statements create something which might be called a programmable scope. Thanks to
__exit__ methods, we are able to capture the events of going in and out of these blocks, and act upon them.
I suspect that when used cleverly, this may result in some pretty powerful and impressive programming techniques, not really related to any sort of resource management. I tried to show a glimpse, but I believe there are many more fruits to be picked from that tree – including:
withblock. This could be further expanded into measurements of varying granularity, thanks to
withblocks. If done correctly, it could result in nice and neat API that wouldn’t hide the control flow of parsing process – unlike most (all?) libraries that rely on explicit specification of grammar.
withcould be used to check whether test code throws an expected exception. It would be much more convenient (and readable) than
assertRaisesmethod from standard
withto put emphasis on certain notable assignments in our code. This is vaguely similar to
whereclauses from functional languages, but of course it doesn’t have any purity implications. The downside: we would often need to use a “fake” context manager if object being assigned isn’t already one:
This list is of course nowhere near complete. I encourage to look at the
with statement in similarly unorthodox way, for there are likely many semantic gems (or at least polished rocks ;]) to be uncovered here.