Deprecate This

As a fact of life, in bigger projects you often cannot just delete something – be it function, method, class or module. Replacing all its usages with whatever is the new recommendation – if any! – is typically outside of your influence, capabilities or priorities. By no means it should be treated as lost cause, though; any codebase would be quickly overwhelmed by kludges if there were no way to jettison them.

To reconcile those two opposing needs – compatibility and cleanliness – the typical approach involves a transition period. During that time, the particular piece of API shall be marked as deprecated, which is a slightly theatrical term for ‘obsolete’ and ‘not intended for new code’. How effective this is depends strongly on target audience – for publicly available APIs, someone will always wake up and start screaming when the transition period ends.

For in-project interfaces, however, the blow may be effectively cushioned by using certain features of the language, IDE, source control, continuous integration, and so on. As an example, Java has the @Deprecated annotation that can be applied to functions or classes:

If the symbol is then used somewhere else, it produces a compiler warning (and visual cue in most IDEs). These can be suppressed, of course, but it’s something you need to do explicitly through a complementary language construct.

So I had this idea to try and add similar mechanism to Python. One part of it is already present in its standard library: we have the warnings module and a built-in category of DeprecationWarnings. These can be ignored, suppressed, caught or even made into errors.
They are also pretty powerful, as they allow to deprecate certain code paths and not just symbols, which can be useful when introducing new meanings for function parameters, among other things. At the same time, it means using them is irritatingly imperative and adds clutter:

And in this particular case, it also doesn’t work as intended, for reasons that will become apparent later on.
What we’d like instead is something similar to annotation approach that is available in Java:

Given that the @-things in Python (decorators, that is) are significantly more powerful than the Java counterparts, it shouldn’t be a tough call to achieve this…

Surprisingly, though, it turns out to be very tricky and quite arcane. The problems lie mostly in the subtle issues of what exactly constitutes “usage” of a symbol in Python, and how to actually detect it. If you try to come up with a few solutions, you’ll soon realize how the one that may eventually require walking through the interpreter call stack turns out to be the least insane one.

But hey, we didn’t go to the Moon because it was easy, right? ;) So let’s see how at least we can get started.

Decorated Functions in C++… Almost

Many languages now include the concept of annotations that can be applied to definitions of functions, classes, or even variables and fields. The term ‘annotation’ comes from Java, while other languages use different names: attributes (C#), decorators (Python), tags (Go), etc.
Besides naming disparities, those features also tend to differ in terms of offered power and flexibility. The Python ones, for example, allow for almost arbitrary transformations, while Go tags are constrained to struct fields and can only consist of text labels. The archetypal annotations from Java lie somewhere in between: they are mostly for storing metadata, but they can also be (pre)processed during compilation or runtime.

Now, what about C++? We know the language has a long history of lacking several critical features (*cough* delegates *cough*), but the recent advent of C++11 fixed quite a few of them all at once.
And at first sight, the lack of annotation support seems to be among them. New standard introduces something called attributes, which appears to fall in into the same conceptual bucket:

That’s misleading, though. Attributes are nothing else than unified syntax for compiler extensions. Until C++11, the job of attributes were done by custom keywords, such as __attribute__ (GCC) or __declspec (Visual C++). Now they should be replaced by the new [[squareBracket]] syntax above.
So there is nothing really new about those attributes. At best, you could compare them to W3C deciding on common syntax for border-radius that forces both -webkit-border-radius and -moz-border-radius to adapt. Most importantly, there is no standard way to define your custom attributes and introspect them later.

That’s a shame. So I thought I could try to fix that, because the case didn’t look completely lost. In fact, there is a precedent for a mainstream language where some people implemented something-kinda-almost-like annotations. That language is JavaScript, where annotations can be realized as functions arranged into a pipeline. Here’s an example from Express framework:

Both loginRequired and cached(...) are functions, tied together by app.get with the actual request handler at the end. They “decorate” that handler, wrapping it inside a code which offers additional functionality.

But that’s JavaScript, with its dynamic typing and callback bonanza. Can we even try to translate the above code into C++?…
Well yes, we actually can! Two new features from C++11 allow us to attempt this: lambda functions and initializer lists. With those two – and a healthy dose of functional programming – we may achieve at least something comparable.

Alternative @property Syntax

As you probably know very well, in Python you can add properties to your classes. They behave like instance fields syntactically, but under the hood they call accessor functions whenever you want to get or set the property value:

Often – like in the example above – properties are read-only, providing only the getter method. It’s very easy to define them, too: just stick a @property decorator above method definition and you’re good to go.

iGet, iSet

Occasionally though, you will want to define a read-write property. (Or read-delete, but those are very rare). One function won’t cut it, since you need a setter in addition to getter. The canonical way Python docs recommend in such a case (at least since 2.6) is to use the @property.setter decorator:

Besides that I find it ugly to split a single property between two methods, this approach will annoy many static code analyzers (including PEP8 checker) due to redefinition of x. Warnings like that are very useful in general, so we certainly don’t want to turn them off completely just to define a property or two.

So if our analyzer doesn’t support line-based warning suppression (like, again, pep8), we may want to look for a different solution.

Decorators with Optional Arguments in Python

It is common that features dubbed ‘syntactic sugar’ are often fostering novel approaches to programming problems. Python’s decorators are no different here, and this was a topic I touched upon before. Today I’d like to discuss few quirks which are, unfortunately, adding to their complexity in a way that often doesn’t feel necessary.

Let’s start with something easy. Pretend that we have a simple decorator named @trace, which logs every call of the function it is applied to:

An implementation of such decorator is relatively trivial, as it wraps the decorated function directly. One of the possible variants can be seen below:

That’s pretty cool for starters, but let’s say we want some calls to stand out in the logging output. Perhaps there are functions that we are more interested in than the rest. In other words, we’d like to adjust the priority of log messages that are generated by @trace:

This seemingly small change is actually mandating massive conceptual leap in what our decorator really does. It becomes apparent when we de-sugar the @decorator syntax and look at the plumbing underneath:

Introduction of parameters requires adding a new level of indirection, because it’s the return value of trace(level=logging.INFO) that does the actual decorating (i.e. transforming given function into another). This might not be obvious at first glance and admittedly, a notion of function that returns a function which takes some other function in order to output a final function might be – ahem – slightly confusing ;-)

But wait! There is just one more thing… When we added the level argument, we not necessarily wanted to lose the ability to invoke @trace without it. Yes, it is still possible – but the syntax is rather awkward:

That’s expected – trace only returns the actual decorator now – but at least slightly annoying. Can we get our pretty syntax back while maintaining the added flexibility of specifying custom arguments? Better yet: can we make @trace, @trace() and @trace(level) all work at the same time?…

Looks like tough call, but fortunately the answer is positive. Before we delve into details, though, let’s step back and try to somewhat improve the way we are writing our decorators.

Udekorowane funkcje

Kilkanaście dni temu opisywałem błąd, który – jak się okazało – był efektem ubocznym pewnej cechy języka Python, uznanej przeze mnie za niemal zupełnie niepotrzebną. Dla równowagi więc dzisiaj przedstawię feature, który wydaje się być bardzo przydatny – żeby nie było, że potrafię tylko krytykować ;-)

Prawdopodobnie najlepiej jest pokazać go na prostym, acz obrazowym przykładzie. Miejmy pewną funkcję na tyle dla nas istotną, że chcemy logować wszystkie jej wywołania. Wydaje się, że nic prostszego:

Ciężko jednak nazwać takie rozwiązanie uniwersalnym. Nie chodzi tu jedynie o jawnie wpisaną nazwę funkcji, ale raczej o konieczność dodawania logging.debug(...) na początku każdej funkcji, jaką chcemy monitorować. Sprawa komplikuje się jeszcze bardziej, gdy interesują nas też wyjścia z funkcji; wówczas jedynym wyjściem jest chyba opakowanie całej treści w jeden wielki blok try–finally. Rezultat na pewno nie będzie piękny :)

I tutaj właśnie z pomocą przychodzą dekoratory – ciekawa opcja języka Python, na pierwszy rzut oka przypominająca adnotacje z Javy. Podobieństwo jest jednak głównie składniowe. Udekorowana wersja naszej ważnej funkcji wygląda bowiem tak:

Znak @ poprzedza tutaj nazwę dekoratora, czyli trace (ang. śledź – i bynajmniej nie chodzi o rybę ;]). Czym jednak jest ów dekorator? Otóż on sam również jest funkcją, mogącą wyglądać choćby tak:

Jej jedynym argumentem jest w założeniu funkcja, zaś rezultatem wywołania jest… również funkcja :) A zatem nasz dekorator potrafi przekształcić jedną funkcję w drugą, a dokładniej w jej udekorowaną, “opakowaną” wersję. To opakowanie definiowane jest wewnątrz dekoratora i polega, jak widać, na poprzedzeniu wywołania oryginalnej funkcji zapisem do loga.
Oczywiście za to, aby wywołanie funkcji opatrzonej dekoratorem było tak naprawdę odwołaniem do jej udekorowanej wersji odpowiada już sam język. Nie jest to zresztą skomplikowane, bo w istocie cały mechanizm jest tylko cukierkiem składniowym. Jest to jednak bardzo smaczny cukierek, który potrafi wydatnie podnieść czytelność kodu – jeśli stosuje się go właściwie.

Do czego jednak – oprócz logowania wywołań – dekoratory mogą się przydać? Ano przede wszystkim do upewniania się, że pewne konieczne warunki wstępne dla funkcji są spełnione na jej wejściu (i ewentualnie wyjściu). Może to być na przykład:

• @connected – sprawdzenie połączenia z serwerem zanim spróbujemy wymieniać z nim dane i nawiązanie go w razie potrzeby
• @authorized – określenie uprawnień wymaganych u aktualnie zalogowanego użytkownika przed wywołaniem funkcji wykonującej potencjalnie niebezpieczną operację
• @synchronized – zabezpieczenie wywołania funkcji semaforem lub sekcją krytyczną

Wspólną cechą takich dekoratorów jest to, że są one swoistymi pseudodeklaracjami, nieodległymi koncepcyjnie zbyt daleko od komentarzy w rodzaju:

Ich przewagą jest jednak rzeczywiste sprawdzanie, czy wymagania zostały spełnione – i to w sposób automatyczny i przezroczysty. Według mnie to właśnie stanowi o ich sporej przydatności.