Posts tagged ‘decorators’

Deprecate This

2013-05-26 14:02

As a fact of life, in bigger projects you often cannot just delete something – be it function, method, class or module. Replacing all its usages with whatever is the new recommendation – if any! – is typically outside of your influence, capabilities or priorities. By no means it should be treated as lost cause, though; any codebase would be quickly overwhelmed by kludges if there were no way to jettison them.

To reconcile those two opposing needs – compatibility and cleanliness – the typical approach involves a transition period. During that time, the particular piece of API shall be marked as deprecated, which is a slightly theatrical term for ‘obsolete’ and ‘not intended for new code’. How effective this is depends strongly on target audience – for publicly available APIs, someone will always wake up and start screaming when the transition period ends.

For in-project interfaces, however, the blow may be effectively cushioned by using certain features of the language, IDE, source control, continuous integration, and so on. As an example, Java has the @Deprecated annotation that can be applied to functions or classes:

  1. public class Foo {
  2.     /**
  3.      * @deprecated Use FooFactory instead
  4.      */
  5.     @Deprecated
  6.     public static Foo create() {
  7.         return new Foo();
  8.     }
  9. }

If the symbol is then used somewhere else, it produces a compiler warning (and visual cue in most IDEs). These can be suppressed, of course, but it’s something you need to do explicitly through a complementary language construct.

So I had this idea to try and add similar mechanism to Python. One part of it is already present in its standard library: we have the warnings module and a built-in category of DeprecationWarnings. These can be ignored, suppressed, caught or even made into errors.
They are also pretty powerful, as they allow to deprecate certain code paths and not just symbols, which can be useful when introducing new meanings for function parameters, among other things. At the same time, it means using them is irritatingly imperative and adds clutter:

  1. class Foo(object):
  2.     def __init__(self):
  3.         warnings.warn("Foo is deprecated", DeprecationWarning)
  4.         # ... rest of Foo constructor ...

And in this particular case, it also doesn’t work as intended, for reasons that will become apparent later on.
What we’d like instead is something similar to annotation approach that is available in Java:

  1. @deprecated
  2. class Foo(object):
  3.     # ...

Given that the @-things in Python (decorators, that is) are significantly more powerful than the Java counterparts, it shouldn’t be a tough call to achieve this…

Surprisingly, though, it turns out to be very tricky and quite arcane. The problems lie mostly in the subtle issues of what exactly constitutes “usage” of a symbol in Python, and how to actually detect it. If you try to come up with a few solutions, you’ll soon realize how the one that may eventually require walking through the interpreter call stack turns out to be the least insane one.

But hey, we didn’t go to the Moon because it was easy, right? ;) So let’s see how at least we can get started.

Tags: , , , , ,
Author: Xion, posted under Programming » Comments Off on Deprecate This

Decorated Functions in C++… Almost

2013-04-28 21:56

Many languages now include the concept of annotations that can be applied to definitions of functions, classes, or even variables and fields. The term ‘annotation’ comes from Java, while other languages use different names: attributes (C#), decorators (Python), tags (Go), etc.
Besides naming disparities, those features also tend to differ in terms of offered power and flexibility. The Python ones, for example, allow for almost arbitrary transformations, while Go tags are constrained to struct fields and can only consist of text labels. The archetypal annotations from Java lie somewhere in between: they are mostly for storing metadata, but they can also be (pre)processed during compilation or runtime.

Now, what about C++? We know the language has a long history of lacking several critical features (*cough* delegates *cough*), but the recent advent of C++11 fixed quite a few of them all at once.
And at first sight, the lack of annotation support seems to be among them. New standard introduces something called attributes, which appears to fall in into the same conceptual bucket:

  1. [[dllexport]] void SomeFunction(int x);

That’s misleading, though. Attributes are nothing else than unified syntax for compiler extensions. Until C++11, the job of attributes were done by custom keywords, such as __attribute__ (GCC) or __declspec (Visual C++). Now they should be replaced by the new [[squareBracket]] syntax above.
So there is nothing really new about those attributes. At best, you could compare them to W3C deciding on common syntax for border-radius that forces both -webkit-border-radius and -moz-border-radius to adapt. Most importantly, there is no standard way to define your custom attributes and introspect them later.

That’s a shame. So I thought I could try to fix that, because the case didn’t look completely lost. In fact, there is a precedent for a mainstream language where some people implemented something-kinda-almost-like annotations. That language is JavaScript, where annotations can be realized as functions arranged into a pipeline. Here’s an example from Express framework:

  1. app.get('/home', [loginRequired, cached({minutes:30})], function(req, res){
  2.     res.render('home');
  3. });

Both loginRequired and cached(...) are functions, tied together by app.get with the actual request handler at the end. They “decorate” that handler, wrapping it inside a code which offers additional functionality.

But that’s JavaScript, with its dynamic typing and callback bonanza. Can we even try to translate the above code into C++?…
Well yes, we actually can! Two new features from C++11 allow us to attempt this: lambda functions and initializer lists. With those two – and a healthy dose of functional programming – we may achieve at least something comparable.

Alternative @property Syntax

2012-12-19 22:04

As you probably know very well, in Python you can add properties to your classes. They behave like instance fields syntactically, but under the hood they call accessor functions whenever you want to get or set the property value:

  1. import os
  3. class Directory(object):
  4.     """Simple class representing a directory in the file system."""
  5.     def __init__(self, path):
  6.         self.path = path
  8.     @property
  9.     def parent(self):
  10.         """Parent directory."""
  11.         return Directory(os.path.join(self.path, os.pardir))
  13. # usage: no () after .parent
  14. home_dir = Directory('/home/xion')
  15. root_dir = home_dir.parent.parent

Often – like in the example above – properties are read-only, providing only the getter method. It’s very easy to define them, too: just stick a @property decorator above method definition and you’re good to go.

iGet, iSet

Occasionally though, you will want to define a read-write property. (Or read-delete, but those are very rare). One function won’t cut it, since you need a setter in addition to getter. The canonical way Python docs recommend in such a case (at least since 2.6) is to use the @property.setter decorator:

  1. class TracedObject(object):
  2.     """Object that tracks changes to its properties."""
  3.     def __init__(self):
  4.         self.changed = False
  6.     @property
  7.     def x(self):
  8.         return self._x
  10.     @x.setter
  11.     def x(self, value):
  12.         self._x = value
  13.         self.changed = True

Besides that I find it ugly to split a single property between two methods, this approach will annoy many static code analyzers (including PEP8 checker) due to redefinition of x. Warnings like that are very useful in general, so we certainly don’t want to turn them off completely just to define a property or two.

So if our analyzer doesn’t support line-based warning suppression (like, again, pep8), we may want to look for a different solution.

Tags: , ,
Author: Xion, posted under Programming » 2 comments

Decorators with Optional Arguments in Python

2011-12-13 18:34

It is common that features dubbed ‘syntactic sugar’ are often fostering novel approaches to programming problems. Python’s decorators are no different here, and this was a topic I touched upon before. Today I’d like to discuss few quirks which are, unfortunately, adding to their complexity in a way that often doesn’t feel necessary.

Let’s start with something easy. Pretend that we have a simple decorator named @trace, which logs every call of the function it is applied to:

  1. @trace
  2. def some_function(*args):
  3.     pass

An implementation of such decorator is relatively trivial, as it wraps the decorated function directly. One of the possible variants can be seen below:

  1. def trace(func):
  2.     def wrapped(*args, **kwargs):
  3.         logging.debug("Calling %s with args=%s, kwargs=%s",
  4.                       func.__name__, args, kwargs)
  5.         return func(*args, **kwargs)
  6.     return wrapped

That’s pretty cool for starters, but let’s say we want some calls to stand out in the logging output. Perhaps there are functions that we are more interested in than the rest. In other words, we’d like to adjust the priority of log messages that are generated by @trace:

  1. @trace(level=logging.INFO)
  2. def important_func():
  3.     pass

This seemingly small change is actually mandating massive conceptual leap in what our decorator really does. It becomes apparent when we de-sugar the @decorator syntax and look at the plumbing underneath:

  1. important_func = trace(level=logging.INFO)(important_func)

Introduction of parameters requires adding a new level of indirection, because it’s the return value of trace(level=logging.INFO) that does the actual decorating (i.e. transforming given function into another). This might not be obvious at first glance and admittedly, a notion of function that returns a function which takes some other function in order to output a final function might be – ahem – slightly confusing ;-)

But wait! There is just one more thing… When we added the level argument, we not necessarily wanted to lose the ability to invoke @trace without it. Yes, it is still possible – but the syntax is rather awkward:

  1. @trace()
  2. def some_function(*args):
  3.     pass

That’s expected – trace only returns the actual decorator now – but at least slightly annoying. Can we get our pretty syntax back while maintaining the added flexibility of specifying custom arguments? Better yet: can we make @trace, @trace() and @trace(level) all work at the same time?…

Looks like tough call, but fortunately the answer is positive. Before we delve into details, though, let’s step back and try to somewhat improve the way we are writing our decorators.

Tags: , , , ,
Author: Xion, posted under Programming » 2 comments

Udekorowane funkcje

2010-09-22 16:46

Kilkanaście dni temu opisywałem błąd, który – jak się okazało – był efektem ubocznym pewnej cechy języka Python, uznanej przeze mnie za niemal zupełnie niepotrzebną. Dla równowagi więc dzisiaj przedstawię feature, który wydaje się być bardzo przydatny – żeby nie było, że potrafię tylko krytykować ;-)

Prawdopodobnie najlepiej jest pokazać go na prostym, acz obrazowym przykładzie. Miejmy pewną funkcję na tyle dla nas istotną, że chcemy logować wszystkie jej wywołania. Wydaje się, że nic prostszego:

  1. import logging
  3. def vif(): # Very Important Function :)
  4.     logging.debug("Called function vif()")
  5.     # ...

Ciężko jednak nazwać takie rozwiązanie uniwersalnym. Nie chodzi tu jedynie o jawnie wpisaną nazwę funkcji, ale raczej o konieczność dodawania logging.debug(...) na początku każdej funkcji, jaką chcemy monitorować. Sprawa komplikuje się jeszcze bardziej, gdy interesują nas też wyjścia z funkcji; wówczas jedynym wyjściem jest chyba opakowanie całej treści w jeden wielki blok tryfinally. Rezultat na pewno nie będzie piękny :)

I tutaj właśnie z pomocą przychodzą dekoratory – ciekawa opcja języka Python, na pierwszy rzut oka przypominająca adnotacje z Javy. Podobieństwo jest jednak głównie składniowe. Udekorowana wersja naszej ważnej funkcji wygląda bowiem tak:

  1. @trace
  2. def vif():
  3.     # ...

Znak @ poprzedza tutaj nazwę dekoratora, czyli trace (ang. śledź – i bynajmniej nie chodzi o rybę ;]). Czym jednak jest ów dekorator? Otóż on sam również jest funkcją, mogącą wyglądać choćby tak:

  1. import logging
  3. def trace(func):
  4.     def _func(*args, **kwargs):
  5.         logging.debug("Calling function %s()", func.__name__)
  6.         return func(*args, **kwargs)
  8.     return _func

Jej jedynym argumentem jest w założeniu funkcja, zaś rezultatem wywołania jest… również funkcja :) A zatem nasz dekorator potrafi przekształcić jedną funkcję w drugą, a dokładniej w jej udekorowaną, “opakowaną” wersję. To opakowanie definiowane jest wewnątrz dekoratora i polega, jak widać, na poprzedzeniu wywołania oryginalnej funkcji zapisem do loga.
Oczywiście za to, aby wywołanie funkcji opatrzonej dekoratorem było tak naprawdę odwołaniem do jej udekorowanej wersji odpowiada już sam język. Nie jest to zresztą skomplikowane, bo w istocie cały mechanizm jest tylko cukierkiem składniowym. Jest to jednak bardzo smaczny cukierek, który potrafi wydatnie podnieść czytelność kodu – jeśli stosuje się go właściwie.

Do czego jednak – oprócz logowania wywołań – dekoratory mogą się przydać? Ano przede wszystkim do upewniania się, że pewne konieczne warunki wstępne dla funkcji są spełnione na jej wejściu (i ewentualnie wyjściu). Może to być na przykład:

  • @connected – sprawdzenie połączenia z serwerem zanim spróbujemy wymieniać z nim dane i nawiązanie go w razie potrzeby
  • @authorized – określenie uprawnień wymaganych u aktualnie zalogowanego użytkownika przed wywołaniem funkcji wykonującej potencjalnie niebezpieczną operację
  • @synchronized – zabezpieczenie wywołania funkcji semaforem lub sekcją krytyczną

Wspólną cechą takich dekoratorów jest to, że są one swoistymi pseudodeklaracjami, nieodległymi koncepcyjnie zbyt daleko od komentarzy w rodzaju:

  1. # Ta funkcja wymaga połączenia z serwerem!
  2. def do_something():
  3.     # ...

Ich przewagą jest jednak rzeczywiste sprawdzanie, czy wymagania zostały spełnione – i to w sposób automatyczny i przezroczysty. Według mnie to właśnie stanowi o ich sporej przydatności.

Tags: , ,
Author: Xion, posted under Programming » 9 comments

© 2017 Karol Kuczmarski "Xion". Layout by Urszulka. Powered by WordPress with