Making Flask’s url_for More Flexible

Flask is one of the countless web frameworks available for Python. It’s probably my favorite, because it’s rather minimal, simple and easy to use. All the expected features are there, too, although they might not be as powerful as in some more advanced tools.

As an example, here’s how you define some simple request handler, bound to a parametrized URL pattern:

This handler responds to requests that go to /post/42 and similar paths. The syntax for those URL patterns is not very advanced: parameters can only be captured as path segments rather than arbitrary groups within a regular expression. (You can still use query string arguments, of course).

On the flip side, reversing the URL – building it from handler name and parameters – is always possible. There is a url_for function which does just that. It can be used both from Python code and, perhaps more usefully, from HTML (Jinja) templates:

Parameters can have types, too. We’ve seen, for example, that post_id was defined as int in the URL pattern for blogpost handler. These types are checked during the actual routing of HTTP requests, but also by the url_for function:

Most of the time, this little bit of “static typing” is a nice feature. However, there are some cases where this behavior of url_for is a bit too strict. Anytime we don’t intend to invoke the resulting URL directly, we might want a little more flexibility.

Biggest case-in-point are various client-side templates, used by JavaScript code to update small pieces of HTML without reloading the whole page. If you, for example, wanted to rewrite the template above to use Underscore templates, you would still want url_for to format the blogpost URL pattern:

Assuming you don’t feel dizzy from seeing two templating languages at once, you will obviously notice that '< %= post.id %>' is not a valid int value. But it’s a correct value for post_id parameter, because the resulting URL (/post/< %= post.id %>) would not be used immediately. Instead, it would be just sent to the browser, where some JS code would pick it up and replace the Underscore placeholder with an actual ID.

Unfortunately, bypassing the default strictness of url_for is not exactly easy.

When and What to Bash

Often I advocate using Python for various automation tasks. It’s easy and powerful, especially when you consider how many great libraries – both standard and third party – are available at your fingertips. If asked, I could definitely share few anecdotes on how some .py script saved me a lot of hassle.

So I was a bit surprised to encounter a non-trivial problem where using Python seemed like an overkill. What I needed to do was to parse some text documents; extract specific bits of information from them; download several files through HTTP based on that; unzip them and place their content in designated directory.

Nothing too fancy. Rather simple stuff.

But then I realized that doing all this in Python would result in something like a screen and a half of terse code, full of tedious minutiae.
The parsing part alone would be a triply nested loop, with the first two layers taken by os.walk boilerplate. Next, there would be the joys of urllib2; heaven forbid it turns out I need some headers, cookies or authentication. Finally, I would have to wrap my head around the zipfile module. Oh cool, seems like some StringIO glue might be needed, too!

Granted, I would probably use glob2 for walking the file system, and definitely employ requests for HTTP work. And thus my little script would have external dependencies; isn’t that making it a full-blown program?…

Hey, I didn’t sign up for this! It was supposed to be simple. Why do I need to reimplement grep and curl, anyway? Can’t I just…

…oh wait.

Everybody Should Learn to Code!… Wait, What?

You may have seen this video, starring Mark Zuckerberg, Drew Houston (from Dropbox) and a couple of other tech stars. They promote the Code.org initiative, advocating that every school kid in the U.S. should have programming classes as part of their educational offer.

It’s a recent development, but the whole idea of promoting computer programming among broad audience is in the spotlight for quite some time now. Not only in the U.S., mind you. In Estonia, for example, children are to be offered coding classes starting from early elementary school.

The message seems to be pretty clear: everyone should learn to code, the earlier the better. In the future, it will be an obvious part of basic literacy, like reading and writing natural language. Everyone will get to do some really cool and useful things with their computers.

Sounds great, doesn’t it?…

So why I can’t help but to roll my eyes when I hear about this? Am I just an obnoxious, elitist jerk who doesn’t want others to experience the feeling of empowerment and satisfaction that creating your own computer programs can evoke every so often?… D’oh, I really hope not.

So maybe there are some actual problems with this idea. I would hesitate to say that it just doesn’t, ahem, compute, but I can certainly see a few issues, misconceptions and dubious assumptions.

Slides from My Talk at Name Collision

Last Friday I had the pleasure of attending the Name Collision hackathon in Warsaw. I haven’t actually taken part in the competition but I served as a mentor, trying to help the teams get stuff done as much I could. It was really fun experience.

Before that, though, I gave a short lightning talk about the Git version control systems. I showed a few (hopefully) lesser known tricks that may prove useful both in everyday workflow and in some exceptional situations.
The slides from this talk are available here. They might be a bit uninformative on their own but I suppose the liberal use of memes can make up for this somewhat ;-)

Ego-Driven Technologies

Last week – while still on the other side of the pond – I attended a meet-up organized by the local Google Developers Group. The meeting included a presentation about Go, aimed mostly at newcomers, which covered the language from the ground up but at very fast pace. This spurred a lot of survey questions, as people evidently wanted to assess the language’s viability in general and fitness for particular domain of applications.
One of them was about web frameworks that are available to use in Go. Answer mentioned few simple, existing ones, but also how people coming from other languages are working to (re)build their favorite ones in Go. The point was, of course, that even though the language does not have its own Django or Rails just yet, it’s bound to happen quite soon.

And that’s when it dawned on me.

See, I wondered for a while now why people are eager to subject themselves to a huge productivity drop (among other hardships) when they switch from one technology, that they are proficient in, to a different but curiously similar one.
Mind you, I’m not talking about exploratory ventures intended to evaluate language X or framework Y by doing one or two non-trivial projects; heck, I do it very often (and you should too). No, I’m talking about all-out switching to a new shiny toy, especially when decided consciously and not through a gradual slanting, in a kind of “best tool for the job” fashion.

Whenever I looked for justification, usually I’d just find a straightforward litany of perks and benefits of $targetTechnology, often having a not insignificant intersection with analogous list for the old one. Add the other, necessary part of risk-benefit calculation – drawbacks – and it just doesn’t balance out. Not by a long shot.

So, I notice that I am confused. There must some be other factor in play, but I couldn’t come up with any candidates – until that day.
As I speculate now, there is actually a big incentive to jump ship whenever a new one appear. And it seems to be one of dirty secrets of the hacker community, because it directly questions the esteemed notion of meritocracy that we are so eager to flaunt.

Now That’s What I Call a History Lesson

“Museums are boring!” If you hold such an opinion, it’s quite likely you haven’t been in those really interesting ones yet. Today (or yesterday, depending on PoV) I happened to be in such a place, and I can assert its awesomeness with unparalleled confidence.

It’s the Computer History Museum in Mountain View, CA. If you ever find yourself in northern California, I definitely recommend paying a visit there.
Inside, you can find all sorts of extremely cool exhibits from the somewhat short but wildly interesting history of computer science. Anything from early vacuum tubes, through Commodores and up to iPhones, with all the surprising stuff in between.

I couldn’t spend a whole day there, sadly, but I took a few pictures. Apologies for the low quality of some; although I took them with a device that won’t land in this museum for quite a while, it’s still just a phone.

Complex Stubbing with Mockito for Java

Mocks are used to “fake” the behavior of objects to the extent required by tests. Most of the time, they can be pretty simple: when calling a method with given set of (constant) argument, then return some predefined value. Mocking frameworks actually strive to translate the previous sentence directly into code, which can be seen in Java’s Mockito library:

What is evident here is that this mock doesn’t maintain any state. As a result, all stubbed method are completely independent and referentially transparent: they always return the same value for the same set of arguments. In reality, that property is even stronger because they are simply constant.

Simple mocks like those work for a good number of cases. Some may even argue that it’s the only way to mock properly, because anything more complicated is a sign of code smell. We can counter that by saying that some complexity is often unavoidable, and it’s up to developers to decide whether they want to put it test or tested code.

And thus more complex stubbing may be needed if, for some case, you prefer the former approach. When that happens, it’s good to know how to fabricate a bit more sophisticated behavior. In Mockito, it’s very possible to achieve almost arbitrary complexity by capturing arguments of stubbed calls and routing them to callbacks. This is done with the help of ArgumentCaptor class and the Answer interface, respectively.
When combined, they can result in pretty clever mocks, as illustrated with this example of some imaginary API for SMTP (e-mail sending protocol):

Note that it doesn’t implement the real SMTP. That, presumably, would mimic the SmtpConnection class exactly, which of course is not the point of mocking at all.
The point is to provide sufficiently complete behavior so that code under test can proceed, reaching all the execution paths we want to exercise. In this case, something can call getOutgoingClientData and rely on certain string being present in the output.

Let’s just hope that this “something” actually has some good reasons to expect that…