Tag programming

APIs should have test error endpoints

Most of the time, when you start coding against an API, either with or without an API client, you end up producing enough errors that your error-checking is fairly robust. However, sometimes most or even everything “just works.” Which produces a conundrum:

You now have no idea how robust your error handling is (or is not).

As an example, that – ahem – may or may not have just inspired this blog post:

Say you are polling something to check for an update, every 60 seconds or so. If it fails, you want to know that and decide whether or not you should keep polling the endpoint. Everything works great. Until you get a 500 which you haven’t handled correctly (or there was a bug in your logging call or… something, ahem). Now you have an exception in your own code which breaks out of your loop independently of your API error handling. So your polling has stopped and you’ve cutoff your own ability to see what went wrong on the other end.

What I propose, is that we as API providers should include error test endpoints. Something like:

/tests/errors/500
/tests/errors/503
etc

(note: 404 is obviously easy)

In theory RESTful APIs would all return errors in the same way with HTTP error codes and the like, but the practical reality is that this just isn’t the case. Some even return 200s with JSON strings that include errors. Even if they we’re all perfect, you still might want to wrap them in behaviors (like sending an email notification) depending on the message contents, which really leaves you in a predicament if the errors are hard to intentionally produce. You can even make a compelling argument that API client development should start with these endpoints.

I’ve opened an Issue to add these endpoints for YourTrove’s API and I will follow this up with a blog post on the Trove blog when those endpoints are deployed.

Python logging tutorial | Pingbacks

This is the easiest-to-follow of the various Python logging guides/posts/docs that I’ve seen:

Python logging tutorial | Pingbacks.

A few great things to read on REST (technical)

I’m not sure how I missed this post by Jacob Kaplan-Moss, where he’s throwing the kind of REST question out there that has, in the past kept me thinking for hours:

It seems like URIs like /people/{my-uid}/photos and /people/{my-uid}/photos/{photo-id} are more “pure.” But now that’s weird because only one single user ever has access to a given URI (e.g only user #7 gets to access the entire space under /people/7). And the information in the URI is redundant with the information in the Authorization header.

http://www.jacobian.org/writing/rest-wankery-question/

Then things get really interesting in the comments, with links to two great posts (which I also missed):

The last constraint is incredibly simple, but nobody actually does it. It’s named Hypertext As The Engine Of Application State. I still haven’t decided how to pronounce the acronym, I always try to say “Hate ee ohs,” which sounds like a breakfast cereal. Anyway, let’s break this down. We’re using Hypertext, fine, that makes sense. But what’s it mean to be an engine? And application state?

Now, when I said ‘nobody’ does this, what I meant was ‘for APIs.’ This is exactly how the Web works. Think about it. You start off on the homepage. That’s the only URL you have to know. From there, a bunch of links point you towards each state that you can reach from there. People would consider it ludicrous if they had to remember a dozen URLs to navigate a website, so why do we expect the consumers of our APIs to do so as well?

Haters gonna HATEOAS

Finally, comes an elegant, much more RESTful solution to the API version dilemma:

You can simply define a new media type – sayapplication/vnd.mycompany.myapp-v2+xml – and associate new multi-email format with it. Clients can then request whichever format they want. Older clients don’t know the new media type so they get served the older single email format.

Newer clients do know the new media type so they can have access to the new functionality.

VERSIONING REST WEB SERVICES

All three of these posts/discussions are worth reading, but if you only read one, read Steve Klabnik’s HATEOAS post.

Hammurabi was the first programmer

Or someone working for him was. I was having dinner last night with my cousin (an SAP consultant) and we covered a range of topics, including lobbying and programming languages. And it struck us that laws are essentially programs.

Prologue of Hammurabi's Code, in the Louvre

Either you’ve already heard this analogy or you might be going “what??!?!” But it’s true. Laws and programs are both essentially linguistic expressions of applied logic: rules and processes, codified.

A program is a law, the data is a case, and the computer is a very, very strict judge. The computer really doesn’t care about intent, it’s going to do exactly what the program says.

So, we can think of Hammurabi’s Code (even the word’s the same!) as saying stuff like this:

if convict.crimes.theft == True:
convict.hand.delete()

Of course there are differences and laws are at a bit of a disadvantage when it comes to version control and testing, but as a way for non-programmers to think of programming, you could do a lot worse than thinking of it as very strict laws.

My answer to What are some of the technology innovations we might see in online advertising in 2011?

I think the two huge things that technology is changing in online advertising are 1) targeting. And 2) what I am starting to think of as “applitisements”, which is to say: display and mobile ads are going to become more and more like mini web applications. Neither of these things is a singular, momentary “innovation” in its own right, nor do I think as trends that they started or will end in 2011, but I think 2011 is when they start really coming together beyond the clumsy first steps.

Targeting is a bit more obvious and already on more people’s radar. As users share and connect more of their data, advertisers are going to be able to hyper-target and personalize their ad buys (think “males 30-40 who follow @mybrand on twitter and live in australia”). I think most people in the advertising world are aware of this at a conceptual level, but 2011 is when you’re going to really start seeing more effective targeted ads.

Applitisements, in the strictest technical sense, have been with us since the first Flash banner, but as online identities become more cross-site pervasive and HTML5 and integration APIs grow (see iAd), online ads are going to become much, much more robust and – hopefully :) – better.

Just to take a really simple example off the top of my head: Imagine you’re watching a movie trailer online. At the end of the trailer, not only are you provided with the “theaters closest to you”, but the applitisement already knows which theaters you buy tickets to the most often and displays those first, it lets you invite other people to go with you and buy the tickets without having to login or enter your CC info (ideally, there’s an in-ad password entry or other auth “reverify”). No jumping between websites, pages or windows or copying and pasting. All within the ad. And it doesn’t matter if you’re viewing the ad on a phone or a desktop, it just works.

Now, I’m not saying you’ll see the above example “this” year, but 2011 is the first year where I can actually say to myself “Okay, if I were going to write that ad, I need X, Y, and Z to happen and I have to do A, B, and C” and none of those things feel like some nebulous far-off “someday.” The pieces are all starting to come together.

What are some of the technology innovations we might see in online advertising in 2011?

Python 2.5 distutils, snow leopard and xcode making gcc happy

As many Python folks have discovered, upgrading to Snow Leopard can cause some pain for development. For the most part, this involves reinstalling a bunch of things (macports, python itself and whatever python packages you use). This has been a hassle for me, but up until today it was just time consuming, rather than actually difficult. Apparently if you upgrade xcode (like I did for the new iphone stuff), unless you explicitly choose the old SDK, xcode blows it away.

So today, while trying to compile multiprocessor on Python 2.5, I got this:

Compiling with an SDK that doesn’t seem to exist: /Developer/SDKs/MacOSX10.4u.sdk
Please check your Xcode installation

Which left me staring at the screen thinking “how the f**k do I fix THAT?” After poking through every file in multiprocessor and then googling quite a bit I finally found this message on the epd-users mailing list, which got me to the promised land.

From what I can gather there are two ways to fix this. One is to reinstall xcode and choose to include the older SDK. The other (and possibly scarier, depending on your taste for mucking around inside installed stuff) is to point distutils at the newer SDK. I did the latter: in a text editor, open up:

/Library/Frameworks/Python.framework/Versions/Current/lib/python2.5/config/Makefile

Then find all the instances of MacOSX10.4u.sdk and change that to something you do have installed. To see them just do

$ ls -la /Developer/SDKs/

If there’s nothing there, well, then you need to install xcode or you’re SOL. There should be four places to make the change in the Makefile. Finally, the newer compiler doesn’t seem to like the “-Wno-long-double” flag, so I had to the delete that as well. And voila, multiprocessor compiled (and I hope any other Python package that I try to install that needed the gcc).

Okay, I’m fiddling with Scala next

I’m sold on trying out Scala, after seeing James Strachan state:

I’m very impressed with it! I can honestly say if someone had shown me the Programming Scala book by by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I’d probably have never created Groovy.

Strachan created the very cool Groovy and is one of the few big Java guys (along with Jerome Louvel) who has some good perspective on the language and has been doing cool stuff.

I even have a couple of small projects I’m fiddling with where I can try it out. For client work I will still be using Python and straight up Java for the time being.

Fiddling with neo4j

So I’ve been fiddling quite a bit with neo4j for an idea I had. I can say I definitely like how fast and intuitive it is to use. I’m a bit worried about storage. I created 1000 nodes with 100 relationships each and my database folder ballooned to 250 megs. That isn’t huge, but it is quite a bit larger than I expected. I’m new to the framework, so I may have done something wrong in that test.

Qi4j

Qi4j is a new framework I’ve been looking at. As my friend Nick put it, “it’s mixins for Java!” which is, well, true; but here’s the description from the site:

Qi4j is a framework for domain centric application development, including evolved concepts from AOP, DI and DDD.

Qi4j is an implementation of Composite Oriented Programming, using the standard Java 5 platform, without the use of any pre-processors or new language elements. Everything you know from Java 5 still applies and you can leverage both your experience and toolkits to become more productive with Composite Oriented Programming today.

Next question is “what is composite oriented programming” right?

Many objects has life cycles that are more extensive than the simple model that Object Oriented Programming model wants us to believe. A few simple examples;

  • An egg becomes a chicken which in turn becomes food.
  • I am a programmer at work, a father+husband at home, a victim in a traffic accident and hunter and pray in the jungle.

But it is more to it than that. The composition of the object may change over time. My home now has a garage and my car have different kind of problems with their own state related to it.
In the programming world, we are constantly faced with change of requirements. These changes are often not related to any real world changes, but people coming to new insights of the problem domain. OOP makes those changes a big deal, and often we have to tear up large chunks of the model and redo the work.

But wait, there is more.

Some objects traverses different scope boundaries to the extreme. For instance, a Person will have its attributes changing slightly over time, new abilities be learnt and so forth, that is mentioned above. But the Person will eventually die, but that doesn’t mean that the Person object should be deleted from a system, since the “memory of” that Person may live on for a long time. In a OOP system, we would need to transfer some of the state from a LivingPerson class to a DeadPerson class. In Composite Oriented Programming, it is the same object with different behavior.

We think that one of the the main flaws in OOP is that it is not object oriented at all, but in fact class oriented. Class is the first class citizen that objects are derived from. Not objects being the first-class citizen to which one or many classes are assigned.

I can’t wait to play with this, but I have not yet had time… :(