Semantic Thoughts

We are currently implementing a prototype for event-based access to media. The basic idea is to organize media like videos or images around the events they were taken at. Examples are videos of a concert or images from some art exhibition. The data is stored as RDF, all URIs are dereferencable and return RDF/N3 by default. However, in order to make use of the dataset easier, we also want to provide a RESTful API that returns JSON representation of the resources and provides additional resources for performing different searches over the dataset. Furthermore, the RESTful API is also supposed to allow modifications of the data.

Now, the RESTful API and the frontend will run on different servers, and we also want to enable mash-ups, so we actually need to allow cross-domain AJAX. However, that is prohibited by most modern browsers. As long as you only want to do GET requests, one way around this issue is JSONP, as explained here.

We require, however, POST requests (and actually also PUT or DELETE). However, JSONP is basically a hack that works only with GET. Some research into this issue revealed that there is actually one way round it, called Cross-Origin Resource Sharing. CORS was proposed by the W3C. An explanation with nice code examples can be found in this blog entry.

The basic idea is that the client sends a header that specifies the Origin of the request, i.e., where the code doing the request is located. The server checks this header and returns some additional headers, most notably the list of origins that are allowed to access the resource in a cross-domain fashion. This approach works for simple GET and POST requests (some standard requests, the details are explained in the proposal).

If the request is non-standard, as in our case, a more complicated mechanism is established. Actually, the client first sends an OPTIONS request with some extra headers, actually asking the server whether it is allowed to do its request or not. The server then answers, again with some special headers, and either allows or denies the anticipated request. Implementing this is pretty easy, once you know the pitfalls. What I didn’t know is that the some headers you send in response to the OPTIONS also have to be sent in response to the POST. Basically, this is contained in the original proposal, but not very explicit for the specific case I was working on. So I missed that detail, which cost me like half a day. Of course, there were no proper error messages and the only observable effect was en empty response body in Firefox. For this kind of debugging, FireBug is a really nice tool. In addition, Firefox and Chromium did behave differently, what didn’t make finding the problem easier.

I’m now going to show what needs to be done on the client side with jQuery and on the server side using Restlet. Assume that our front-end lives on some domain http://allevents.example.com and the RESTful API on http://eventapi.example.org. Despite the similarity of both URLs, they are in fact different Doing a POST request with jQuery is fairly simple:

 $.ajax({
    type: "POST",
    dataType: "json",
    contentType: "application/json",
    url: "http://eventapi.example.com/event",
    data: {starttime:\"2008-12-01\", endtime:\"2010-12-02\", limit:10, offset:10},
    success: function(response, status, xhr){
        $("#text").append(response.events[0]);
    }
});

We use the ajax function in order to do a POST request. We specify with dataType that we expect JSON as the response and with the contentType that we are also sending JSON. The URL POSTed to is http://eventapi.example.com/event. The data is a simple JSON object containing a start, an end date, and some pagination information. Actually, this is supposed to be a search request for all events that took place in the given time interval. Obviously, we could realize this as a GET, but in reality the search parameters might be much more complex and much longer, so that GET is a bad choice du to some length restrictions present in different clients and servers. The success field is bound to a callback function that just appends the first event to some html element.

But due to the different domains the frontend (and thus the AJAX) is served from and the POST is directed at, the client will not perform the POST without some extra effort. The client side extra effort is handled by the browser, so we are finished with that. On the server side, the RESTful API has to implement some extra methods. What actually will happen is the following. The browser will send an OPTIONS request that will more or less look like:

Origin: http://allevents.example.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-Type

In reality there will be some additional headers, but these are the ones actually implementing CORS – Cross-Origin Resource Sharing. The cliet actually says: Hey, I running content from http://allevents.example.com, specified in the Origin: header, I want to POST (Access-Control-Request-Method), and I will send an extra header Content-Type (Access-Control-Request-Headers). The latter is required, since we don’t send any default content, but JSON. This is specified using the Content-Type header, which is not among the default headers allowed.

In Restlet we can now answer this OPTIONS request:

@Options
public void doOptions(Representation entity) {
    Form responseHeaders = (Form) getResponse().getAttributes().get("org.restlet.http.headers");
    if (responseHeaders == null) {
        responseHeaders = new Form();
        getResponse().getAttributes().put("org.restlet.http.headers", responseHeaders);
    }
    responseHeaders.add("Access-Control-Allow-Origin", "*");
    responseHeaders.add("Access-Control-Allow-Methods", "POST,OPTIONS");
    responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
    responseHeaders.add("Access-Control-Allow-Credentials", "false");
    responseHeaders.add("Access-Control-Max-Age", "60");
}

Here we actually create a response with only headers, so no response body is transmitted. The first header says any domain might access the API (responseHeaders.add("Access-Control-Allow-Origin", "*");). In the next line we say POST and OPTIONS are allowed HTTP methods. We allow the extra header as requested. We don’t allow any credentials to be sent, and we actually allow the client to cache this response for 60 seconds. That means in the next 60 seconds, the OPTIONS request will not be send again.

Now, when the response indicates that the anticipated POST is allowed by the server (which is the case here) the browser will do the actual POST request. On the server side it is now important to include the Access-Control-Allow-Origin header from the OPTIONS response again, or the POST response will not correctly be handled. The actual behavior with Firefox was that the response body was empty. Content-Length and other headers were retrieved correctly, but the response body was empty, although the server did send a body. Again, the Restlet code:

@Post
public Representation acceptEvent(Representation entity) {
    Form responseHeaders = (Form) getResponse().getAttributes().get("org.restlet.http.headers");
    if (responseHeaders == null) {
        responseHeaders = new Form();
        getResponse().getAttributes().put("org.restlet.http.headers", responseHeaders);
    }
    responseHeaders.add("Access-Control-Allow-Origin", "*");
    Representation result = null;

    // generate and return JSON
    return new StringRepresentation(generateJSON(), MediaType.APPLICATION_JSON);
}

The important part is at the top of the method, where we add the additional headers to the response. The rest is a normal Restlet function that returns some JSON.

I hope that this small blog posts clarifies things for some people having a similar problem.