Sep
4

Clarifying REST

By kellabyte  //  Architecture  //  37 Comments

For many REST is a vastly misunderstood term. The experts in the REST domain know this all too well. To the point they are thinking of renaming it to separate themselves because the confusion is out of control and impossible to re-align with the true definition.

Having an HTTP server accept GET requests or POST and responding with XML, JSON or some other data format is not automatically RESTful. This statement will seem very odd to many reading this.

Unfortunately many of the HTTP-enabled API’s out there such as Twitter, who promote themselves as REST, have created confusion on what REST really is.

The definition of REST lost its way when popular services emerged calling themselves REST which were not. An API like Twitter is ignoring the values of the web and the browser by using HTTP as a transport for RPC (Remote Procedure Call). SOAP services also use HTTP in this way. Instead of doing RPC, HTTP should be used to transfer application state which is what REST is designed for.

It’s not bad for a service to not be RESTful if the problem needing a solution calls for it. Each situation is different and REST has it own set of advantages and disadvantages to consider. The problem is the term REST is being used where it does not belong.

REST constraints

Roy Fielding, one of the principal authors of the HTTP 1.0 and 1.1 specifications introduced REST in 2000 in his doctoral dissertation.

Conforming to the constraints for REST from Roy’s dissertation is referred to as being RESTful. As pointed out earlier these terms are used improperly in cases where the constraints are violated. Simply having a HTTP service that responds with JSON or XML does not make a RESTful service.

Client-Server constraint

Separation of concerns is the purpose of the client-server constraint which is intended to separate user interface concerns from data storage concerns. With the popularity of HTTP-enabled API’s being an integration point for online services the separation of the client and server was already chosen since the client is typically a 3rd party.

Stateless constraint

The client-server interaction must be stateless in nature. Each request from the client to the server must contain all of the data that is necessary to handle the request. Tackling concurrency and scalability becomes drastically easier when you don’t have to transition state between servers and handle scenarios when state has been mutated in multiple locations. Be careful not to confuse state and resources (data).

As Roy mentions in his dissertation making a choice to adopt an architecture style always has trade-offs. One disadvantage of being stateless is the repetition of data sent in a series of requests.

This constraint in my opinion is one of the first ones violated in self proclaimed REST services. Having the server retain state also has its advantages which for one service may make total sense.

Cache constraint

The data within a response to a request must be implicitly or explicitly labeled as cacheable or non-cacheable. The advantage of this constraint reduces partially or completely client to server interactions for a request or series of requests. Gains come in many facets such as user perceived performance and reliability (the service isn’t always required). Data reliability is a trade-off however.

Because of the stateless constraint, understanding when a cache is invalid is easier to identify because you don’t have any cases where server-side state has been mutated on one server but not others and consistency is easily achieved.

Layered system constraint

The layered constraint was added to address improving internet sized scalability requirements. Each layer cannot see beyond the immediate layer with which it is communicating with. This places boundaries on the overall complexity of the system.

Like most solutions for scalability there are usually trade-offs. Latency is increased with the introduction of layers but the cache-constraint above can certainly help reduce the amount of requests over the network.

I would imagine that many would still consider your service RESTful if you bent the rules on this constraint and solved scalability differently. From the outside your clients could interpret your service as being completely RESTful even if you did this differently.

One has to wonder what other constraints you broke if your layers cross all sorts of boundaries. I imagine violating the stateless constraint would cause all kinds of odd topologies as you start to wrestle with the concurrency challenges that introduces.

Pandoras box opened have you? State you’ve allowed Hmm?

Code-on-demand constraint

The code-on-demand constraint allows clients to be extendable by downloading and executing code. Similarly to java script in a web browser this allows you to add functionality without re-deploying client software.

The code-on-demand constraint is optional.

Uniform interface

Information is transferred in a standardized form which is certainly a trade-off. You gain in a common standard for information but you lose in efficiency by the inability to handle a specific applications needs.

REST has 4 interface constraints. I’m not going to go through all of them but I’ll give a quick summary:

Identification of resources

Each resource has a URI and is access through a defined set of HTTP methods (GET, PUT, POST, DELETE)

Manipulation of resources through representations

Each resource can have one or more representations. Such as application/xml, application/json, text/html, etc. Clients and servers negotiate to select representation.

Self-descriptive messages

Requests and responses contain not only data but additional headers describing how the content should be handled. Such as if it should be cached, authentication requirements, etc.

Hypermedia as the engine for application state

I’m going to focus on the last interface constraint Hypermedia as the engine for application state because I think there is a lot to learn when it comes to hypermedia and why many services are not RESTful by violating this constraint.

A REST API must be hypertext driven

Hypermedia as the engine for application state is the 4th Uniform interface constraint.

Roy himself has gotten frustrated by the misuse of the term REST and blogged about it with some great info on why certain services are not RESTful.

After talking with Darrel Miller and reading Roy’s post the key awareness I came out of it was that the use of hypermedia, particularly hypertext and linking are very important.

Roy states

A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC's functional coupling].

An example of hypertext aware clients are web browsers.

In my observation many services which claim to be REST but are not exclude hypertext and requires deep knowledge of the services API to understand how to navigate the application.

Let’s look into a Twitter response and discover what is missing and what hypertext enables which is required for REST.

Not RESTful (Twitter)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
<?xml version="1.0" encoding="utf-8"?>
<statuses type="array">
<status>
<created_at>Thu Sep 01 22:19:10 +0000 2011</created_at>
<id>109389864092442624</id>
<text>Login issue on dev.twitter.com is now fixed.</text>
<source>web</source>
<user>
<id>6253282</id>
<name>Twitter API</name>
<screen_name>twitterapi</screen_name>
<location>San Francisco, CA</location>
<profile_image_url>
http://a2.twimg.com/profile_images/1438634086/avatar_normal.png
</profile_image_url>
</status>
</statuses>

The Twitter API also has API for looking at a users profile. Notice how the <user> element has no hyperlink to direct the client to the location of where the user resource is located. This means unlike a browser, a Twitter client needs to understand deep knowledge of the Twitter API to navigate to the user resource. If the Twitter API was RESTful within the response the <user> element would have a hyperlink which the client could automatically understand. The benefits of this are many and one of the most obvious ones is if Twitter changes its URI for user profiles the client would require no changes because like a browser the server response includes hyperlinks and the client understands them.

RESTful (Twitter modified)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
<?xml version="1.0" encoding="utf-8"?>
<statuses type="array">
<status>
<created_at>Thu Sep 01 22:19:10 +0000 2011</created_at>
<id>109389864092442624</id>
<text>Login issue on dev.twitter.com is now fixed.</text>
<source>web</source>
<user href="http://api.twitter.com/1/users/show.xml?screen_name=TwitterAPI">
<id>6253282</id>
<name>Twitter API</name>
<screen_name>twitterapi</screen_name>
<location>San Francisco, CA</location>
<profile_image_url>
http://a2.twimg.com/profile_images/1438634086/avatar_normal.png
</profile_image_url>
</status>
</statuses>

Notice the href in the above XML markup. With the inclusion of hypertext, a RESTful client can navigate this all by it’s own just like a browser does when it makes links clickable by the user.

It is important to point out that in the world of REST a link may have uses outside of user navigation. The client could use links to navigate the data. A link could also be used for previous or next records for example.

Ever read a multi-page article on a website that has a next page link? The browser just knows how to handle that. RESTful clients often behave the same way.

Let’s take an Atom reader for example. The Atom format supports hypertext and you can easily see the benefits over the Twitter response.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:as="http://atomserver.org/namespaces/1.0/"
xmlns:os="http://a9.com/-/spec/opensearchrss/1.1/">
<os:totalResults>65801</os:totalResults>
<os:startIndex>0</os:startIndex>
<os:itemsPerPage>2</os:itemsPerPage>
<as:endIndex>153</as:endIndex>
<link href="/myserver/v1/widgets/acme?start-index=153&amp;max-results=2" rel="next" />
<link href="/myserver/v1/widgets/acme?start-index=0&amp;max-results=2" rel="self" />
<author>
<name>AtomServer APP Service</name>
</author>
<title type="text">acme entries</title>
<updated>2007-10-05T19:17:42.750Z</updated>
<id>tag:atomserver.org,2007:v1:acme</id>
<entry>
<id>/myserver/v1/widgets/acme/205390.en.xml</id>
<as:entryId>205390</as:entryId>
<title type="text"> Entry: acme 205390.en</title>
<author>
<name>AtomServer Atom Service</name>
</author>
<link href="/myserver/v1/widgets/acme/205390.en.xml" rel="self" />
<link href="/myserver/v1/widgets/acme/205390.en.xml/2" rel="edit" />
</entry>
</feed>

Lines 9 and 10 show how Atom allows the client to understand it’s current location as well as where to navigate for the next collection of results. Atom clients can be far more dynamic than Twitter clients because of the use of hypermedia.

Line 25 you can see the inclusion of an edit link. By using the rel attribute Atom clients can automatically understand where to navigate for editing. The use of hypermedia specifically hypertext enables dynamic scenarios non-RESTful clients don’t get with responses that include no hypermedia. For this reason Hypertext is a constraint. Without including hypermedia a service is not RESTful.

Another good source of information on hypermedia is a blog post from Glenn Block. He talks about Hypermedia and forms.

RESTful Clients

When we build RESTful systems the way we build clients also changes. The clients behave very much like a browser and the clients start to benefit in the same ways that a browser does. Hypermedia unlocks many scenarios we don’t typically think of when we are accustom to building RPC-style services.

Conclusion

What should be clear after reading this is that many of the services you’ve used and interacted with which call their API a REST API don’t exhibit many of the benefits that the true definition of REST provides and violate many of the key constraints. To re-iterate this does not mean the service is poorly built or does not solve the problem set it attempts to solve. It simply means they are not RESTful. REST comes with it own set of advantages and disadvantages. I described some but not all of these in this post.

Understanding how clients could benefit from hypermedia and particularly hypertext was the “AHA” moment for me which encouraged me to look into this deeper. There is so much more to grasp and understand on this topic I’ve only begun.

I highly suggest reading Roy’s doctoral dissertation where he defines REST.

After spending all weekend on this post I’m going to get some REST myself. Oh who am I kidding…

  • Ismael

    It seems there’s something more to a RESTful service than having support for PUT and DELETE verbs, after all. I guess there was a need for labeling all those plain xml/json and http-friendly services, and REST is (or used to be) a catchy name. Nice post.

  • Darko

    I’ve always used hyperlinks in developing RESTful services, but I was never aware of how integral that is to REST itself. Nice article!

  • http://theamiableapi.com Ferenc Mihaly

    Well-written, useful summary of Fielding’s dissertation. I’m just as guilty as others of using the terms REST and RESTful in contexts that would make Fielding frown, simply because we don’t have a better word for describing web services interfaces built using HTTP as an application protocol. So even when we are perfectly aware that REST is the name of an architectural style, not a specific protocol or interface, we are contributing to the confusion. You mentioned in the introductory paragraph something about the experts thinking about renaming it. Could you share any details?

  • Gopakumar

    Good article! It seems I need to read deeper to understand the actual REST. I will need to delete my perception of REST.
    Thanks for insight!

  • Darrel Miller

    Farenc: At the recent RESTfest, during the keynote Jon Moore made the claim that the term REST had “jumped the shark” and he called for people to use the term Hypermedia API. See this video http://vimeo.com/27942428 at around 4 minutes in. It is also worth watching the entire keynote where he makes his point that if you are not doing hypermedia, conforming to the other REST constraints is not worth the cost.

    • http://theamiableapi.com Ferenc Mihaly

      Thanks for the link, Darrel. I must admit it is not easy to get the Hypermedia As The Engine of State (HATEOAS) principle. Fielding’s thesis only mentions it, but it does not explain it. The couple of books I read on REST simplify it to mean “don’t hardcode URLs into clients, pass them as links in resource representations”. This is fine for reducing coupling, but of course it does explain the “engine of state” part. Hearing Jon Moor talk about “finite state machines” made a lot more sense. At least you can Google it and get something back. “Engine of state”? Not so much! I still need to listen to the entire keynote to understand the “not worth the cost” argument, though.

      • http://blogs.msdn.com/gblock Glenn Block

        Ferenc, you might want to check out “REST in Practice” as the book covers using hypermedia for state transitions within an application.

        On the subject of the REST term, Sebastian Lambla and I had a very similar conversation at QCon London. I see the advantage of the new term (whatever it is) as a way to clear the air going forward. We’ll have to try to make it less buzzworthy so the marketeers don’t take it over :-)

        • http://theamiableapi.com Ferenc Mihaly

          Thanks, Glenn. You mean Chapter 5: Hypermedia Services, right? I’m going to read it. I’ve got the book from Safari Books Online.

  • Keith

    Very nice post. I started reading this post thinking I already knew REST and it would be a little refresher for me. This was not the case at all and will help a lot when I start writing a REST service over the coming months.

  • http://www.azulraul.com Raul Macias

    Nice post; thanks for sharing it.

  • http://timehub.net Nicolás Hock Isaza

    I really loved this post! I think one of the main problems with REST is that people es really lazy to actually read the “real” REST document.

    I have a small question though, in the modified twitter response, you placed the user href as an attribute to the tag instead of a nested element, did you have a reason for that?

    Why not using URL-HERE as Github’s API V3 does for example (same with pagination links that are returned in the headers of the response in Github API).

    Thanks and great post!

    Nicolás Hock Isaza

  • http://extreme-java.com/ Sandeep

    Please read the following post on REST interview questions which definitely have questions which test the above mentioned points.

    http://extreme-java.com/rest-web-service-interview-questions/

  • http://www.markbaker.ca Mark Baker

    Nice write-up. So nice in fact, that it reads eerily similarly to my own article on the topic 8-)

    http://www.infoq.com/articles/mark-baker-hypermedia

  • http://linkednotbound.net Andrew Wahbe

    Good summary. One thing though — the self-descriptive messaging constraint isn’t just about headers but also the media type. Converting the data into a standard media type (vs. puking internal data structures onto the wire as JSON) is how RESTful systems get data encapsulation and take advantage of the network effects of existing clients and infrastructure (another benefit of Atom). I’ve written about this here: http://linkednotbound.net/2010/07/19/self-descriptive-hypermedia/

    APIs that use a non-standard format are really “Option 3″ and not REST.

  • Pingback: Distributed Weekly 119 — Scott Banwart's Blog

  • Priya

    Great article, and thank you for the “reality check”! I really hate it when people claim to have written a “REST” based service, where you need to do an http “Get” to update an entity!!

    cheers,
    Priya

  • Pingback: Revue de Presse Xebia | Blog Xebia France

  • http://profile.earth2marsh.com Marsh Gardiner

    In the Twitter example, wouldn’t it be better to refer to the user resource with a relative URL, so href=”/1/users/show.xml?screen_name=TwitterAPI”? It seems that an advantage of HATEOAS is that it appreciates the relationships of resources. Those resources are relative to each other, so why not use a relative resource? A proxy use-case might be a good example of why you’d want relative instead of absolute.

  • Pingback: Clarifying REST : AppleGrew's Mind

  • Pingback: REST: Links, News and resources (2) « Angel “Java” Lopez on Blog

  • Pingback: Clarifying REST | Venturin.net

  • http://camelcase.blogspot.com Maurizio

    Nice post indeed. I think many API designers still don’t get how fundamental it is to strictly adhere to pure REST principles. It’s not about being dogmatic, it’s all about honest communication.
    REST is for me a “design protocol” which communicates intentions to others: if you break the protocol because of performance needs, convenience or whatever, you are violating a contract with your clients. If you stick the word “REST” over an API because it is fashionable but you break the protocol, then you are probably opening the Pandora’s box to any kind of weird compromise: how can I keep trusting your API then?

  • Martin Payne

    I am in the middle of writing a RESTful API, at least its as RESTful as I am prepared to make it. The only point I find that I can’t follow the ‘standard’ on is the ‘stateless’ constraint. I’m not sure that I’m violating that constraint, but my API requires authentication and it seems the lesser of two evils to violate the stateless constraint, and create a minimal session, rather than have the each request authenticate.

    What’s your take on this?

    Thanks
    Martin

  • http://TypeCastException.com xivSolutions

    Very well-written post, and most informative for thos eof us trying to learn this stuff. Thank you for your effort to provide such well-written commentary.

  • Pingback: Openly Connect » Overdue Ideas

  • Pingback: Learn Simple.Data & Nancy at SkillsMatter – codeface

  • http://joeyguerra.com Joey Guerra

    Great! You just described a web page where a resource is represented in HTML. I love it! Just wish people would start realizing that a web site IS an API already, just with resources represented in HTML. I can write an API all in HTML and not have to build 2 separate apps. What’s really cool is I can make my 1 web site represent resources in different formats like JSON, XML, ATOM all by just creating “views” for the same resource so that when a client makes a request to /users.xml they get the users resource represented as XML, or /users.json and get a JSON representation or /users.atom and get an ATOM and when going to /users or /users.html that just get HTML. Why does this seem so novel?

  • http://christopherbrookes.fr Klaitos

    Such an useful article. If only i could have it when i needed to dev a RESTful API.. U just clarified to me how REST work in 3 paragraph, nice work !

  • http://dinogambone.com Dino Gambone

    Good read and I’m just as guilty of using ReST and ReSTful when I should really be saying Resource Based Web Services (or ReBased). I definitely will need to brush up more on ReST and decided if what I’m really after is ReST or ReBased web services.

    Martin: Check out this URL regarding authentication using just HTTP – http://bit.ly/HfDFpF

  • Pingback: ASP.Net Web API links « Fast .NET

  • Pingback: REST e APIs Hipermídia « Blog de André Bires

  • Pingback: 2012 Archive of Hanselman's Newsletter of Wonderful Things - Scott Hanselman

  • Pingback: Building a Simple REST Controller with Microsoft Visual Studio 2012 and WebAPI - PeterKellner.net

  • Pingback: Difference between WCF and Web API and WCF REST and Web Service | Sriramjithendra Nidumolu

  • Pingback: Difference between WCF and Web API and WCF REST and Web Service | Dot Net Blogs Discussion

  • Pingback: A best explanation of REST | Road to Data Professional

  • Pingback: [Dev tip] Difference between WCF and Web API and WCF REST and Web Service | sudo man