During the redesign of the diretto APIs, we focussed on a style that follows more the principles of REST and paid attention to a consistent use of patterns and guidelines for all of our APIs. While some of them are well established, others are rather controversial to some extent in the REST community. This article gives a short overview hereof.

Extensive documentation

Although in theory a single starting URI should be enough to describe a REST API, we wanted to provide extensive documentation for all of our APIs. This not just helps us to keep track of our API features, it also helps developers to getting started with it.

The documentation lists all of the resources, possible representations and operations on these resources. For operations, sample entities as well as a list of possible response codes are listed. The documentation also provides URI template information and authentication requirements.

We developed our own template for creating REST documentations that transforms a XML-based descriptions of an API into a HTML file (or other formats such as plaintext or PDF). All example entities (in our case only JSON) have been validated and formatted, to assure correct and readable snippets.

URI as only ID

Another change was the introduction of URIs as the only way of identifying platform resources. While this is an obvious characteristic of REST, we were still referring to our internal resource IDs in some of our API calls in the previous version. Now we use URIs for everything.

Hyperlinks between resources

A side benefit of the URI-only approach is the increased linking between different resources. When a user creates a resource, it’s representation will not list his name or internal ID as the creator, but it will contain a hyperlink to the user’s URI. By providing useful links to the most important resources on the entry resource, a client can now navigate through the entire platform without having to know any URI pattern or template in advance.

Collection Resources

This is a bit controversial when it comes to URI design, but we decided to differentiate between a resource path prefix and a resource collection path prefix. So when the URI of a resource is /resource/{internal-id}, the corresponding collection resource is /resources. The separation prevents collisions when the collection resource has sub-resources itself.

Pagination using cursors

Our APIs contain a lot of resources that provide lists to other resources. With unbound length, it is not reasonable to return an entire list in a single request. We therefore use pagination to split up these lists. Pagination is done using cursors allowing stateless pages. Each page contains a list of links pointing to the previous and next page as well as the first page of the list. A page is identified by the first hit on the page.

The diretto platform is not a single application. It is rather a set of modular components both in terms of usage and of general project architecture. This blog entry describes the different layers and the underlying ideas of the system.

The common ground is an elaborated conception, what distributed reporting should be for the diretto platform. The key entity is a document, that is a testimony of an event. A document possess a geo-spatial and temporal source of origin. While documents are abstract entities, there are always at least one attachment appended to it. Further attachments allow enhanced or processed variants of the original piece. Users can apply tags and comments to a document, propose updated time/place origins and vote on all of these items. Key value pairs provide additional tags beneficial for machines and links allow to connect causally related documents.
While this was a rather generic description, here is an example: Alice is eye-witness of an event an takes a photo (document) with her smartphone. She uploads the raw image (attachment), and appends the time and position. Unfortunately, Alice does not have GPS enabled, so the position is inaccurate. Bob is not on-site but accesses the platform via browser. He sees the images and recognizes the exact position. He then proposes an improved position and appends some useful tags. He downloads the image and improves the contrast. He then uploads the outcame as an additional attachment. Other examples for documents are videos, audio recordings, text messages, sensor reports etc.

To make this conception computationally comprehensible, it is described in form of a language-agnostic API. In detail, we have chosen a RESTful approach, based on HTTP. Thus our API is a set of (connected) resources and valid operations on these resources. Different concerns are separated by different APIs. So there is for instance an API that handles the upload, download and storage of raw documents that have been submitted. But handling comments, votes etc. is handled by another API, and so on. This not just modularizes the API, it also allows you to use only subsets of the API, depending on your use case.

Basically, you could start to implement components for the diretto platform by implementing the two sides of the API – the service providers (servers), and the service consumers (clients). However, we offer a middleware that already provides the platform internals, based on the APIs. More precisely, we offer a backend service implementation that represents a service provider. On client side, we offer ready-to-use client libraries so you don’t have to deal with low-level communications.

So relying on our middleware allows you to build your specific reporting application on top of the platform. Whether you are bird-watching or you are going to provide live-coverge of a sports event – you don’t have to start from scratch. Instead, you can select from the diretto platform the parts of the toolkit that you actually need and directly start to develope it.

The Task API is an additional service API that provides support for advanced task management between different platform users while reporting, but also thereafter. It represents an optional service API that does not belong to the core platform, although deployments will be beneficial in most scenarios.

Task API: http://diretto.github.com/diretto-api-doc/v2/diretto-ext/task.html
Repository:  https://github.com/diretto/diretto-api-doc

In short, users can create tasks, and other users can participate in these tasks. In terms of the diretto platform, a task is a request of specific footage. Reporters may take in part in tasks and contribute required documents as submissions. A task is always bound to a location and a time frame. A title and description define the task and optional tags allow to categorize it. Votes help to prioritize different tasks within the community. Tasks can be commented for exchange of ideas. Finally, every user can submit an existing document as footage for the task. The community can then rate the suitability of the submission regarding the task request. Besides the of directed collection of live footage, the Task API can also be used for specialized search queries. In this case, a task represents a search query and all potential results are submissions. This user-driven search is helpful for queries when only users can reason about appropriate hits by analyzing the documents.

Example 1 – Directed reporting
Scenario: The diretto platform has been deployed for a major sports event, let’s say a marathon.
Spectators are encouraged to upload live photos from their wayside position, thus creating a large collection of different photos from this event. Due to special interest in photos of the current title holder a contender, a task has been created for this. When taking pictures of these runners, a reporter may add the document as a submission to this task.

Example 2 – User-driven search
Scenario: A tsunami has devastated a larger coastal area. In the aftermath, a diretto platform has been deployed in order to collect corresponding media footage. Although there was a major infrastructure breakdown, eyewitnesses had been able to make photos and videos using their cameras or smartphones. A few days later, many of these documents have been uploaded and are now available on the internet.
The diretto platform is used to aggregate these documents combined with temporal and spatial meta data. A task is then created querying for documents that capture the exact moment of the flooding for further analysis. Platform users may now survey all existing document and suggest appropriate footage to the task as submission.

The diretto project will be featured at Streiflicht 2011. This event presents results of various elective courses and student works from the Institute of Media Informatics at Ulm University.

The upcoming diretto v2 release will restructure the current API and introduce several new functions. We decided to split up the API into several modules.

Image by nataliejohnson; cc-by-nc/2.0

Core API
The Core API represents the only mandatory API that is necessary for a diretto platform (although in most cases at least the Storage API will be required as well). The Core API provides access to the common operations like adding user-generated documents and contents as well as query and retrieve methods.

Storage API
The Storage API handles the upload, storage and access of multimedia files on platform. This API is independent of the actual backend storage used.

Feed API
The Feed API provides different news feed formats, including ATOM and KML. A PubSubHubbub publisher node enables push-style notifications of changes.

Streaming API
The Streaming API exposes an event-based notification service. As an alternative to the PubSubHubbub-enabled feeds, the notification streams can help to create real-time applications and services for the diretto platform.

Registry API
The Registry API allows external services to register. This facilitates lookup and discovery of additional services for the clients.