.png){kind=small}
Hear from our Head of Engineering, Nick Brook give an introduction to Application Programming Interface (APIs)
TL,DR*
Your API is your public face onto the world, so it pays to get it right. But have you actually designed it, or has it just grown organically over time? In this article I’ll give some tips on how to restructure your design workflow and produce beautiful documentation at the same time.
APIs are everywhere
The first program I ever wrote (aged 8 on a Commodore VIC 20: 3.5KB of RAM and built like a tank) was basically this:
It’s about as simple as you can get, and doesn’t need to use any other software.
Unfortunately for everything else that’s even slightly more complex you are going to have to talk to some other software, and for that you’ll need to use their API.
An Application Programming Interface is the public face of software. It defines the kinds of interactions that can be made, the data formats to use and the conventions to follow. APIs can be local (e.g a Python or Javascript library that runs in the same machine as your code), or remote (e.g over HTTP).
In SaaS platforms (Software as a Service) whenever someone talks about an API they usually mean HTTP “REST” APIs.
What is a REST API?
Well, for a start ignore the acronym, as it’s not terribly helpful (it stands for Representational state transfer: I did warn you). A REST API is just an API that adheres to certain architectural constraints. While it can be run over any underlying protocol, in practice most web APIs are run over HTTP and use the JSON data format.
The HTTP protocol is the fundamental protocol of the Web: your browser is making hundreds of HTTP requests to various servers whenever it loads a web page. It’s a very simple protocol, it’s ubiquitous, and it gets through most corporate firewalls (which is certainly one of the reasons for it’s rapid early adoption).
A HTTP request consists of a method (e.g GET), a url (e.g https://app.kisanhub.com/api/farm) and occasionally a payload (e.g a JSON document). The REST architecture basically insists that these HTTP methods are used in a standardised way. There are quite a few methods, but really only the following 5 are in common use:
GET: fetches data
PUT: replaces existing data
POST: creates new data
PATCH: partially updates existing data
DELETE: deletes data
There’s a lot of leeway in what these actually mean in practice, but the main aim of REST is to use a standardised system that doesn’t surprise your API consumer i.e the developer. After all, there’s nothing to stop you using the GET method to delete something or the DELETE method to update something, but it certainly would not be what your users were expecting.
REST APIs should also be stateless. That means the request itself should contain everything it needs to describe the action required. This may seem obvious, but actually a lot of earlier web APIs made heavy use of “session state”, whereby the server keeps a record of data associated with your session. This makes your API much harder to write, document and test, so REST doesn’t allow it.
A side effect of statelessness is that it becomes possible to cache GET requests: for example, suppose you call an API that returns a list of farm details. If the underlying data is not changing frequently, performing the same request again would return you the same list. This lets you add a cache between you and the server to help reduce the load. However it’s important to note that this is not a solution for poorly performant APIs: it’s purely a cost reduction measure as it’s usually cheaper to deploy a cache than to deploy extra API servers (or buy a bigger database).
Design it first
All software will benefit from careful design, and APIs are no exception. At KisanHub we design our API operations first using the OpenAPI 3.0 spec (formerly known as “Swagger”). The spec fully describes a REST API, and is pretty straightforward to learn and edit. As an example, here’s a cut down version of the farm API mentioned earlier:
This illustrates the main features of the OpenAPI spec. It describes a GET operation called “list-farms” that will return a page of details on farms in our database. For example:
(Note: the above won’t work unless you have an account with KisanHub. Sorry!)
Thinking Considered Useful
You could be forgiven for thinking that all software developers carefully design and plan before they write any code. Unfortunately that’s often not the case: it’s surprising how little thought often goes into something as complex and prone to break as a software platform.
There are a number of reasons for this - impatient managers, overbearing investors, angry customers, but the most common one is that sitting down and thinking about what you’re about to do just doesn’t feel like progress. This is particularly true in startups where the team is small and time to market is crucial.
This is of course a blatantly false economy, since it is always more expensive to fix problems down the line when your code is being used by paying customers than it is to just not cause the problem in the first place. That is particularly true when it comes to APIs: once your API is in common use it’s extraordinarily hard to change, so it really pays to get it right the first time.
Anything that forces your developers to sit down and think about what they are about to write is a Good Thing. Having all your APIs in the same spec also makes it much easier to spot commonality and avoid duplication of effort (for example, with the “page parameter” in the spec above).
Keeping it Real
Of course one of the problems with moving your API definition out into a separate file is that it’s no longer closely tied to your code, and unless you are careful the two could diverge. We have found that the best way to protect against this is to ensure that you validate your requests and responses against the spec in your unit tests using one of the many testing libraries: at Kisanhub we use the Python openapi-core library with a few additional tweaks to tie it more closely to the Django test framework (which I will push back to Github as a fork when I get some time).
Privacy is overrated
Generally when people talk about private APIs what they really mean is either “not publicly documented” or it’s evil cousin “just not documented at all”. There’s no such thing as private in a webapp: anyone can view every HTTP call your app makes simply by opening up Chrome debugger and clicking on the “network” tab.
Private APIs are often a Bad Idea as they encourage sloppy design. If you start from the assumption that a paying customer is going to be looking at your APIs then you are far more likely to build a coherent, well organised and sensibly named schema. Also if your API is useful to you it’s quite likely that it will be useful to some of your customers as well, and after all it is their data: they should be allowed to manipulate it.
Documentation
Of course there’s no point writing a spec unless your users can read it, and this is where OpenAPI really shines. The state of a platform’s documentation is usually an excellent litmus test for their quality in general. I will be very reluctant to use a software service or library if their documentation is poor: after all, if they’ve skimped on the public face of their code, what’s it like under the hood?
There are lots of SaaS platforms that will take an OpenAPI spec and automatically generate lovely interactive documentation: ReadMe, StopLight and SwaggerHub are examples that spring to mind. However it turns out that one of the nicest documentation generators is free: ReDoc.
ReDoc is a library that takes a spec and converts it into a Single Page web application that you can just host like any other static content. You can head on over to https://docs.kisanhub.com to see how this works in practice. This site is generated entirely from our spec, and is in fact auto-deployed by our CI/CD system (we use Bitbucket Pipelines to build and deploy to Firebase Hosting after a successful test suite run). I should mention that there is now a SaaS version of ReDoc called Redoc.ly but at the moment it doesn’t really give you more than the free version, although of course I’m sure that will change in time.
About the author
Nick has been writing software for nearly 40 years, and oddly enough is still enjoying it. He's worked on an alarmingly diverse array of languages and platforms, but is currently playing with Python, Angular and DevOps automation with Kubernetes.
*TL;DR stand for 'Too Long; Didn't Read' provides a summary what this article is about. It is commonly used on online articles and blogs.