Elixir: Document your API

In order to have someone who will actually use your API, you need to provide some documentation and usages for it. Developer eXperience is also very important when making an API. Making back-end is difficult but having a great front-end too, that is why using your API should be a breeze.

I really like to have my documentation as near as my code, but I want it to be decoupled too. There’s different libraries that offers such a feature, some might use controller annotations like ExDoc, others generate the documentation from the test cases using Bureaucrat, personally I prefer to add my documentation explicitly manually for a matter of separation of concerns using PhoenixSwagger. This way I can eventually not provide external documentation for some endpoints, and have more test cases. It’s a matter of taste.

For the sake of standardisation I prefer to use the OpenAPI Specification (aka Swagger) in order to document my API, it provides some great tooling and is widely supported. That is why I used the phoenix_swagger library and added the documentation in my controllers. You can have a look at the OpenAPI 2.0 Specs to know more about supported formats, parameters, headers and authentication. In Phoenix we have a DSL allowing us to generate the swagger.json file based on our definitions.

To make it work just follow the phoenix_swagger installation guide, you’ll mainly need to provide your router and endpoint module name in the config.ex and a definition of swagger_info/0 in your router

def swagger_info do
  %{
    info: %{
      version: "1.0",
      title: "MY API",
      consumes: [
        "multipart/form-data",
        "application/json",
        "application/vnd.api+json"
      ]
    },
    securityDefinitions: %{
      bearerAuth: %{
        type: "apiKey",
        name: "Authorization",
        in: "header"
      }
    },
    security: [
      %{bearerAuth: []}
    ]
  }
end

This is mine, you can see that you need to define all of your API supported content-type. I’ve also defined that my API ask for a bearer authorization token.

I’ve also added a route to access the generated API at /api/swagger. It will allow developers to access the last available version of the documentation easily :

scope "/api/swagger" do
  pipe_through([:api_doc_auth])

  forward("/", PhoenixSwagger.Plug.SwaggerUI, otp_app: :MY_API, swagger_file: "swagger.json")
end

You can see that I have a pipeline applied to this route, this is because we want to add a basic authentication to the route.

pipeline :api_doc_auth do
  plug(BasicAuth, use_config: {:gi_api, :api_doc_auth})
end

We used the basic_auth Phoenix plug to make this possible and easy, that’s how you can configure it to use environment variables as login / password for your API documentation route.

config :gi_api,
  api_doc_auth: [
    username: System.get_env("BASIC_AUTH_API_USERNAME"),
    password: System.get_env("BASIC_AUTH_API_PASSWORD"),
    realm: "API Doc Area"
  ]

Now that you’ve configured how you documentation will be available, you just need to declare it :) This is done in your controller, you’ll need to call swagger_path/2 in order to define each endpoint, they can then refer to more complexes schemas that needs to be loaded calling swagger_definitions/0.

swagger_path :index do
  PhoenixSwagger.Path.get("/api/datas")
  consumes("application/vnd.api+json")
  produces("application/vnd.api+json")

  operation_id("index")

  tag("Data")

  paging(size: "page[page_size]", number: "page[page]")

  description("List data")

  response(200, "Success", Schema.ref(:Data))
  response(401, "Not Authenticated")
end

This is my definition for the paginated data schema as a JSON-API resource.

%{
  DataResource:
    JsonApi.resource do
      description("A data.")

      relationship(:user)

      relationship(:media, type: :has_many)

      attributes do
        some_attribute(:string, "Data attribute")
      end
    end,
  Data: JsonApi.single(:DataResource),
  Datas: JsonApi.page(:DataResource)
}

On our project we use JsonApi through JaSerializer and pagination thanks to Scrivener. As it is a common stack PhoenixSwagger helpers around JSON-API resources. Very useful when you’re using it as your Data Transfer Protocol. You can define a single resource with JsonApi.single/1 or a paginated resource (based on the paging parameter) with JsonApi.page/1.

Another example with a POST request getting a file from a multipart/form-data form could be.

swagger_path :create do
  PhoenixSwagger.Path.post("/api/datas/{data_id}/medias")
  consumes("multipart/form-data")
  produces("application/json")

  operation_id("create")

  tag("Medias")

  description("Create a media")

  parameters do
    data_id(:path, :string, "Data UUID", required: true)
    kind(:formData, :string, "Should be [image|document]", required: true)
    file(:formData, :file, "Attached media", required: true)
  end

  response(200, "Success", Schema.ref(:Media))
  response(404, "Not Found")
  response(422, "Unprocessable Entity")
  response(401, "Not Authenticated")
end

The :formData specify to Swagger that the API accepts a form formatted field and the type :file means that we will have a binary field and expect a multipart/form-data content type.

With all this you’ll have a smooth API documentation available at /api/swagger, protected by a login / password which supports JWT token authorization header.

You can go further by looking at the PhoenixSwagger documentation. The documentation is not as up to date as I thought but you can access the @doc annotations directly in the source code to have some more examples. I had to find in passed issues too for some of my needs. Besides it also offer easier controller testing through schema validation, but this is another story :)

I hope that this will help you provide a great developer experience to your front end developers and that they will let you waste more time on 9gag now :)

If you like this Elixir / Phoenix blog post serie please share it or drop a comment.

Using CDN caching with GraphQL

Introduction

Hi, today we'll see how-to do a CDN GraphQL caching system in some quick, easy and dirty steps. The idea came to me to make this article because the other day I was reading a Blog post about some reasons you should not use GraphQL in production. I've been using it for a bit on production websites and I disagree with some of the arguments presented in the paper.

One of them was that you cannot do CDN caching with GraphQL because almost no CDN manage POST body cache management. That's why I'll show you in this article how you could (but shouldn't) implement GraphQL GET request caching to do (almost) the same as a JSON REST API.

Creating a simple GraphQL server

So let's begin by creating a GraphQL server with a really simple schema offering a simple hello world field. I used Koa (because it is better than express :troll:) and expose a /graphql endpoint.

const Koa = require('koa');
const Router = require('koa-router');
const graphqlHTTP = require('koa-graphql');

const {buildSchema} = require('graphql');

const app = new Koa();
const router = new Router();

const schema = buildSchema(`
  type Query {
    hello: String
  }
`);

const root = {
  hello: () => {
    return 'Hello world!';
  },
};

router.all('/graphql', graphqlHTTP({
 schema,
 rootValue: root,
 graphiql: true
}))

app.use(router.routes()).use(router.allowedMethods())

app.listen(4000)

Then we do a graphql query to see if it works. Annnnnd Yes.

curl -X GET "http://0.0.0.0:4000/graphql?query=%7Bhello%7D"
{"data":{"hello":"Hello world!"}}

As you can see I'm passing a query parameter to define which GraphQL action I want to do on server.

How-to

Actually our caching could already work as is, as GraphQL query is passed on GET method and not on POST. But, because GraphQL queries can be very big, the default option is to use POST. The solution to still use GET request is to compress the query using lz-string in order to not have any problem while passing our queries.

So to do that, we'll create a fake client that will add query string compression and force GET request.

const rp = require('request-promise');
const LZString = require('lz-string');

function prepareQuery(query) {
 return LZString.compressToBase64(query);
}

async function sendQuery() {
 return rp({
  method: 'GET',
  uri: 'http://0.0.0.0:4000/graphql',
  qs: {
   query: prepareQuery('{hello}')
  }
 })
}

sendQuery()
 .then(res => console.log(res))

So the query will now look like this :

REQUEST emitting complete http://0.0.0.0:4000/graphql?query=N4CwpgNhD2C%2BQ%3D%3D%3D
{"data":{"hello":"Hello world!"}}

Now to handle this type of query we need to add some reaaaally simple logic to our GraphQL middleware that will decode the query before sending it to our schema treatment routine. This is done by using the function handler that is executed at the very beginning of GraphQL koa middleware.

router.all('/graphql', graphqlHTTP((request, response, ctx) => {
 const decodedQuery = LZString.decompressFromBase64(request.query.query);
 ctx.query = {query: decodedQuery};
 return {
  schema,
  rootValue: root,
  graphiql: true
 }
}))

As you can see it very simple and straightforward to make it happen. So why isn't it the default GraphQL implementation ?

Conclusion

Actually GraphQL implementations, I'm mainly talking about Apollo Server, as it is the one I know most, implement caching a very different way. By caching received data on client side, they optimized the requesting process, because clients manage locally their own caches you can avoid a lot of useless requests to the server.

The other drawback of this CDN server caching is that GraphQL queries may slightly change and therefore create lot of cache entries for each clients.

To conclude, I'd say this implementation could be useful on some specific cases where clients always do the same requests and you cannot add client caching system (on some legacy system) but it should be avoided as much as possible as it is not the most efficient way to do GraphQL query caching.

Go see my website for some more data about who am I : https://www.flyersweb.org