Elixir: Cryptography

Because we’re managing sensitive data we do not want to have it spread all over, this is why we decided to encrypt with AES our saved logs to be sure that nobody except us can access it, but to allow others to be sure we authored the data we signed it using RSA.

Thus I’ll explain how you can use the ex_crypto library to do such operations. You may refresh your knowledges by going on some AES and RSA documentation to really understand how we’ll use the library.

First of all we want to encrypt our data, for this we’ll need a private key that you should never give to anyone. For this you can use the ExCrypto.generate_aes_key/2 which allows you to generate raw key or base64 encoded. For simplicity we’ll consider that the key is in base64.

Then we encrypt our data using the ExCrypto.encrypt/3 which will generate an initialization vector and expect a tag that we’ll leave blank.

defp encrypt(key, message) do
  with {:ok, {_ad, {init_vec, cipher_text, cipher_tag}}} <- ExCrypto.encrypt(key, "", message) do
    ExCrypto.encode_payload(init_vec, cipher_text, cipher_tag)
  end
end

Then we call the ExCrypto.encode_payload/3 which will serialize the three components (initialization vector, ciphered text and ciphered tag) in a base64 string for transportation.

Now that we have our encoded payload we want to sign it to show that we are the author of this generated data. This is done using asymmetric signature. Same as before we need a private key that can be generated using ExPublicKey.generate_key/2. To make it transportable you can use the ExPublicKey.pem_encode/1 function which will generate a PEM string.

Then we’ll sign our base64 payload with our freshly generated private key. Tips: for those who can’t use the key try to unescape line return special characters from the string as it can be escaped on some systems if used as an environment variable.

defp sign(payload) when is_bitstring(payload) do
  with {:ok, private_key} <- ExPublicKey.loads(key),
       {:ok, signature} <- ExPublicKey.sign(payload, private_key) do
    {:ok, Enum.join([payload, Base.url_encode64(signature)], "|")}
  end
end

Once we have signed the payload we attach the signature (as base64) at the end of the payload separated by a pipe. Note: JWS protocol rather use a dot separator.

Now that we have a signed payload we need to see how to verify it. This could be done by anyone with your public RSA key. But for fun we’ll do it in Elixir, first you’ll need to separate the encoded payload from the signature using

[encoded_payload, signature] = String.split(encoded_payload, "|")

defp verify(encoded_payload, signature) do
  with {:ok, private_key} <- ExPublicKey.loads(key),
       {:ok, public_key} <- ExPublicKey.public_key_from_private_key(private_key) do
    ExPublicKey.verify(encoded_payload, Base.url_decode64!(signature), public_key)
  end
end

Then you can call the verify/2 function defined above which will load the key as PEM string, get the public key from your private key and verify the provided payload. Note: in real usages you'll provide the public key directly.

Now that you are sure the payload is yours you can safely decrypt it using your private AES key as follow

defp decrypt(key, encoded_payload) do
  with {:ok, {d_init_vec, d_cipher_text, d_cipher_tag}} <-
         ExCrypto.decode_payload(encoded_payload) do
    ExCrypto.decrypt(key, "", d_init_vec, d_cipher_text, d_cipher_tag)
  end
end

We see that we decode the payload first to load all necessary decryption information, the initialization vector, the tag and the encrypted text using ExCrypto.decode_payload/1. Then we’re ready to decrypt it using ExCrypto.decrypt/5.

And voila, you’re now able to crypt and sign data transportable on the wire if you need to. Another interesting usage would have been to crypt using RSA public key so someone else could decrypt data at the other end. Signature on the other end is done using your private RSA key.

Note that all AES related features are available in ExCrypto module meanwhile all RSA features are in ExPublicKey for a nice separation of concerns. Note that ExCrypto is mainly a sugar around the :crypto Erlang module.

I hope you’ve learn something thanks to this post and that it will make you want to give a try to Elixir.

Elixir: Log data changes

So far we’ve made our Authentication and Sign up, Sign in, Sign out features based on Guardian JWT Token and Uberauth OAuth 2 standards. We now want to add a feature that will log each modification for a specified model. This part is a little more tricky because there is no standard way to do it. In this post I’ll show you my way of implementing it. First of all let’s understand what we are working on : we needed to store all model changes which could have files attached to it. We want to track every creation, edition, deletion and media management action done by the user.

In order to manage all kinds of log we’ll use a PostgreSQL BSON field allowing us to have no predefined structure for our logs. That's how we defined our log table, attached to a data entry with a comment map field.

defmodule MyApp.Log do
  use Ecto.Schema

  import Ecto.Changeset

  alias MyApp.Log

  @primary_key {:id, Ecto.UUID, autogenerate: true}

  @derive {Phoenix.Param, key: :id}

  schema "logs" do
    field(:comment, {:map, :string})

    belongs_to(
      :data,
      MyApp.Data,
      on_replace: :update,
      foreign_key: :data_id,
      type: Ecto.UUID
    )

    timestamps()
  end

  @doc false

  def changeset(%Log{} = log, attrs) do
    log
    |> cast(attrs, [:comment])
    |> validate_required([:comment])
    |> foreign_key_constraint(:data_id)
  end
end

We have just a single map field in our schema which will contain all the logged data we need and of course the foreign key to the followed object. Keep in mind this simple structure as we’ll be using it afterwards.

As a reminder we’ll see how we insert a model to the datastore and we will see how to get back the changeset.

def create_data(%User{} = user, attrs \\ %{}) do
  # create the data changeset that validate changes

  g =
    %Data{medias: []}
    |> Data.changeset(attrs)
    |> Ecto.Changeset.put_assoc(:user, user)

  # create the log changeset by getting data changes

  bc_attrs =
    Map.put(
      bc_attrs,
      :comment,
      Map.merge(g.changes, %{
        user: Ecto.Changeset.get_field(g, :user, %User{}).id
      })
    )

  bc_log = Log.changeset(%Log{}, bc_attrs)

  # insert data and store log in a transaction to prevent unknown states

  Multi.new()
  |> Multi.insert(:data, g)
  |> Multi.run(:log, &create_log_data_relation(&1.data, bc_log))
  |> Repo.transaction()
end

defp create_log_data_relation(%Data{} = data, %Ecto.Changeset{} = bc_log) do
  Ecto.Changeset.put_assoc(bc_log, :data, data) |> Repo.insert()
end

So to be able to log we first need a changeset for the data we're monitoring, that’s why we create the changeset first. Then we generate a log changeset for the transaction because we do not want the log to be saved separately from original data. We store every changes stored in Ecto.Changeset.changes/0 in the comment field and add the attached user information by using Ecto.Changeset.get_field/3 which try to get the information from changes or in original data if there is no modification, this way we can track associated user too.

Then we use Ecto.Multi for a transactional request. The specific is the Ecto.Multi.run/3 function allowing us (through partial application) to pass the last Ecto.Multi.insert/3 result as first parameter with the current log changeset so we can associate the inserted data before saving it in our logging table.

As you can see we’re performing a customizable change recording machine in just about 15 code lines. It is condensed in term of feature but the code seems pretty readable when knowing the context. As a training you could try to make this code more generic so it can support also editing the data.

For deletion logging, it is more tricky. As Ecto will mark your data as deletable, you’ll not be able to associate it to you logging entry. My solution was simply to change the operation order in my transaction so Ecto will let me store my log first and then delete the entry.

Multi.new()
|> Multi.run(:log, fn _ -> create_log_data_relation(data, bc_log) end)
|> Multi.delete(:data, g)
|> Repo.transaction()

Just be aware that on a hard deletion there is no changes applied as no attributes is modified. You’ll need to manually add your own necessary log information accordingly as of the user association.

Hope you'll better understand what are the changeset and how you can use them to manipulate your application models.

Next we'll see some more Elixir libraries to make some cryptography and documentation.