Packaging Elixir Applications With Docker

February 15, 2023 ยท 27 min

Working with elixir can feel euphoric when you’re in the zone. The BEAM VM being built around fault tolerance and inter-process communication that’s exposed to you via abstractions such as GenServer and Task, which are pleasent to work with, means that errors are not more uncommon (such as in Rust) but they shoot you in the foot just as infrequently because a Supervisor will just restart your application instead of throwing itself off a cliff.

With all that said, most existing documentation aimed at beginners is for starting a project in Elixir while very few are for packing it for deployment (unless you’re using phoenix). Digging into the documentation for mix release can feel daunting, so here’s my attempt at explaining how to package an Elixir application with Docker.

The code for this project can be found on Github.

Drinking the FP Koolaid

We’ll be building a simple web server, configuring it as an application, and then going through the process of mix release before delving into the land of docker. Skip to whichever section you’d like, but there’s no harm in reading top-to-bottom.

Building a Simple Web Server

Let’s start a new project:

 1โžœ  mix new sydney
 2* creating README.md
 3* creating .formatter.exs
 4* creating .gitignore
 5* creating mix.exs
 6* creating lib
 7* creating lib/sydney.ex
 8* creating test
 9* creating test/test_helper.exs
10* creating test/sydney_test.exs
11
12Your Mix project was created successfully.
13You can use "mix" to compile it, test it, and more:
14
15    cd sydney
16    mix test
17
18Run "mix help" for more commands

To make it an actual application, we’ll define a webserver using plug. We’ll need to add it to our mix.exs file as a dependancy first.

 1# mix.exs
 2
 3defmodule Sydney.MixProject do
 4  use Mix.Project
 5
 6  def project do
 7    [
 8      app: :sydney,
 9      version: "0.1.0",
10      elixir: "~> 1.14",
11      start_permanent: Mix.env() == :prod,
12      deps: deps()
13    ]
14  end
15
16  # Run "mix help compile.app" to learn about applications.
17  def application do
18    [
19      extra_applications: [:logger]
20    ]
21  end
22
23  # Run "mix help deps" to learn about dependencies.
24  defp deps do
25    [
26      {:plug_cowboy, "~> 2.0"}
27    ]
28  end
29end

Now we’re able to actually write a simple web server.

 1# lib/sydney/server.ex
 2
 3defmodule Sydney.Server do
 4  use Plug.Router
 5
 6  plug(:match)
 7  plug(:dispatch)
 8
 9  get "/" do
10    conn
11    |> send_resp(200, "Ok")
12  end
13
14  match _ do
15    conn
16    |> send_resp(404, "Page not found")
17  end
18
19  def start_link(_) do
20    Plug.Adapters.Cowboy.http(Server, [])
21  end
22end

It’s not much, sure, but it’ll do for our purposes. Fancier things could require non-Elixir dependencies such as Discord bots using ffmpeg for audio transcoding, but at that point you know what you’re getting yourself into. Thankfully, the Elixir/Erlang community seems to shy away from cross-language contamination, but not nerely as bad as golang is with cgo.

application.ex

Let’s make an simple application.ex file to define Sydney. This can be considered the entrypoint of our program, but in reality it’s more of a top-level overview of how our application if structured. Things such as an Ecto.Repo process for managing the database connection or a Phoenix.PubSub process would be defined here.

We’ll use a Supervisor watch over Sydney’s server and restart it on a crash.

 1# lib/sydney/application.ex
 2
 3defmodule Sydney.Application do
 4  @moduledoc false
 5  use Application
 6
 7  @impl true
 8  def start(_type, _args) do
 9    children = [
10      {Plug.Cowboy, scheme: :http, plug: Sydney.Server, options: [port: 8080]}
11    ]
12
13    opts = [
14      strategy: :one_for_one,
15      name: Sydney.Supervisor
16    ]
17
18    Supervisor.start_link(children, opts)
19  end
20end

Now we can add Sydney.Application to the function application/0 in mix.exs under the mod key. This specifies which module will be invoked when the application starts up.

1  # mix.exs:14-20
2
3  # Run "mix help compile.app" to learn about applications.
4  def application do
5    [
6      mod: {Sydney.Application, []},
7      extra_applications: [:logger]
8    ]
9  end

To sanity check ourselves, let’s check to see if Sydney works.

 1โžœ  mix deps.get
 2Resolving Hex dependencies...
 3Dependency resolution completed:
 4New:
 5  cowboy 2.9.0
 6  cowboy_telemetry 0.4.0
 7  cowlib 2.11.0
 8  mime 2.0.3
 9  plug 1.14.0
10  plug_cowboy 2.6.0
11  plug_crypto 1.2.3
12  ranch 1.8.0
13  telemetry 1.2.1
14* Getting plug_cowboy (Hex package)
15* Getting cowboy (Hex package)
16* Getting cowboy_telemetry (Hex package)
17* Getting plug (Hex package)
18* Getting mime (Hex package)
19* Getting plug_crypto (Hex package)
20* Getting telemetry (Hex package)
21* Getting cowlib (Hex package)
22* Getting ranch (Hex package)
23You have added/upgraded packages you could sponsor, run `mix hex.sponsor` to learn more
24โžœ  mix run --no-halt
25==> mime
26Compiling 1 file (.ex)
27Generated mime app
28===> Analyzing applications...
29===> Compiling telemetry
30==> plug_crypto
31Compiling 5 files (.ex)
32Generated plug_crypto app
33===> Analyzing applications...
34===> Compiling ranch
35==> plug
36Compiling 1 file (.erl)
37Compiling 41 files (.ex)
38Generated plug app
39===> Analyzing applications...
40===> Compiling cowlib
41===> Analyzing applications...
42===> Compiling cowboy
43===> Analyzing applications...
44===> Compiling cowboy_telemetry
45==> plug_cowboy
46Compiling 5 files (.ex)
47Generated plug_cowboy app
48==> sydney
49Compiling 3 files (.ex)
50Generated sydney app
51

There’s no output saying that our application is running, but we can test it pretty easily.

1โžœ  curl http://localhost:8080/
2Ok

Looks good to me!

mix release

mix release is the command used to package and release an Elixir application. The docs are here, but it can be a long read. I’ll summarize the absolute necessities below.

We’ll start by defining the release options in the project/0 function within our mix.exs file.

 1# mix.exs:
 2  def project do
 3    [
 4      app: :sydney,
 5      version: "0.1.0",
 6      elixir: "~> 1.14",
 7      start_permanent: Mix.env() == :prod,
 8      deps: deps(),
 9      
10      # --- This is new ---
11      releases: [
12        sydney: [
13          include_executables_for: [:unix]
14        ]
15      ]
16    ]
17  end

This is actually the only required option. There are a ton of other settings, which you should definitely look into, but none of them are required. Actually, include_executables_for is also not required, but most elixir applications run on unix-based operating systems, so there’s no reason to build a windows version.

Here are the most useful settings, in my opinion: - :steps to build a .tar.gz artifact. - :strip_beams for a smaller release by stripping debug info. - :path to set where the build output path. - :overlays more on them here.

Once we have everything set, building the application is as simple as running mix release sydney.

 1โžœ  mix release sydney
 2Generated sydney app
 3* assembling sydney-0.1.0 on MIX_ENV=dev
 4* skipping runtime configuration (config/runtime.exs not found)
 5
 6Release created at _build/dev/rel/sydney
 7
 8    # To start your system
 9    _build/dev/rel/sydney/bin/sydney start
10
11Once the release is running:
12
13    # To connect to it remotely
14    _build/dev/rel/sydney/bin/sydney remote
15
16    # To stop it gracefully (you may also send SIGINT/SIGTERM)
17    _build/dev/rel/sydney/bin/sydney stop
18
19To list all commands:
20
21    _build/dev/rel/sydney/bin/sydney
22

As you can see, the built application is located at _build/dev/rel/sydney/bin/sydney. If you build with MIX_ENV=prod then the base path will be _build/prod instead of _build/dev.

Let’s follow the directions and list all commands.

 1โžœ  _build/dev/rel/sydney/bin/sydney
 2Usage: sydney COMMAND [ARGS]
 3
 4The known commands are:
 5
 6    start          Starts the system
 7    start_iex      Starts the system with IEx attached
 8    daemon         Starts the system as a daemon
 9    daemon_iex     Starts the system as a daemon with IEx attached
10    eval "EXPR"    Executes the given expression on a new, non-booted system
11    rpc "EXPR"     Executes the given expression remotely on the running system
12    remote         Connects to the running system via a remote shell
13    restart        Restarts the running system via a remote command
14    stop           Stops the running system via a remote command
15    pid            Prints the operating system PID of the running system via a remote command
16    version        Prints the release name and version to be booted

It works! Let’s take a look at bin/sydney and see how big the binary is.

1โžœ  ls -lh _build/dev/rel/sydney/bin/sydney
2-rwxr-xr-x 1 digyx digyx 5.2K Feb 15 10:00 _build/dev/rel/sydney/bin/sydney*

That’s…tiny. That can’t be everything, right? Let’s take a look at the file.

  1#!/bin/sh
  2set -e
  3
  4SELF=$(readlink "$0" || true)
  5if [ -z "$SELF" ]; then SELF="$0"; fi
  6RELEASE_ROOT="$(CDPATH='' cd "$(dirname "$SELF")/.." && pwd -P)"
  7export RELEASE_ROOT
  8RELEASE_NAME="${RELEASE_NAME:-"sydney"}"
  9export RELEASE_NAME
 10RELEASE_VSN="${RELEASE_VSN:-"$(cut -d' ' -f2 "$RELEASE_ROOT/releases/start_erl.data")"}"
 11export RELEASE_VSN
 12RELEASE_COMMAND="$1"
 13export RELEASE_COMMAND
 14RELEASE_PROG="${RELEASE_PROG:-"$(echo "$0" | sed 's/.*\///')"}"
 15export RELEASE_PROG
 16
 17REL_VSN_DIR="$RELEASE_ROOT/releases/$RELEASE_VSN"
 18. "$REL_VSN_DIR/env.sh"
 19
 20RELEASE_COOKIE="${RELEASE_COOKIE:-"$(cat "$RELEASE_ROOT/releases/COOKIE")"}"
 21export RELEASE_COOKIE
 22RELEASE_MODE="${RELEASE_MODE:-"embedded"}"
 23export RELEASE_MODE
 24RELEASE_NODE="${RELEASE_NODE:-"$RELEASE_NAME"}"
 25export RELEASE_NODE
 26RELEASE_TMP="${RELEASE_TMP:-"$RELEASE_ROOT/tmp"}"
 27export RELEASE_TMP
 28RELEASE_VM_ARGS="${RELEASE_VM_ARGS:-"$REL_VSN_DIR/vm.args"}"
 29export RELEASE_VM_ARGS
 30RELEASE_REMOTE_VM_ARGS="${RELEASE_REMOTE_VM_ARGS:-"$REL_VSN_DIR/remote.vm.args"}"
 31export RELEASE_REMOTE_VM_ARGS
 32RELEASE_DISTRIBUTION="${RELEASE_DISTRIBUTION:-"sname"}"
 33export RELEASE_DISTRIBUTION
 34RELEASE_BOOT_SCRIPT="${RELEASE_BOOT_SCRIPT:-"start"}"
 35export RELEASE_BOOT_SCRIPT
 36RELEASE_BOOT_SCRIPT_CLEAN="${RELEASE_BOOT_SCRIPT_CLEAN:-"start_clean"}"
 37export RELEASE_BOOT_SCRIPT_CLEAN
 38
 39rand () {
 40  dd count=1 bs=2 if=/dev/urandom 2> /dev/null | od -x | awk 'NR==1{print $2}'
 41}
 42
 43release_distribution () {
 44  case $RELEASE_DISTRIBUTION in
 45    none)
 46      ;;
 47
 48    name | sname)
 49      echo "--$RELEASE_DISTRIBUTION $1"
 50      ;;
 51
 52    *)
 53      echo "ERROR: Expected sname, name, or none in RELEASE_DISTRIBUTION, got: $RELEASE_DISTRIBUTION" >&2
 54      exit 1
 55      ;;
 56  esac
 57}
 58
 59rpc () {
 60  exec "$REL_VSN_DIR/elixir" \
 61       --hidden --cookie "$RELEASE_COOKIE" \
 62       $(release_distribution "rpc-$(rand)-$RELEASE_NODE") \
 63       --boot "$REL_VSN_DIR/$RELEASE_BOOT_SCRIPT_CLEAN" \
 64       --boot-var RELEASE_LIB "$RELEASE_ROOT/lib" \
 65       --vm-args "$RELEASE_REMOTE_VM_ARGS" \
 66       --rpc-eval "$RELEASE_NODE" "$1"
 67}
 68
 69start () {
 70  export_release_sys_config
 71  REL_EXEC="$1"
 72  shift
 73  exec "$REL_VSN_DIR/$REL_EXEC" \
 74       --cookie "$RELEASE_COOKIE" \
 75       $(release_distribution "$RELEASE_NODE") \
 76       --erl "-mode $RELEASE_MODE" \
 77       --erl-config "$RELEASE_SYS_CONFIG" \
 78       --boot "$REL_VSN_DIR/$RELEASE_BOOT_SCRIPT" \
 79       --boot-var RELEASE_LIB "$RELEASE_ROOT/lib" \
 80       --vm-args "$RELEASE_VM_ARGS" "$@"
 81}
 82
 83export_release_sys_config () {
 84  DEFAULT_SYS_CONFIG="${RELEASE_SYS_CONFIG:-"$REL_VSN_DIR/sys"}"
 85
 86  if grep -q "RUNTIME_CONFIG=true" "$DEFAULT_SYS_CONFIG.config"; then
 87    RELEASE_SYS_CONFIG="$RELEASE_TMP/$RELEASE_NAME-$RELEASE_VSN-$(date +%Y%m%d%H%M%S)-$(rand).runtime"
 88
 89    (mkdir -p "$RELEASE_TMP" && cat "$DEFAULT_SYS_CONFIG.config" >"$RELEASE_SYS_CONFIG.config") || (
 90      echo "ERROR: Cannot start release because it could not write $RELEASE_SYS_CONFIG.config" >&2
 91      exit 1
 92    )
 93  else
 94    RELEASE_SYS_CONFIG="$DEFAULT_SYS_CONFIG"
 95  fi
 96
 97  export RELEASE_SYS_CONFIG
 98}
 99
100case $1 in
101  start)
102    start "elixir" --no-halt
103    ;;
104
105  start_iex)
106    start "iex" --werl
107    ;;
108
109  daemon)
110    start "elixir" --no-halt --pipe-to "${RELEASE_TMP}/pipe" "${RELEASE_TMP}/log"
111    ;;
112
113  daemon_iex)
114    start "iex" --pipe-to "${RELEASE_TMP}/pipe" "${RELEASE_TMP}/log"
115    ;;
116
117  eval)
118    if [ -z "$2" ]; then
119      echo "ERROR: EVAL expects an expression as argument" >&2
120      exit 1
121    fi
122
123    export_release_sys_config
124    exec "$REL_VSN_DIR/elixir" \
125       --cookie "$RELEASE_COOKIE" \
126       --erl-config "$RELEASE_SYS_CONFIG" \
127       --boot "$REL_VSN_DIR/$RELEASE_BOOT_SCRIPT_CLEAN" \
128       --boot-var RELEASE_LIB "$RELEASE_ROOT/lib" \
129       --vm-args "$RELEASE_VM_ARGS" --eval "$2"
130    ;;
131
132  remote)
133    exec "$REL_VSN_DIR/iex" \
134         --werl --hidden --cookie "$RELEASE_COOKIE" \
135         $(release_distribution "rem-$(rand)-$RELEASE_NODE") \
136         --boot "$REL_VSN_DIR/$RELEASE_BOOT_SCRIPT_CLEAN" \
137         --boot-var RELEASE_LIB "$RELEASE_ROOT/lib" \
138         --vm-args "$RELEASE_REMOTE_VM_ARGS" \
139         --remsh "$RELEASE_NODE"
140    ;;
141
142  rpc)
143    if [ -z "$2" ]; then
144      echo "ERROR: RPC expects an expression as argument" >&2
145      exit 1
146    fi
147    rpc "$2"
148    ;;
149
150  restart|stop)
151    rpc "System.$1()"
152    ;;
153
154  pid)
155    rpc "IO.puts System.pid()"
156    ;;
157
158  version)
159    echo "$RELEASE_NAME $RELEASE_VSN"
160    ;;
161
162  *)
163    echo "Usage: $(basename "$0") COMMAND [ARGS]
164
165The known commands are:
166
167    start          Starts the system
168    start_iex      Starts the system with IEx attached
169    daemon         Starts the system as a daemon
170    daemon_iex     Starts the system as a daemon with IEx attached
171    eval \"EXPR\"    Executes the given expression on a new, non-booted system
172    rpc \"EXPR\"     Executes the given expression remotely on the running system
173    remote         Connects to the running system via a remote shell
174    restart        Restarts the running system via a remote command
175    stop           Stops the running system via a remote command
176    pid            Prints the operating system PID of the running system via a remote command
177    version        Prints the release name and version to be booted
178" >&2
179
180    if [ -n "$1" ]; then
181      echo "ERROR: Unknown command $1" >&2
182      exit 1
183    fi
184    ;;
185esac

Ohhhhh so it’s just a launch script. Makes sense. Actually how many other files are there? How big are they in total?

1โžœ  du -sh _build/dev/rel/sydney/
216M     _build/dev/rel/sydney/
3โžœ  find _build/dev/rel/sydney -type f | wc -l
4874

WOW. That’s a lot of files. Total size isn’t bad, though.

Unlike other languages such as Go and Rust, Elixir compiles for the BEAM virtual machine, which is to Erlang what the JVM is to Java. Taking a look at the file tree of _build/dev/rel/sydney, we can see a bunch of individual files. Unlike the JVM, Elixir actually bundles the runtime with the application. This means the machine we’re running the application on doesn’t need Elixir to be installed.

1โžœ  docker run -it \
2      --volume "$PWD/_build/dev/rel/sydney:/opt/sydney" \
3      -p 8080:8080 \
4      ubuntu /bin/bash
5root@29f14932efe3:/# /opt/sydney/bin/sydney start
6warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell)

Ubuntu doesn’t run utf8 by default? Huh, alright. Welp, that doesn’t matter to us right now, so we’ll ignore it.

1โžœ  curl http://localhost:8080/
2Ok

Fantastic!

Caveaught

Now, this doesn’t mean Sydney is infinitely portable. She still has some dependencies, just ones that are more “reasonable”. For example, the following doesn’t work.

1โžœ  docker run -it \
2      --volume "$PWD/_build/dev/rel/sydney:/opt/sydney" \
3      -p 8080:8080 \
4      debian:latest /bin/bash
5root@e138865fb957:/# /opt/sydney/bin/sydney start
6/opt/sydney/erts-13.1.3/bin/erlexec: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /opt/sydney/erts-13.1.3/bin/erlexec)

And we can see why by checking our glibc versions.

 1โžœ  ldd --version
 2ldd (GNU libc) 2.37
 3Copyright (C) 2023 Free Software Foundation, Inc.
 4This is free software; see the source for copying conditions.  There is NO
 5warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 6Written by Roland McGrath and Ulrich Drepper.
 7โžœ  docker run -it debian:latest /bin/bash
 8root@031ee395c38d:/# ldd --version
 9ldd (Debian GLIBC 2.31-13+deb11u5) 2.31
10Copyright (C) 2020 Free Software Foundation, Inc.
11This is free software; see the source for copying conditions.  There is NO
12warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
13Written by Roland McGrath and Ulrich Drepper.

My Arch Linux installation has glibc 2.37 while the debian images only have glibc 2.31. We can solve this issue by simply building the application in a docker container with a lower glibc version and then moving the artifacts to another container so…

Docker Time

Let’s start with a simple, impossible to mess up configuration.

 1FROM elixir:latest
 2
 3# Config
 4ENV MIX_ENV prod
 5WORKDIR /opt/build
 6
 7# Dependendies
 8COPY mix.* ./
 9
10RUN mix local.hex --force && \
11  mix local.rebar --force && \
12  mix deps.get --only prod && \
13  mix deps.compile
14
15# Build project
16COPY lib ./lib
17RUN mix release sydney
18
19ENTRYPOINT ["/opt/sydney/bin/sydney"]
20CMD ["start"]

As a note, you’ll want to copy over your config/ file and anything else necessary for the project to build. We don’t want to do a COPY . . for two reason:

  1. We could accidentally copy our own _build folder over, which is totally not something that gave me multiple days worth of headaches and;

  2. This will slow down our build by reducing the amount we can cache since any change to any file would require Docker to fully rebuild or dependencies instead of just our application.

Anyway, let’s build the thing.

 1โžœ  docker build -t sydney .
 2[+] Building 37.6s (15/15) FINISHED
 3 => [internal] load build definition from Dockerfile                                           0.0s
 4 => => transferring dockerfile: 480B                                                           0.0s
 5 => [internal] load .dockerignore                                                              0.0s
 6 => => transferring context: 2B                                                                0.0s
 7 => [internal] load metadata for docker.io/library/elixir:latest                               0.1s
 8 => [internal] load metadata for docker.io/library/elixir:slim                                 0.2s
 9 => [build_stage 1/6] FROM docker.io/library/elixir:latest@sha256:6b43caee72c7d30e338cfd1419  19.4s
10 => => resolve docker.io/library/elixir:latest@sha256:6b43caee72c7d30e338cfd14193162f981bf9c4  0.0s
11 => => sha256:6b43caee72c7d30e338cfd14193162f981bf9c47aa6ea6226424ab7e8ab67bb 1.42kB / 1.42kB  0.0s
12 => => sha256:c3aa11fbc85a2a9660c98cfb4d0a2db8cde983ce3c87565c356cfdf1ddf26 10.88MB / 10.88MB  0.5s
13 => => sha256:2c9304afafd20dc07418015142cd316b6a09b09a4e1d7d9e41af25ebaa331ec 2.22kB / 2.22kB  0.0s
14 => => sha256:7770942456a46c17a6d4d4744c2830c8e4e5c628e6b620c46388759b81e91de 8.81kB / 8.81kB  0.0s
15 => => sha256:6c1024729feeb2893dad43684fe7679c4d866c3640dfc3912bbd93c5a51f32d 5.17MB / 5.17MB  0.3s
16 => => sha256:aa54add66b3a47555c8b761f60b15f818236cc928109a30032111efc98c6f 54.59MB / 54.59MB  5.7s
17 => => extracting sha256:6c1024729feeb2893dad43684fe7679c4d866c3640dfc3912bbd93c5a51f32d2      0.1s
18 => => sha256:9e3a60c2bce7eed21ed40f067f9b3491ae3e0b7a6edbc8ed5d9dc7dd9e4 196.90MB / 196.90MB  5.9s
19 => => sha256:e3b7d942a0e149a7bea9423d528d6eb3feab43c98aa1dfe76411f722e12 243.12MB / 243.12MB  8.2s
20 => => extracting sha256:c3aa11fbc85a2a9660c98cfb4d0a2db8cde983ce3c87565c356cfdf1ddf2654c      0.2s
21 => => extracting sha256:aa54add66b3a47555c8b761f60b15f818236cc928109a30032111efc98c6fcd4      1.6s
22 => => sha256:a575a6735350a64d448af63013b1e6a8d2a8ee41c2718c0e22501828961 198.62kB / 198.62kB  5.9s
23 => => sha256:59db57e4c8acbbdbf667b31b8fd4713e2a4fac33afd9593c843bae0641c 988.28kB / 988.28kB  6.0s
24 => => sha256:6a6fa9498b870a0b0a4f2b9d9f5ff584bbba599c281c0172a9b2fdfc7ee8cf0 6.20MB / 6.20MB  6.5s
25 => => extracting sha256:9e3a60c2bce7eed21ed40f067f9b3491ae3e0b7a6edbc8ed5d9dc7dd9e4a0f92      5.0s
26 => => extracting sha256:e3b7d942a0e149a7bea9423d528d6eb3feab43c98aa1dfe76411f722e122f5bb      6.0s
27 => => extracting sha256:a575a6735350a64d448af63013b1e6a8d2a8ee41c2718c0e2250182896136621      0.0s
28 => => extracting sha256:59db57e4c8acbbdbf667b31b8fd4713e2a4fac33afd9593c843bae0641cb8d33      0.0s
29 => => extracting sha256:6a6fa9498b870a0b0a4f2b9d9f5ff584bbba599c281c0172a9b2fdfc7ee8cf06      0.1s
30 => [stage-1 1/3] FROM docker.io/library/elixir:slim@sha256:c9db6016833b2ffa57a5a37276215c8d  10.4s
31 => => resolve docker.io/library/elixir:slim@sha256:c9db6016833b2ffa57a5a37276215c8dcbf13f996  0.0s
32 => => sha256:c9db6016833b2ffa57a5a37276215c8dcbf13f9967679490d65f4e2c8131d4b 1.41kB / 1.41kB  0.0s
33 => => sha256:7bb623d2806f43d77c1cbe68489a1737714a6682d13d54f7e41be37c8972fdf6 952B / 952B     0.0s
34 => => sha256:96a79f0cd42af47aa8ec2495afbad867d02c0edddc3b7521a4fd55cb6028e2d 5.96kB / 5.96kB  0.0s
35 => => sha256:121475a6527dd0c5a079a61159de0ec80f0befe4d5e2e733eb06b6ba826ea 65.93MB / 65.93MB  8.4s
36 => => sha256:7fa7197cb3fda651dcd6412d3c61de5ba2cff3d71969bf11a76d0508c966455 6.70MB / 6.70MB  7.3s
37 => => extracting sha256:121475a6527dd0c5a079a61159de0ec80f0befe4d5e2e733eb06b6ba826ea8f8      1.6s
38 => => extracting sha256:7fa7197cb3fda651dcd6412d3c61de5ba2cff3d71969bf11a76d0508c966455a      0.2s
39 => [internal] load build context                                                              0.0s
40 => => transferring context: 228B                                                              0.0s
41 => [stage-1 2/3] WORKDIR /opt/sydney                                                          0.0s
42 => [build_stage 2/6] WORKDIR /opt/build                                                       0.0s
43 => [build_stage 3/6] COPY mix.* ./                                                            0.0s
44 => [build_stage 4/6] RUN mix local.hex --force &&   mix local.rebar --force &&   mix deps.g  16.0s
45 => [build_stage 5/6] COPY lib ./lib                                                           0.0s
46 => [build_stage 6/6] RUN mix release sydney                                                   1.4s
47 => [stage-1 3/3] COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney        0.1s
48 => exporting to image                                                                         0.4s
49 => => exporting layers                                                                        0.4s
50 => => writing image sha256:3fd48d8431e482766324ebdd8d0271ab2da79339e704d37ea725a6136e5414bc   0.0s
51 => => naming to docker.io/library/sydney                                                      0.0s

Sheesh, I forgot how verbose docker can be. From now on, we’ll be passing the --quiet flag to make this more readable.

Anyway, let’s try running the thing.

1โžœ  docker run -p 8080:8080 sydney
2

Okay, no output, but we expect that. Let’s test it using curl.

1โžœ  curl http://localhost:8080/
2Ok
3โžœ  curl http://localhost:8080/404
4Page not found

Whoot!

Out of curiosity, how big is the container?

1โžœ  docker images sydney
2REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
3sydney       latest    92b52ae16b4c   15 seconds ago   1.6GB

… Oh no…

Miniturization

Let’s start with the low hanging fruit: slim images.

1FROM elixir:slim
2
3# The rest is the same
1โžœ  docker build --quiet --tag sydney .
2sha256:fdb4ea030f3a2984bdc5260d910eae2bd6a8e01a7359e930173f47ff5128e2cc
3โžœ  docker images sydney
4REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
5sydney       latest    fdb4ea030f3a   18 seconds ago   430MB

Better, but still unacceptable. What if we use build stages?

For those unfamiliar, build stages allow us to build our artifacts in one container before copying them to another container. This means our final image will only have runtime dependencies instead of build time and runtime dependencies.

 1FROM elixir:latest AS build_stage
 2
 3# Config
 4ENV MIX_ENV prod
 5WORKDIR /opt/build
 6
 7# Dependendies
 8COPY mix.* ./
 9
10RUN mix local.hex --force && \
11  mix local.rebar --force && \
12  mix deps.get --only prod && \
13  mix deps.compile
14
15# Build project
16COPY lib ./lib
17RUN mix release sydney
18
19FROM debian:bullseye-slim
20
21WORKDIR /opt/sydney
22COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
23
24ENTRYPOINT ["/opt/sydney/bin/sydney"]
25CMD ["start"]
1โžœ  docker build --quiet --tag sydney .
2sha256:9b2438ea418100d02d83a97fc9b74026c8adff001e1af0d1d089ae4f1cd29413
3โžœ  docker images sydney
4REPOSITORY   TAG       IMAGE ID       CREATED              SIZE
5sydney       latest    1e6e55f851f0   About a minute ago   145MB

Another 3x improvement is great, but those will stop happening eventually. We also get out good ol’ friend latin1 by using debian:bullseye-slim instead of elixir:slim.

1โžœ  docker run -p 8080:8080 sydney
2warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell)
3

Our next, and most drastic, step is switching to alpine images. Those never cause issues.

 1FROM elixir:alpine AS build_stage
 2
 3# Config
 4ENV MIX_ENV prod
 5WORKDIR /opt/build
 6
 7# Dependendies
 8COPY mix.* ./
 9
10RUN mix local.hex --force && \
11  mix local.rebar --force && \
12  mix deps.get --only prod && \
13  mix deps.compile
14
15# Build project
16COPY lib ./lib
17RUN mix release sydney
18
19FROM alpine:latest
20
21WORKDIR /opt/sydney
22COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
23
24ENTRYPOINT ["/opt/sydney/bin/sydney"]
25CMD ["start"]

Okay, does it work?

 1โžœ  docker run -p 8080:8080 sydney
 2Error loading shared library libncursesw.so.6: No such file or directory (needed by /opt/sydney/erts-13.1.4/bin/beam.smp)
 3Error loading shared library libstdc++.so.6: No such file or directory (needed by /opt/sydney/erts-13.1.4/bin/beam.smp)
 4Error loading shared library libgcc_s.so.1: No such file or directory (needed by /opt/sydney/erts-13.1.4/bin/beam.smp)
 5Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEmmPKcm: symbol not found
 6Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __cxa_begin_catch: symbol not found
 7Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: tgetflag: symbol not found
 8Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _Znwm: symbol not found
 9Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt20__throw_length_errorPKc: symbol not found
10Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __cxa_guard_release: symbol not found
11Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNKSt8__detail20_Prime_rehash_policy11_M_next_bktEm: symbol not found
12Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __popcountdi2: symbol not found
13Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: tgetent: symbol not found
14Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt20__throw_out_of_rangePKc: symbol not found
15Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0_RS_: symbol not found
16Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt17__throw_bad_allocv: symbol not found
17Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm: symbol not found
18Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm: symbol not found
19Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: tputs: symbol not found
20Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt18_Rb_tree_incrementPKSt18_Rb_tree_node_base: symbol not found
21Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructEmc: symbol not found
22Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __cxa_end_catch: symbol not found
23Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __cxa_guard_acquire: symbol not found
24Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNKSt8__detail20_Prime_rehash_policy14_M_need_rehashEmmm: symbol not found
25Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt19__throw_logic_errorPKc: symbol not found
26Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: tgetnum: symbol not found
27Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt28__throw_bad_array_new_lengthv: symbol not found
28Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZSt18_Rb_tree_decrementPSt18_Rb_tree_node_base: symbol not found
29Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: pthread_getname_np: symbol not found
30Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEm: symbol not found
31Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: tgetstr: symbol not found
32Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __cxa_rethrow: symbol not found
33Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _Unwind_Resume: symbol not found
34Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZdlPvm: symbol not found
35Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
36Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
37Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
38Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
39Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
40Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
41Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
42Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
43Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
44Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
45Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
46Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
47Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
48Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
49Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
50Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: _ZTVN10__cxxabiv121__vmi_class_type_infoE: symbol not found
51Error relocating /opt/sydney/erts-13.1.4/bin/beam.smp: __gxx_personality_v0: symbol not found

Shit.

What about just using elixir:alpine as our final stage? Yeah, sure, we won’t save nerely as much, but it should be something…right?

 1...
 2
 3COPY lib ./lib
 4RUN mix release sydney
 5
 6# --- New image! ---
 7FROM elixir:alpine
 8
 9WORKDIR /opt/sydney
10COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
11
12...
1โžœ  docker build --quiet --tag sydney .
2sha256:a6feb6fd7139c71a71a57c47127ba33a07abcd2d551e6df8af86930c66b4a50e
3โžœ  docker run -p 8080:8080 sydney
4
1โžœ  curl http://localhost:8080/
2Ok

Okay, good, it works. How big is it?

1โžœ  docker images sydney
2REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
3sydney       latest    a6feb6fd7139   14 seconds ago   96.1MB

Under 100mb, that’s good! We can still do better. If we look through the error logs when using the alpine:latest image, we can see that we’re missing a few libraries. Let’s add the following:

 1...
 2
 3FROM alpine:3.16
 4
 5WORKDIR /opt/sydney
 6# --- Let's install some runtime deps ---
 7RUN apk add \
 8    --update \
 9    --no-cache \
10    openssl ncurses libstdc++
11COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
12
13...
1โžœ  docker build --quiet --tag sydney .
2sha256:67e05f2fcbad23bb00eabe90a520555beea423353031be687f4b87c3b930546b
3โžœ  docker images sydney
4REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
5sydney       latest    67e05f2fcbad   20 seconds ago   23.5MB

And there we have it! Going from 1.6GB to 23.5MB feels pretty good.

Note: we must use alpine:3.16 because that’s what erlang:alpine uses which is what elixir:alpine is based on (click the alpine tag to see the image’s dockerfile). If we don’t, we run into issues linking libcrypto.so.1.1, but this will probably be resolved eventually. Either way, always check what erlang:alpine is based on first. That will guarantee compatibility…probably.

Technically…

We can go further. I mentioned earlier that adding :tar to the :steps section of your releases section in mix.exs will also produce a .tar.gz archive. Well, that archive is about half the size.

 1  def project do
 2    [
 3      app: :sydney,
 4      version: "0.1.0",
 5      elixir: "~> 1.14",
 6      start_permanent: Mix.env() == :prod,
 7      deps: deps(),
 8      releases: [
 9        sydney: [
10          include_executables_for: [:unix],
11          # --- This is new! ---
12          steps: [:assemble, :tar]
13        ]
14      ]
15    ]
16  end
1โžœ  mix release sydney --overwrite --quiet
2* skipping runtime configuration (config/runtime.exs not found)
3โžœ  ll _build/dev/sydney-0.1.0.tar.gz 
4-rw-r--r-- 1 digyx digyx 6.9M Feb 15 16:42 _build/dev/sydney-0.1.0.tar.gz

This means we can instead ship a .tar.gz in the container and then decompress it before running. As you can imagine, this is excessive, so…

 1# tar.Dockerfile
 2FROM elixir:alpine AS build_stage
 3
 4# Config
 5ENV MIX_ENV prod
 6WORKDIR /opt/build
 7
 8# Dependendies
 9COPY mix.* ./
10
11RUN mix local.hex --force && \
12  mix local.rebar --force && \
13  mix deps.get --only prod && \
14  mix deps.compile
15
16# Build project
17COPY lib ./lib
18RUN mix release sydney
19
20FROM alpine:3.16
21
22RUN apk add \
23    --update \
24    --no-cache \
25    openssl ncurses libstdc++
26
27WORKDIR /opt/sydney
28COPY entrypoint.sh entrypoint.sh
29RUN chmod +x entrypoint.sh
30COPY --from=build_stage /opt/build/_build/prod/sydney-0.1.0.tar.gz sydney-0.1.0.tar.gz
31
32CMD ["/opt/sydney/entrypoint.sh"]
1#!/usr/bin/env sh
2# entrypoint.sh
3
4tar -xzf /opt/sydney/sydney-0.1.0.tar.gz
5/opt/sydney/bin/sydney start
1โžœ  docker build --quiet --tag sydney --file Dockerfile_tar .
2sha256:858056ca2956d73e9ab8d477df8f309c681cd17e37288ebe156efe73225a7a41
3โžœ  docker images sydney
4REPOSITORY   TAG       IMAGE ID       CREATED              SIZE
5sydney       latest    858056ca2956   About a minute ago   16.1MB

Wow, that entire docker container is smaller than our uncompressed code. Of course, this is a very niche extreme that you should not try to emulate.

Full Dockerfile

 1# Dockerflile
 2FROM elixir:alpine AS build_stage
 3
 4# Config
 5ENV MIX_ENV prod
 6WORKDIR /opt/build
 7
 8# Dependendies
 9COPY mix.* ./
10
11RUN mix local.hex --force && \
12  mix local.rebar --force && \
13  mix deps.get --only prod && \
14  mix deps.compile
15
16# Build project
17COPY lib ./lib
18RUN mix release sydney
19
20FROM alpine:3.16
21
22WORKDIR /opt/sydney
23RUN apk add \
24    --update \
25    --no-cache \
26    openssl ncurses libstdc++
27COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
28
29ENTRYPOINT ["/opt/sydney/bin/sydney"]
30CMD ["start"]

Distroless?

sigh

 1FROM elixir:latest AS build_stage
 2
 3# Config
 4ENV MIX_ENV prod
 5WORKDIR /opt/build
 6
 7# Dependendies
 8COPY mix.* ./
 9
10RUN mix local.hex --force && \
11  mix local.rebar --force && \
12  mix deps.get --only prod && \
13  mix deps.compile
14
15# Build project
16COPY lib ./lib
17RUN mix release sydney
18
19FROM gcr.io/distroless/base
20
21COPY --from=build_stage /opt/build/_build/prod/rel/sydney /opt/sydney
22
23ENTRYPOINT ["/opt/sydney/bin/sydney"]
24CMD ["start"]
1โžœ  docker build --quiet --tag sydney .
2sha256:99ba5748d07947241ad86f35a83800e7bd34ec5c47e38716858a15e1e869711a
3โžœ  docker images sydney
4REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
5sydney       latest    99ba5748d079   33 seconds ago   84.9MB
6โžœ  docker run -p 8080:8080 sydney
7exec /opt/sydney/bin/sydney: no such file or directory

Not only is it larger, but it also just doesn’t run. My guess is the wrong glibc version or another runtime dep not being there, but the image uses debian11 as a base (aka. bullseye), so that doesn’t seem to be it. I’m honestly not sure what’s wrong, and, imo, it’s not worth it to figure it out. Just use alpine.