Working with .env files in docker and docker-compose

One thing that annoyed me in the past week was dealing with environment variables in docker-compose. I digged the web and found out that some good fellas had already discussed and solved my problems over here –env-file option #6170, and here Support for –env-file option for docker-compose #6535. I decided to make that a little bit more obvious for anyone lost enough to end up here.

The first thing I should clarify is that both docker and docker-compose have a --env-file option. Although similar, they are meant to achieve different purposes. Let’s try to understand how they work by example.

--env-file in docker

Creating this file in my current directory.

# .env.container
TEST1=Remember this: "Be it a rock or a grain of sand, in water they sink as the same." - Woo-jin Lee

And then running docker run -it --env-file .env.container ubuntu:latest bash -c "echo \$TEST1" we find out in the output that the contents of .env.container are loaded inside the ubuntu container. The echo command is written with a trailing slash to prevent my current shell to expand that variable. Conclusion? Running docker run --env-file gets your .env variable’s exported inside the container.

--env-file in docker-compose

Now, what if we want to replicate this exact behaviour but using docker-compose?

# docker-compose.yml
version: '3.7'

services:
  ubuntu:
    image: ubuntu:latest
    entrypoint: bash -c
    tty: true
    command: 
    - echo $$TEST1
    env_file: 
    - .env.container

After defining our ubuntu service, we can run it with docker-compose run ubuntu, and see that it prints out exactly the same as our docker command.

Where am I trying to get here? Well, let’s try to feed our compose with some variables read from .env.compose and understand how the --env-file behaves in docker-compose.

# .env.compose
TEST2=I thought I'd lived a simple life. But I've sinned too much - Dae-su Oh 
TAG_UBUNTU=18.04

Let’s update our compose to look like this.

version: '3.7'

services:
  ubuntu:
    image: ubuntu:${TAG_UBUNTU:-latest}
    entrypoint: bash -c
    tty: true
    command:
    - echo $$TEST1; echo $TEST2; echo $$TEST2
    environment:
    - TEST2=$TEST2
    env_file: 
    - .env.container

Check this out. docker-compose --env-file .env.compose config outputs:

services:
  ubuntu:
    command:
    - echo $$TEST1; echo I thought I'd lived a simple life. But I've sinned too much
      - Dae-su Oh; echo $$TEST2
    entrypoint: bash -c
    environment:
      TEST1: 'Remember this: "Be it a rock or a grain of sand, in water they sink
        as the same." - Woo-jin Lee'
      TEST2: I thought I'd lived a simple life. But I've sinned too much - Dae-su
        Oh
    image: ubuntu:18.04
    tty: true
version: '3.7'

Interesting… let’s run this and try to take conclusions after.

Running docker-compose --env-file .env.compose run ubuntu outputs

Remember this: "Be it a rock or a grain of sand, in water they sink as the same." - Woo-jin Lee
I thought Id lived a simple life. But Ive sinned too much - Dae-su Oh
I thought I'd lived a simple life. But I've sinned too much - Dae-su Oh

I added some echos of variables for us to better understand what’s going on.

How is compose dealing with the variables? We are clearly importing TAG_UBUNTU from .env.compose. And compose seems to understand really well what to pass to our container using our environment and env_file tags. Looking at the echo command in docker-compose config. $$TEST1 is not replaced by any value. That seems obvious since env_file is feeding that variable directly to the container (it’s not available for compose to use). What about TEST2? Well, we can see that $TEST2 is being replaced, that’s because compose has direct access to it (.env.compose). $$TEST2 is not replaced when we look at docker config, but we see that the variable it’s outputted when we run the container. $$ is basically telling to compose to not print your local TEST2 variable, but to print the container’s one. The thing is, when you run docker-compose config you are not actually running containers and executing commands. The config command is simply showing you what compose is interpreting from the yaml and variables, may they be placed in a .env file or exported directly in your shell.

I hope that this helped someone in need.

Takeaways

Versions:

Docker version 19.03.5, build 633a0ea838
docker-compose version 1.25.4, build unknown
· docker, docker-compose, .env