Creating local Go dev environment with Docker and live code reloading

This post is a quick how-to for starting a new project in Go. It features:

• Hot code reloading
• Running multiple Docker containers with Docker Compose
• Using Go Modules for managing dependencies

It’s best to show the above working together with an example project. We’re going to set up two separate services communicating with messages over NATS. The first one will receive messages on an HTTP endpoint and then publish them to a NATS topic. The other will subscribe to this topic and print incoming messages on the standard output.

It’s pretty simple architecture, right? To keep the examples short and make handling messages easier, we’re going to use the Watermill library.

I won’t go through all Go code here, but you can clone the example repository.

Requirements

You’re going to need Go 1.11+, Docker and Docker Compose installed on your local machine.

Go Modules

Go Modules were introduced in Go 1.11 and are (hopefully) the final solution for managing dependencies.

I recommend keeping the repository outside of GOPATH if you’re going to use them. If you’d prefer to use GOPATH anyway, make sure you’ve set GO111MODULE=on variable.

Let’s initialize modules for each of our packages (inside subscriber and publisher directories).

go mod init github.com/ThreeDotsLabs/nats-example/subscriber
go mod init github.com/ThreeDotsLabs/nats-example/publisher

Download dependencies:

go get github.com/ThreeDotsLabs/watermill
go get github.com/nats-io/go-nats-streaming  

Add an extra ULID library only for the publisher:

go get github.com/oklog/ulid

Dockerfile

The next step is creating a Docker image for containers that will run both applications. The Dockerfile is fairly simple:

FROM golang:1.11.2-stretch
RUN go get github.com/cespare/reflex
COPY reflex.conf /
ENTRYPOINT ["reflex", "-c", "/reflex.conf"]

The image is based on the official Go image. Usually, you’d want to stick with the smaller alpine version, but since it is missing some dependencies for compiling reflex, I’m using the stretch version here.

It installs and runs reflex which will be used for hot recompiling the code. It can be very useful for quickly testing your changes.

Reflex is configured by reflex.conf, which in our case is just one line:

-r '(\.go$|go\.mod)' -s go run .

What the command means is: “watch for changes to go.mod and all files ending in .go and execute go run . when it happens. The -s flag stands for service and will make reflex kill previously run command before starting it again, which is exactly what we want.

Docker Compose

We’ll use Docker Compose for running multiple services within a shared network. It allows you to spin up containers with services or databases without cluttering your local machine. Also, other developers can set up the project by cloning the repository and running one command, instead of manually installing dependencies and compiling the application.

Put the docker-compose.yml file in the repository:

version: '3'
services:
  publisher:
    build: .
    volumes:
      - ./publisher:/app
      - $GOPATH/pkg/mod/cache:/go/pkg/mod/cache
    working_dir: /app
    env_file:
      - .env
    ports:
      - 5000:5000

  subscriber:
    build: .
    volumes:
      - ./subscriber:/app
      - $GOPATH/pkg/mod/cache:/go/pkg/mod/cache
    working_dir: /app
    env_file:
      - .env

  nats:
    image: nats-streaming:0.11.2
    restart: on-failure

It defines three services. Both publisher and subscriber are based on the Dockerfile we’ve created before (this is done by the build: . option). Each application’s code is mounted in the working directory, so go run . will run the proper package. The ports part in the publisher service will bind port 5000 on your local system and map it to the port inside container.

I’ve also mounted $GOPATH/pkg/mod/cache directory, to use existing cache for dependencies, speeding up downloads.

The services are configured by environment variables, kept in a separate file:

PORT=5000
NATS_TOPIC=example
NATS_CLUSTER_ID=test-cluster
NATS_URL=nats://nats:4222

Notice the nats hostname in the NATS_URL variable - this is the name of the service defined in the docker-compose.yml file.

Running

To start the environment, run:

docker-compose up

You’ll see the output of all three services. In another terminal, try sending an example request to the publisher service with curl:

$ curl -X POST http://localhost:5000 --data "this is my message"
Sent message: this is my message with ID 01D09P02SBW5D0QPWP14QQZJWH

You should see the message delivered in the subscriber log output:

subscriber_1  | [00] 2019/01/03 11:01:27 received message: 01D09P02SBW5D0QPWP14QQZJWH, payload: this is my message

Now, try editing either of the main.go files in your editor. After you save the file, appropriate service should be recompiled and restarted.

When finished, you can kill the environment with Ctrl-C in the terminal it is running. You can also delete all containers by running docker-compose down.

What’s next?

Now that you’ve built Docker images of the services it’s the first step towards deploying it. We’ll look into it in the next post.

If you’d like to explore more docker-compose definitions, check Watermill Getting started for Kafka and Google Cloud Pub/Sub examples.