Develop F# live in a Docker container
I think F# and Docker are both pretty cool technologies. Here is a way to create an awesome development environment with them.
Here are the different versions of the software that I'm using:
- OSX : 10.11.3 (15D21)
- Docker : 1.11.1-beta13.1 (build: 8193)
- Mono : (Stable 18.104.22.168/832de4b Wed Mar 30 13:57:48 PDT 2016)
- Forge via brew : samritchie/forge/forge: stable 0.7.0
To begin let's create a new directory, initialize our git repository, and create the new forge project.
Although there is a suave template, I chose to use the console library so that I could understand the basic pieces of F# and Suave before moving on to a pre-built template.
Before you commit the generated project scaffolding, you can choose whether you want to add the packages folder to your gitignore file or not. Packaging compiled dependencies versus having declarative dependency files is a debate that is out of the scope of this post. I am going to choose to add it to the gitignore file so that the program will download the needed dependencies when the container is being built.
If we want to run this new project we can simply build and then run the compiled executable located in the `build/` folder.
This is going to be a Suave app, so lets add in the dependency using forge (forge will automatically add the reference to the .fsproj file so we don't have to). We will also add in the FSharp.Compiler.Service dependency so that we can send updated files to be compiled and reload the application.
Now that we have our dependencies installed, we can move on to the core files that will enable us to do interactive development: app.fsx, build.fsx, the Dockerfile, and the docker-compose.yml file.
The app.fsx file will live in the ssdweb/ folder and will be the entrypoint to our Suave application.
The build.fsx will be our FAKE build script responsible for watching the project directory for changes and restarting the application (app.fsx) when a file is changed.
The Dockerfile represents the steps required to build our application in a single docker container.
The docker-compose.yml file represents the file we will use to start up our application in docker-compose. Running our app via docker-compose will allow us to easily add containers for other useful services which our application can utilize to provide more functionality (think databases, cache servers, etc).
app.fsx lives in the fsproj directory (`suavedockerdev/sddweb/app.fsx`) is just the hello world example from their website for now.
build.fsx lives in the top level directory (`suavedockerdev/build.fsx`) and is the core of how the hot-reloading of our application works.
While other languages have their own programs for hot-reloading changed files (nodemon for node, dotnet-watch for .net core, etc), this is something that is written by hand in our build script.
This file is mostly taken from Tomas Petricek's found here: https://github.com/tpetricek/suave-xplat-gettingstarted/blob/master/build.fsx Not only did he create the base for this file, but also helped me figure out how to get it working in docker-compose!
This file works by doing a few things:
First it sets up a F# compiler service session named `fsiSession`. This allows us to send changed code to be compiled on a different thread and returned to us in it's final form.
It then creates the `reportFsiError` helper function for writing errors to output should there be an issue.
Next it defines the `reloadScript` function. This function defines `appFsx` to represent the app.fsx file. If your file is in a different location, make sure this is pointing to the correct spot! It uses the `EvalInteraction` and `EvalExpression` to re-evaluate the app.fsx file. It's basically a hand made repl for your app.fsx file.
`currentApp` is defined as a mutable (ref) placeholder for your app so that once the compiler service evaluates your new interpereted app.fsx file, that it can re-assign the value of `currentApp` to the result of the evaluation.
`serverConfig` is then defined with a few differentiations from the Suave default config. One special thing to note here is that I used the ip `0.0.0.0`. Docker by default binds all available interfaces (INADDR_ANY) at the address 0.0.0.0 unless configured to do otherwise. I wanted to leave that alone, so I just made the change in the build file.
The `reloadAppServer` function comes next. It takes the sequence of changed files, logs a message, calls the `reloadScript` function, then swaps in the new app value.
The last piece of this script is the actual build target. This is what is called when FAKE is executed by the `build.sh` script. First it calls an initial `reloadAppServer` with the app.fsx file. It then starts the server asynchronously with the initial value returned. Next it sets the source directories variable to include .fs, .fsx, and .fsproj files to be watched. Once the directories are set, it creates a `System.IO.FileSystemWatcher` and calls the `WatchChanges` method to watch those files for changes, composing the `reloadAppServer` function on to the end to be called whenever a file changes. It then calls `Thread.Sleep(-1)` to hold the thread open so that when the application is run in a detatched container (docker-compose) that it continues to keep the process open.
Local Live Editing
With these two files in place, you should now be able to live edit your web server code.
To do this open up two terminals. In the first terminal run `sh build.sh` from the projects root directory. The output should look something like this:
Now in your second terminal, you should be able to run curl to see the output.
Now in whatever editor you're using, go and change the string in the `app.fsx` file to "Hello Suave Interactive!"
If you go back to the first terminal, you should see output indicating that the watcher process noticed the file change and reloaded the application.
And if we run curl again we should see the new output!
Setting up Docker for in-container development
Live-reloading of an app is cool, but that could break if I tried it on a friend's machine who has an old copy of xamarin, unity3d, brew, and 3 other versions of mono linked in weird ways which they forgot about and have never gone back to check on since they never use it themselves and all the apps that need it are working at the moment. It would be pretty sweet if I could tell them that all they needed to do to start playing around with the project was `git clone` and a `docker run` command. I would also sleep easy knowing that any other places I would run it (CI server, Azure, AWS, DigitalOcean, etc) would have the same output. So let's get this working in docker.
All that we need to do is create a Dockerfile and pass some parameters to the `docker run` command to get this to work. To be fair, we did do a little bit of setup earlier with the specific IP address and Thread.Sleep call that were in the `build.fsx` file.
The Dockerfile is pretty simple.
It uses the fsharp image as it's base (which takes care of making sure mono and other fsharp dependencies are present)
Then it copies the application code over to the `/app/` directory of the container (absolute path).
The `WORKDIR` command changes the directory into the copied application directory.
We `EXPOSE` the 8083 port from the container to the container host so that there is a way to communicate with the application.
Finally we run our same build script as the entrypoint into our docker container.
Now that the docker file is set up we can do the same 2 terminal setup from earlier.
First let's build the container into a container image that docker can run. We'll also give it a tag with a specific version so that we can keep track of our image.
Now that we have a container image built we can run the docker image in terminal 1.
This docker command has a few parameters which are important:
- -it gives us an interactive terminal so that we can interact with the running application. This is opposite of a detached container, where the image runs in the background until the process ends.
- `--rm` removes the container in which the image is run after the process ends. I included this flag as more of a convenience so that I don't have to clean up a bunch of unused containers later.
- -v `pwd`:/app is the what makes the live-reloading work even in the docker container. By default, when docker builds an image, it compresses the copied directories and uncompresses them in the container. This makes it so that the files are actually different files than the files on your local directory. The -v command overlays a directory onto a containers directory so that when docker runs, it sees that directory instead. Since the container sees our local directory, it will correctly notice when a file changes and reloads the app. If we did not use this option, the file watcher would be watching the compressed->decompressed files that were created when the image was built instead of the files we edit locally on our computer.
- -p forwards a port from the docker container to the docker host. This is what allows us to communicate with the docker container via `localhost:8083`.
If you go do the curl->edit->curl test, you should see that you can now interactively develop inside of a reusable docker container!
Live Editing in Docker Compose
Docker itself gives us a utility for reliable code execution. Docker compose takes that up a level to wire multiple docker containers together to allow you to bring up a reliable infrastucture. First we'll create the docker-compose.yml file that will allow us to live-edit in docker compose, and then we'll see how docker compose allows us to easily add other utilities to our applications infrastructure.
This file declares a volume `web` which is built from the current directory, and then passes the same parameters that we were passing when running `docker run`.
Run `docker-compose up` to pull up our container.
Once more curl->edit->curl and see that we're good to go.
Reaping the rewards
We could have settled with the local live-editing for developing our Suave application, but we decided to get it set up in docker compose with some extra work.
Let's see how easy it is to add a redis box to our development environment.
That's it, we can just run `docker-compose up` again and now our Suave app has a redis app which it can communicate with.
Now eventually we will want to have another build target so that we have something that can run in production, but this works for fleshing out ideas for now.
Thanks to Tomas Petricek and Reed Copsey, Jr. for the help they provided over on the #fsharp-beginners channel of the functionalprogramming slack!