Creating a generalised Docker image for building ASP.NET Core apps using ONBUILD

This is a follow-up to my recent posts on building ASP.NET Core apps in Docker:

In this post I'll show how to create a generalised Docker image that can be used to build multiple ASP.NET Core apps. If your app conforms to a standard format (e.g. projects in the src directory, test projects in a test directory) then you can use it as the base image of a Dockerfile to create very simple Docker images for building your own apps.

As an example, if you use the Docker image described in this post (andrewlock/aspnetcore-build:2.0.7-2.1.105), you can build your ASP.NET Core application using the following Docker image:

# Build image
FROM andrewlock/aspnetcore-build:2.0.7-2.1.105 as builder

# Publish
RUN dotnet publish "./src/AspNetCoreInDocker.Web/AspNetCoreInDocker.Web.csproj" -c Release -o "../../dist" --no-restore

#App image
FROM microsoft/aspnetcore:2.0.7  
WORKDIR /app  
ENV ASPNETCORE_ENVIRONMENT Local  
ENTRYPOINT ["dotnet", "AspNetCoreInDocker.Web.dll"]
COPY --from=builder /sln/dist .

This multi-stage build image can build a complete app - the builder only has two commands, a FROM statement, and a single RUN statement to publish the app. The runtime image build itself is the same as it would be without the generalised build image. If you wish to use the builder image yourself, you can use the andrewlock/aspnetcore-build repository, available on Docker Hub.

In this post I'll describe the motivation for creating the generalised image, how to use Docker's ONBUILD command, and how the generalised image itself works.

The Docker build image to generalise

When you build an ASP.NET Core application (whether "natively" or in Docker), you typically move through the following steps:

Restore the NuGet packages
Build the libraries, test projects, and app
Test the test projects
Publish the app

In Docker, these steps are codified in a Dockerfile by the layers you add to your image. A basic, non-general, Dockerfile to build your app could look something like the following:

Note, this doesn't include the optimisation described in my earlier post or the follow up:

FROM microsoft/aspnetcore-build:2.0.7-2.1.105 AS builder  
WORKDIR /sln  

# Copy solution folders and NuGet config
COPY ./*.sln ./NuGet.config  ./

# Copy the main source project files
COPY ./src/AspNetCoreInDocker.Lib/AspNetCoreInDocker.Lib.csproj ./src/AspNetCoreInDocker.Lib/AspNetCoreInDocker.Lib.csproj
COPY ./src/AspNetCoreInDocker.Web/AspNetCoreInDocker.Web.csproj ./src/AspNetCoreInDocker.Web/AspNetCoreInDocker.Web.csproj

# Copy the test project files
COPY test/AspNetCoreInDocker.Web.Tests/AspNetCoreInDocker.Web.Tests.csproj test/AspNetCoreInDocker.Web.Tests/AspNetCoreInDocker.Web.Tests.csproj

# Restore to cache the layers
RUN dotnet restore

# Copy all the source code and build
COPY ./test ./test  
COPY ./src ./src  
RUN dotnet build -c Release --no-restore

# Run dotnet test on the solution
RUN dotnet test "./test/AspNetCoreInDocker.Web.Tests/AspNetCoreInDocker.Web.Tests.csproj" -c Release --no-build --no-restore

RUN dotnet publish "./src/AspNetCoreInDocker.Web/AspNetCoreInDocker.Web.csproj" -c Release -o "../../dist" --no-restore

#App image
FROM microsoft/aspnetcore:2.0.7  
WORKDIR /app  
ENV ASPNETCORE_ENVIRONMENT Local  
ENTRYPOINT ["dotnet", "AspNetCoreInDocker.Web.dll"]
COPY --from=builder /sln/dist .

This Dockerfile will build and test a specific ASP.NET Core app, but there are a lot of hard-coded paths in there. When you create a new app, you can copy and paste this Dockerfile, but you'll need to tweak all the commands to use the correct paths.

By the time you get to your third copy-and-paste (and your n-th inevitable typo, you'll be wondering if there's a better, more general, way to achievev the same result. That's where Docker's ONBUILD command comes in. We can use it to create a generalised "builder" image for building our apps, and remove a lot of the repetition in the process.

The `ONBUILD` Docker command

In the Dockerfile shown above, the COPY and RUN commands are all executed in the context of your app. For normal builds, that's fine - the files that you want to copy are in the current directory. You're defining the commands to be run when you call docker build ..

But we're trying to build a generalised "builder" image that we can use as the base for building other ASP.NET Core apps. Instead of defining the commands we want to execute when building our "builder" file, the commands should be run when an image that uses our "builder" as a base is built.

The Docker documentation describes it as a "trigger" - you're defining a command to be triggered when the downstream build runs. I think of ONBUILD as effectively automating copy-and-paste; the ONBUILD command is copy-and-pasted into the downstream build.

For example, consider this simple builder Dockerfile which uses ONBUILD:

FROM microsoft/aspnetcore-build:2.0.7-2.1.105 AS builder
WORKDIR /sln

ONBUILD COPY ./test ./test
ONBUILD COPY ./src ./src

ONBUILD RUN dotnet build -c Release

This simple Dockerfile doesn't have any optimisations, but it uses ONBUILD to register triggers for downstream builds. Imagine you build this image using docker build . -tag andrewlock/testbuild. That creates a builder image called andrewlock/testbuild.

The ONBUILD commands don't actually run when you build the "builder" image, they only run when you build the downstream image.

You can then use this image as a basic "builder" image for your ASP.NET Core apps. For example, you could use the following Dockerfile to build your ASP.NET Core app:

FROM andrewlock/testbuild

ENTRYPOINT ["dotnet", "./src/MyApp/MyApp.dll"]

Note, for simplicity this example doesn't publish the app, or use multi-stage builds to optimise the runtime container size. Be sure to use those optimisations in production.

That's a very small Dockerfile for building and running a whole app! The use of ONBUILD means that our downstream Dockerfile is equivalent to:

FROM microsoft/aspnetcore-build:2.0.7-2.1.105 AS builder
WORKDIR /sln

COPY ./test ./test
COPY ./src ./src

RUN dotnet build -c Release

ENTRYPOINT ["dotnet", "./src/MyApp/MyApp.dll"]

When you build this Dockerfile, the ONBUILD commands will be triggered in the current directory, and the app will be built. You only had to include the "builder" base image, and you got all that for free.

That's the goal I want to achieve with a generalised builder image. You should be able to include the base image, and it'll handle all your app building for you. In the next section, I'll show the solution I came up with, and walk through the layers it contains.

The generalised Docker builder image

The image I've come up with, is very close to the example shown at the start of this post. It uses the dotnet restore optimisation I described in my previous post, along with a workaround to allow running all the test projects in a solution:

# Build image
FROM microsoft/aspnetcore-build:2.0.7-2.1.105 AS builder
WORKDIR /sln

ONBUILD COPY ./*.sln ./NuGet.config  ./

# Copy the main source project files
ONBUILD COPY src/*/*.csproj ./
ONBUILD RUN for file in $(ls *.csproj); do mkdir -p src/${file%.*}/ && mv $file src/${file%.*}/; done

# Copy the test project files
ONBUILD COPY test/*/*.csproj ./
ONBUILD RUN for file in $(ls *.csproj); do mkdir -p test/${file%.*}/ && mv $file test/${file%.*}/; done 

ONBUILD RUN dotnet restore

ONBUILD COPY ./test ./test
ONBUILD COPY ./src ./src
ONBUILD RUN dotnet build -c Release --no-restore

ONBUILD RUN find ./test -name '*.csproj' -print0 | xargs -L1 -0 dotnet test -c Release --no-build --no-restore

If you've read my previous posts, then much of this should look familiar (with extra ONBUILD prefixes), but I'll walk through each layer below.

FROM microsoft/aspnetcore-build:2.0.7-2.1.105 AS builder
WORKDIR /sln

This defines the base image and working directory for our builder, and hence for the downstream apps. I've used the microsoft/aspnetcore-builder image, as we're going to build ASP.NET Core apps.

Note, the microsoft/aspnetcore-builder image is being retired in .NET Core 2.1 - you will need to switch to the microsoft/dotnet image instead.

The next line shows our first use of ONBUILD:

ONBUILD COPY ./*.sln ./NuGet.config ./*.props ./*.targets  ./

This will copy the .sln file, NuGet.config, and any .props or .targets files in the root folder of the downstream build.

# Copy the main source project files
ONBUILD COPY src/*/*.csproj ./
ONBUILD RUN for file in $(ls *.csproj); do mkdir -p src/${file%.*}/ && mv $file src/${file%.*}/; done

# Copy the test project files
ONBUILD COPY test/*/*.csproj ./
ONBUILD RUN for file in $(ls *.csproj); do mkdir -p test/${file%.*}/ && mv $file test/${file%.*}/; done

The Dockerfile uses the optimisation described in my previous post to copy the .csproj files from the src and test directories. As we're creating a generalised builder, we have to use an approach like this in which we don't explicitly specify the filenames.

ONBUILD RUN dotnet restore

ONBUILD COPY ./test ./test
ONBUILD COPY ./src ./src
ONBUILD RUN dotnet build -c Release --no-restore

The next section is the meat of the Dockerfile - we restore the NuGet packages, copy the source code across, and then build the app (using the release configuration).

ONBUILD RUN find ./test -name '*.csproj' -print0 | xargs -L1 -0 dotnet test -c Release --no-build --no-restore

Which brings us to the final statement in the Dockerfile, in which we run all the test projects in the test directory. Unfortunately, due to limitations with dotnet test, this line is a bit of a hack.

Ideally, we'd be able to call dotnet test on the solution file, and it would test all the projects that are test projects. However, this won't give you the result you want - it will try to test non-test projects which will give you errors. There are several different issues looking at this problem, along with some workarounds, but most of them require changes to the app itself, or the addition of extra files. I decided to use a simple scripting approach based on this comment instead.

Using find with xargs is a common approach in Linux to execute a command against a number of different files.

The find command lists all the .csproj files in the test sub-directory, i.e. our test project files. The -print0 argument means that each filename is suffixed with a null character. The

The xargs command takes each filename provided by the file command and executes it with the command dotnet test -c Release --no-build --no-restore. The additional -0 argument indicates that we're using a null character delimiter, and the -L1 argument indicates we should only use a single filename with each dotnet test command.

This approach isn't especially elegant, but it does the job, and it means we can avoid having to explicitly specify the paths to the test project.

That's as much as we can do in the builder image - the publishing step is very specific to each app, so it's not feasible to include that in the builder. Instead, you have to specify that step in your own downstream Dockerfile, as shown in the next section.

Using the generalised build image

You can use the generalised Docker image, to create much simpler Dockerfiles for your downstream apps. You can use andrewlock/aspnetcore-build as your base image, then all you need to do is publish your app, and copy it to the runtime image. The following shows an example of what this might look like, for a simple ASP.NET Core app.

# Build image
FROM andrewlock/aspnetcore-build:2.0.7-2.1.105 as builder

# Publish
RUN dotnet publish "./src/AspNetCoreInDocker.Web/AspNetCoreInDocker.Web.csproj" -c Release -o "../../dist" --no-restore

#App image
FROM microsoft/aspnetcore:2.0.7  
WORKDIR /app  
ENV ASPNETCORE_ENVIRONMENT Local  
ENTRYPOINT ["dotnet", "AspNetCoreInDocker.Web.dll"]
COPY --from=builder /sln/dist .

This obviously only works if you apps use the same conventions as the builder app assumes, namely:

Your app and library projects are in a src subdirectory
Your test projectts are in a test subdirectory
All project files have the same name as their containing folders
There is only a single solution file

If these conventions don't match your requirements, then my builder image won't work for you. But now you know how to create your own builder images using the ONBUILD command.

Summary

In this post I showed how you could use the Docker ONBUILD command to create custom app-builder Docker images. I showed an example image that uses a number of optimisations to create a generalised ASP.NET Core builder image which will restore, build, and test your ASP.NET Core app, as long as it conforms to a number of standard conventions.

Andrew Lock | .NET Escapades Andrew Lock

Creating a generalised Docker image for building ASP.NET Core apps using ONBUILD

The Docker build image to generalise

The `ONBUILD` Docker command

The generalised Docker builder image

Using the generalised build image

Summary

Popular Tags

The Docker build image to generalise

The ONBUILD Docker command

The generalised Docker builder image

Using the generalised build image

Summary

The `ONBUILD` Docker command