In this post I describe some of the changes to Microsoft's .NET Docker images in .NET 8. I describe some differences in tagging, some newly supported image types, as well as breaking changes in the images.

• Support for chiseled containers
• Smaller images and improved R2R with composite images
• Running container images with non-root users
• ASP.NET Core apps now use port 8080 by default
• Dependency changes in alpine .NET 8 images
• Windows images use version-specific tags in .NET 8
• "Bookworm" Debian images are the new default

Support for chiseled containers

In .NET 8 Microsoft have begun shipping docker images built directly on chiseled containers. Chiseled containers are stripped-down versions of Ubuntu, where only the essential packages for running your application are included. There are two main benefits to chiseled containers:

• The images are much smaller. The chiseled version of the .NET Ubuntu images are ~100MB smaller than their un-chiseled equivalents.
• Fewer packages means less attack surface area. Including less software in the docker images means there's less an attacker can do if they somehow manage to compromise your apps!

Microsoft have been producing chiseled Ubuntu images since .NET 6, but until now they were only published to the nightly docker repository. With .NET 8, Microsoft are now publishing the images to the main .NET Docker repositories as well.

Microsoft are currently publishing chiseled containers for the dotnet/runtime, dotnet/runtime-deps, and dotnet/aspnet repositories. Note that there aren't chiseled containers for the dotnet/sdk images. That's partly because the benefits of chiseled containers don't really apply to the SDK images. SDK images are meant for building your apps; chiseled containers are meant for running your app.

To really hammer that home, in chiseled containers you don't have access to any of the following:

• apt—so you can't install any new packages
• curl—so you can't make web requests
• bash—so you can't run any scripts or shell commands!

That last point is an important one. It means that you can't really customise your "final" application docker image much at all seeing as you can't use RUN commands in the dockerfile. You can copy files into it using docker's COPY command, and set environment variables etc, but that's about it!

Note that if you're using multi-stage builds you can still run scripts in the SDK images, it's only in the final "publish" image that you can't run anything. I recently tried these out and realised I couldn't even create a new directory, because I couldn't run mkdir 😅 If you want to read more about chiseled containers I also enjoyed this post by Carlos Pons.

The following .NET 8 images are currently being pushed to the dotnet repositories (these are the least-specific versions of the images, you can also find 3-part version numbers and architecture-specific tags too):

• mcr.microsoft.com/dotnet/runtime-deps:8.0-jammy-chiseled
• mcr.microsoft.com/dotnet/runtime-deps:8.0-jammy-chiseled-extra
• mcr.microsoft.com/dotnet/runtime:8.0-jammy-chiseled
• mcr.microsoft.com/dotnet/aspnet:8.0-jammy-chiseled

You'll note that as well as the -chiseled tag there's also the -chiseled-extra tag for the dotnet/runtime-deps image. The -chiseled-extra image is the same as -chiseled except but with additional localization/globalization data:

• The -chiseled images set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true and don't have any ICU or time zone data. -chiseled-extra doesn't set this.
• The -chiseled-extra images additionally include libicu70_libs, tzdata_zoneinfo, and tzdata_zoneinfo-icu, whereas the -chiseled images don't.

Remember that you can't install extra libraries in a -chiseled image (as there's no package manager), so if you need globalization with an ASP.NET Core app you will need to take one of two approaches:

• Don't use -chiseled images. That way you can install any additional packages you need. The 8.0-jammy (non-chiseled) image already includes the tzdata and libicu70 packages, so you likely don't need to do anything else.
• Build your own ASP.NET Core runtime image based on the -chiseled-extra images. All of the dockerfiles that Microsoft use to build their docker images are public, so you could use the same approach they do, but starting from the chiseled-extra images instead if you wish.

I didn't mention it previously, but Microsoft actually ship a further chiseled docker image:

• mcr.microsoft.com/dotnet/aspnet:8.0-jammy-chiseled-composite

This image is slightly different, in that it uses a different install method for the ASP.NET Core runtime: composite read-to-run images.

Smaller images and improved R2R with composite images

In .NET 8, Microsoft are shipping "composite" ASP.NET Core docker images. As per the GitHub discussion post

Composite ready-to-run images are a more customizable form of ahead-of-time compilation for CoreCLR-based apps. We've configured ready-to-run image generation to be more frugal, valuing container image size above other metrics. However, startup performance is still very good (and likely measureably better when including registry pull cost).

The composite images are not compatible with apps that use NuGet packages that include binaries that are included in .NET, like for example System.Reflection.Metadata.

Ready-to-Run (R2R) is a technology that allows you to ahead-of-time (AOT) compile (as opposed to just-in-time, JIT) your code. There are various complexities and caveats to this, but the upshot is that when you install the .NET runtime on your machine, you're generally using R2R images which contain both managed and native code.

Note that this is very different to the Native AOT which is a focus of ASP.NET Core in .NET 8. With R2R, you still have the managed code and a JIT, whereas with Native AOT you don't!

Composite R2R images take advantage of the fact that you're always going to ship the whole of ASP.NET Core together. The composite R2R images AOT compile the whole ASP.NET Core framework into a single R2R file. This gives two main advantages:

• By producing a single R2R image instead of one for each assembly, the runtime has to do less work loading from multiple files.
• By compiling the whole framework together instead of treating assemblies as independent, the resulting code can be more optimised. With composite images the runtime can, for example, inline methods across assembly boundaries, producing more performant code than is possible if assemblies are individually AOT compiled.

The net result is that that composite images are slightly smaller than their non-composite equivalents. For example, if you compare the size of the aspnet:8.0-jammy-chiseled and aspnet:8.0-jammy-chiseled-composite images, you can see they're a bit smaller:

• mcr.microsoft.com/dotnet/aspnet:8.0-jammy-chiseled-composite—101.5MB
• mcr.microsoft.com/dotnet/aspnet:8.0-jammy-chiseled—109.2MB
• mcr.microsoft.com/dotnet/runtime:8.0-jammy-chiseled—85.3MB

So the composite images are ~7.5MB smaller. That doesn't seem like much, but when remove the size of the runtime image from the equation, it's a difference of 23.9MB vs. 16.2MB, so it's actually quite an improvement!

The composite images won't definitely work for your app, as you can't use NuGet packages that shim behaviour built into the core framework (such as the System.Reflection.Metadata package mentioned above) but otherwise they should work much the same.

Running container images with non-root users

One big change in the .NET images is support for running with a non-root user. This was partly required in order to support chiseled containers, but is also a good idea in and of itself.

By default, when you run a container, you're typically running as the root user. That means if an attacker manages to compromise your container, they'll be able to do pretty much anything in the container. Just as running as root on your desktop machine is not a good idea, you can now easily take the same approach in your containers. It's classic least-privilege and defence-in-depth, so a good idea if you can do it.

All the .NET 8 images have a non-root user enabled, app, which you can use to run as non-root. You can specify this at runtime by adding -u app to your docker run command:

docker run --rm -u app mcr.microsoft.com/dotnet/aspnet:8.0

Alternatively, you can set your docker image to always use the non-root user by default, using the USER command in your dockerfile:

USER $APP_UID

Note that this uses the $APP_UID environment variable set in the runtime-deps base image to be the ID of the non-root user. You would typically use this command as one of the commands in your final published image when building your docker images. For example, in a multi-stage dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /source

COPY . .
RUN dotnet publish-o /app

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app .
# 👇 set to use the non-root USER here
USER $APP_UID 
ENTRYPOINT ["./aspnetapp"]

If you later want to run as root in a docker image that was built to use the non-root user (by specifying USER $APP_ID), you can use -u root in your docker run command to force using the root user

Note that while all the .NET 8 images (except the SDK images) have the capability to run as a non-root user, you will still run as root by default. The exception is for chiseled images which use the non-root app user by default.

If you want to run as a non-root user in Kubernetes you'll need to use the securityContext section in your manifest, as described in this post.

ASP.NET Core apps now use port 8080 by default

Another knock-on effect of supporting chiseled images and non-root users is that the ASP.NET Core images no longer default to using port 80. Port 80 is a "privileged" port that requires running with the root user. With the change to support using a non-root user, you now have to use a different port. In .NET 8, the new default port is port 8080.

That means that if you were previously using docker run commands like the following (exposing port 80 inside the container as port 5000 outside the container):

docker run --rm -it -p 5000:80 aspnetapp

then you'll need to change the 80 to 8080 instead:

docker run --rm -it -p 5000:8080 aspnetapp

Note that the external port where you expose your app doesn't need to change. It's only the port inside your application, which the ASP.NET Core app is listening on, that changes.

If you really want to, you can keep your app listening on port 80, but you need to make sure to use the root user in that case. For example:

docker run --rm -it -p 5000:80 -u root -e ASPNETCORE_HTTP_PORTS=80 aspnetapp

Note that this example uses the new ASPNETCORE_HTTP_PORTS variable which provides an alternative approach of specifying which ports your application should listen on compared to ASPNETCORE_URLS. When you use ASPNETCORE_HTTP_PORTS, your app listens on the specified ports on any address. Using ASPNETCORE_HTTP_PORTS (and ASPNETCORE_HTTPS_PORTS) is generally simpler than using ASPNETCORE_URLS where you need to specify something like ASPNETCORE_URLS=http://*:8080.

A common problem I've seen people (*cough* me *cough*) hit is listening to the loopback address (localhost) inside docker containers. Unfortunately, if you do this, your app won't respond to requests coming from outside the container, so it's typically not what you want to do, as I discussed in a previous post.

The .NET 8 docker images all set ASPNETCORE_HTTP_PORTS=8080 by default. You can override it at runtime as I showed above, or you can change it in your dockerfile. If you use ASPNETCORE_URLS (or DOTNET_URLS) instead, then that takes precedence.

Dependency changes in alpine .NET 8 images

There were some subtle changes to the alpine images in .NET 8, but the most important thing to note is that while previous alpine images (for .NET 6 and .NET 7) were small, they unfortunately didn't include up-to-date packages! Oops.

As it turns out, the Alpine dockerfiles weren't correctly updating packages that were already included in the base image. Previously the base images were using apk add --no-cache, now they use apk add --upgrade --no-cache, which ensures the package is updated, even if it is already installed.

To give a concrete example, the alpine docker image uses apk add --no-cache libssl3 to specify that the libssl3 package is a dependency. However, as this package is already available in alpine, the latest version was not previously being installed.

The upside is you're now (as of August 2023) using actual updated versions of the dependencies instead of potentially insecure older versions. The downside is the images get bigger. This is because docker works by keeping separate versions of files when they change, so when you update a package you're specifying new versions of various files, so the images inevitably get bigger. It's only a few MB though, and it's far more important that the packages are up-to-date!

In addition to updating the packages, the .NET 8 alpine images also remove a couple of packages that are included in the .NET 6 and .NET 7 images:

• Removed ca-certificates. ca-certificates-bundle is already installed, and they're essentially equivalent, so this shouldn't impact anyone.
• Removed krb5-libs. This provides kerberos support, but that's a relatively niche requirement, so it doesn't make sense for the image to include it by default. You can always reinstall the package if you need it.
• Removed libintl package. This package was previously a dependency of the .NET Core runtime itself, but has not been a requirement since .NET 5. Note that this means pre-.NET 5 self-contained apps running in the .NET 8 alpine runtime image will crash with the message Error loading shared library libintl.so.8.

Windows images use version-specific tags in .NET 8

One big change to the docker images in .NET 8 is how Microsoft are tagging the Windows images. In previous versions of .NET, Microsoft released all the docker images as "multi-platform" images, so if you run:

docker pull mcr.microsoft.com/dotnet/aspnet:7.0

Then you get the linux/amd64 when using Linux containers on an x64 machine, and linux/arm64 when using Linux containers on an arm64 machine. Similarly, if you are using Windows containers on an x64 machine you would get a Windows nano-server x64 image.

In .NET 8, they're not going to publish any Windows images using the "simple" 8.0 tag. If you're using Windows containers and you try to pull the dotnet/aspnet:8.0 image you'll get an error:

$ docker pull mcr.microsoft.com/dotnet/aspnet:8.0
no matching manifest for windows/amd64 10.0.22621 in the manifest list entries

Instead, you'll need to use the "specific" tags, which include the image type (nanoserver/windowsservercore) and version (1809/ltsc2022), for example:

docker pull mcr.microsoft.com/dotnet/aspnet:8.0-nanoserver-1809
docker pull mcr.microsoft.com/dotnet/aspnet:8.0-windowsservercore-ltsc2022

The reason behind the change is explained in this issue. It is essentially due to the fact that the "matching algorithm" used to decide which image to pull when using Windows containers is a bit odd and counterintuitive. The windows containers that the .NET runtime images are based on already only provide the "specific" tags, so the .NET images now just follow the same pattern.

One upshot of this change is that you now can't create a simple Dockerfile that can easily be used with both Linux and Windows containers. For example, with .NET 7 you could do something like this:

FROM mcr.microsoft.com/dotnet/runtime:7.0
ENTRYPOINT ["dotnet", "--version"]

But this won't work with .NET 8. Instead, you either need to create separate dockerfiles for each OS, or alternatively use a --build-arg to specify the tag to use in the base image:

# Specify that TAG must be provided at build time
ARG TAG
# Use the TAG argument in the image   👇
FROM mcr.microsoft.com/dotnet/runtime:{TAG}
ENTRYPOINT ["dotnet", "--version"]

You can then specify the tag at build time differently for Linux and Windows, for example:

# For Linux
docker build --build-arg TAG=8.0 .
# For Windows
docker build --build-arg TAG=8.0-nanoserver-ltsc2022 .

I don't imagine this limitation will impact many people. I mostly felt it because of my work wrangling the CI for the Datadog .NET tracer where we test dozens of permutations of docker images 😅

"Bookworm" Debian images are the new default

The final docker update I've found around updates to the supported and default linux distros. The "default" 8.0 image tags are now based on Debian 12 (AKA "Bookworm"). Microsoft regularly update the base images they use for their Docker images, and in this case the change is in anticipation of OpenSSL 1.x moving to EOL. The previous default image uses Debian 11 (AKA "Bullseye") which ships with OpenSSL 1.1.

It's unlikely that moving to Debian 12 will cause any issues for you if you're currently using Debian 11. If you're using your own base images it's worth noting that .NET 8 has dropped support for several Linux distributions, and increased its glibc requirements. The following table shows how the supported distributions have changed from .NET 7 to .NET 8:

In most cases, even if a specific version of a distro isn't supported, .NET 8 will likely still be compatible. The big exception is CentOS 7: .NET 8 will not run on CentOS 7, because the required glibc version has increased to 2.23 (CentOS 7 uses 2.16).

Summary

In this post I covered most of the changes I've found in how the .NET Docker images are produced for .NET 8. If you're doing a straight upgrade from .NET 7 then you shouldn't run into too many issues, as most of the changes are opt-in. Most of the changes are focused on security hygiene; using chiseled containers and non-root users provide extra layers of protection. Other changes (changes to the default port) were made to support the chiseled container effort but apply even if you're using the "standard" images.