Using and authoring .NET tools

In this post I describe some of the complexities around authoring .NET tools, particularly where you don't know which version of the .NET runtime customers will have installed. Finally, I provide some tips for working with and testing .NET tools in a continuous integration (CI) environment.

What are .NET tools?

.NET tools are programs that are distributed via NuGet and can be installed using the .NET SDK. They can either be installed globally on a machine or locally to a specific folder.

There are a number of first-party global tools from Microsoft, like the EF Core tool, but you can also write your own. In the past I've described creating a tool that uses the TinyPNG API to squash images, and a tool for converting web.config files to appsettings.json format. The majority of .NET tools are command-line tools, but there's no reason they need to be. MonoGame's Content Builder tools for example include GUI tools as well.

Working with local tools

Some tools make the most sense as global tools. If they're broadly applicable to multiple applications, and you generally don't need a specific version of the tool for use in different projects (DiffEngineTray is a good example) then global tools make sense. However, that's not always the case. Sometimes the version of the tool does matter, and you want different versions for different applications.

In these cases, local tools are a better option. You can define the tools that are required for a specific project by creating a dotnet-tools manifest. This is a JSON file which lives in your repository and is checked-in to source control. You can create a new tool-manifest by running the following in the root of your repository:

dotnet new tool-manifest

By default, this creates the following manifest JSON file dotnet-tools.json inside the .config folder of your repository:

{
  "version": 1,
  "isRoot": true,
  "tools": { }
}

The initial manifest doesn't include any tools, but you can install new ones by running dotnet tool install (i.e. without the -g or --tool-path flag). So you can, for example, install the Cake tool for your project by running:

> dotnet tool install Cake.Tool

You can invoke the tool from this directory using the following commands: 'dotnet tool run dotnet-cake' or 'dotnet dotnet-cake'.
Tool 'cake.tool' (version '0.35.0') was successfully installed. Entry is added to the manifest file C:\repos\test\.config\dotnet-tools.json.

This updates the manifest by adding the cake.tool reference to the tools section, including the version required (the current latest version - you can update the version manually as required), and the command you need to run to execute the tool (dotnet-cake):

{
  "version": 1,
  "isRoot": true,
  "tools": {
    "cake.tool": {
      "version": "5.0.0",
      "commands": [
        "dotnet-cake"
      ]
    }
  }
}

When a colleague clones the repository and wants to run the Cake tool, they can run the following commands to first restore the tool, and then run it:

# Restore the NuGet packages specifed in the manifest
dotnet tool restore

# Run the tool using one of the following forms:
dotnet tool run dotnet-cake
# or you can use:
dotnet dotnet-cake
# or even shorter:
dotnet cake

Alternatively, as of .NET 10 preview 6, you can use the even simpler dnx or dotnet dnx tools to one-shot run the tools. When the tool is specified in the tool manifest, you can just use dnx and it will automatically use the version specified in the manifest:

# One-shot run the tool using either of the following:
dnx Cake.Tool
dotnet dnx Cake.Tool

.NET tools are basically just a .NET application packed into a NuGet package, so they are subject to all the same requirements. One of the most important aspects is the fact that .NET applications are compiled against a specific runtime. And that's also where things can get a bit tricky.

Ensuring compatibility by multi-targeting

There are a couple of slightly annoying difficulties working with, and authoring, .NET tools. .NET tools are just normal framework-dependent .NET apps, so they are dependent on the correct .NET runtime being available on your machine. As a concrete example, if you build a .NET tool, and it targets net8.0, then you must have the .NET 8 runtime installed on the target machine, regardless of which version of the SDK you install the tool with.

As a consequence, if you want to support any the of the runtimes a customer might have installed on their machine, then you need to build and pack your tool for multiple target frameworks.

That's easy enough to do in principal, as you can "just" add all the target frameworks you need to support in your project's <TargetFrameworks> element in the .csproj file

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <!-- 👇 Targeting ALL the frameworks!-->
    <TargetFrameworks>netcoreapp2.1;netcoreapp3.0;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0</TargetFrameworks>
    <LangVersion>latest</LangVersion>
    <PackAsTool>true</PackAsTool>
    <ToolCommandName>sayhello</ToolCommandName>
  </PropertyGroup>
</Project>

As you can see from the above list, if you really want to support everything, then that's a lot of target frameworks to add. And it's not without its downsides.

For a start, when you create multi-targeted apps like this, you'll generally be limited to only APIs present in the lowest target framework. In the example above, that means .NET Core 2.1 APIs😬

What's more, each target framework you add here increases the size of the NuGet package. When you pack your app, you'll build it for each of the target frameworks, and pack everything into the same NuGet package:

The contents of the package with multiple runtimes packed into the same package

This can significantly increase the size of the package, which isn't generally a problem, except that it makes all restore and dnx operations (for example) slower.

Building and packaging for all the target runtimes that you support is the "safest" approach to supporting the widest range of customers that you can. However, if you're willing to take a little risk there's an alternative approach.

Configuring your tools to roll forward

In the previous section I said that explicitly targeting all the .NET runtimes that you support in a .NET tool is the "best" way to make sure your tool can run on a customer's environment, regardless of which runtime they use.

However, an alternative (and in many ways, complementary) approach, is to not target all these runtime versions. Instead, you allow your application to run with a newer version of the runtime than it was built for, by using the <RollForward> element.

For example, let's say you have a tool that works on .NET 6 and you don't want to have to multi-target it for .NET 7, .NET 8, .NET 9 etc as well. Given that each version of .NET has very high compatibility with the previous version, you could instead only build your tool for .NET 6, and then tell the dotnet host to allow using any runtime that's available for .NET 6 or above. You can do this by setting RollForward=Major in your project file:

<PropertyGroup>
  <TargetFramework>net6.0</TargetFramework>
  <RollForward>Major</RollForward>
</PropertyGroup>

Setting <RollForward> in your project ensures that this property is copied to the runtimeonfig.json file that is deployed with your tool. You can find this inside your NuGet packge; note the rollFoward property below that mirrors the value you set in your project:

{
  "runtimeOptions": {
    "tfm": "net6.0",
    "rollForward": "Major",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "6.0.0"
    },
    "configProperties": {
      "System.Reflection.Metadata.MetadataUpdater.IsSupported": false
    }
  }
}

With the rollForward configured for your tool, as long as someone is able to install your .NET tool (which they can do as long as they have the .NET 6+ SDK installed) then they will be able to run the app, even though you only built and packaged your app for .NET 6.

Note that this isn't completely safe, as the .NET runtime isn't guaranteed to be compatible across major versions. Nevertheless, in practice it's relatively safe, and is generally recommended.

One of the best reasons to set RollForward=Major in your project even if you do pack for multiple target frameworks is to support currently unreleased .NET versions that come out in the future. For example, let's say you have a .NET tool published that supports .NET 9. By default, when .NET 10 comes out, people won't be able to run your tool unless you go back and explicitly add a net10.0 target. By setting RollForward=Major you can ensure there's some support immediately.

Handy `dotnet tool` tips

The final section in this post is a bit of a grab-bag of handy options available when working with .NET tools, particularly when you're doing things in continuous integration (CI) systems. These are generally things I have run into when working with them myself, and they aren't always obvious.

Testing locally built packages with `--source` and `--tool-path`

As mentioned previously, .NET tools are basically just .NET apps, so for the most part you can test them the way you would test any other apps. However, you may also want to explicitly test the final artifact that you're producing, i.e. the .nupkg file.

When you're testing a tool you've produced locally, I recommend using both the --source and --tool-path settings:

--source Specifies where to install the tool from. Point it to a folder containing .nupkg files to install from those packages only, instead of other NuGet sources.
--tool-path Specifies where to install the tool to. The .NET tool will be installed and unpacked to this directory and can then be run from this directory.

For example:

# Install version 1.2.3 of the dd-trace package 
# that is found in the /app/install/ folder
# and install it into the /tool path
dotnet tool install dd-trace \
    --source /app/install/. \
    --tool-path /tool \
    --version 1.2.3

Using both of these settings when you're testing locally-built packages ensures that you are both actually installing the tool that you think you are (instead of accidentally installing from a remote source), and that you're not "polluting" your local NuGet cache with these test files.

Installing pre-release versions with `--prerelease`

If you're producing a package that has a pre-release suffix (i.e. it has a version like 1.0.0-beta or 0.0.1-preview instead of just 1.0.0 or 0.0.1) then you may be surprised to find you can't easily test it locally. This is because you must pass the --prerelease flag when installing a pre-release version:

# The --prerelease flag is required when installing pre-release versions
dotnet tool install dd-trace \
    --source /app/install/. \
    --tool-path /tool \
    --version 1.2.3-preview \
    --prerelease

Note that this flag is only available from .NET 5+ of the .NET SDK

Provide robustness with `--allow-downgrade`

If you're installing a .NET tool in CI you should generally specify a version, to make sure that your CI is repeatable. But what happens if the tool is already installed?

> dotnet tool install -g dotnet-serve --version 1.10.175
The requested version 1.10.175 is lower than existing version 1.10.190.

As shown in the above example, if you try to install a version of a tool that is lower than the currently installed version, this will fail.

I think that historically you actually couldn't install any new version of a tool if it was already installed on the machine, and instead you would have to use dotnet tool update, but as of at least .NET 9 it seems you can technically update to a newer version of a package using the above dotnet tool install command.

The dotnet tool update command works by uninstalling the tool and then installing a new version, so you might think that you can use that instead, but no:

> dotnet tool update -g dotnet-serve --version 1.10.175
The requested version 1.10.175 is lower than existing version 1.10.190.

You get the exact same error message. The key here is that you need to include the --allow-downgrade option when running dotnet tool update

> dotnet tool update -g dotnet-serve --version 1.10.175 --allow-downgrade
Tool 'dotnet-serve' was successfully updated from version '1.10.190' to version '1.10.175'.

Note that dotnet tool install --allow-downgrade also works. It seems like the two commands do exactly the same thing these days, so I don't know why update hasn't been deprecated to be honest 😅

That's the last of my tips for now. In the next post we'll look at some new features coming to .NET tool packages in .NET 10!

Summary

In this post I discussed how to work .NET tools. I described how to install local tools using a tools manifest, and some of the considerations when you're authoring tools. In particular, I discussed the considerations about multi-targeting to ensure maximum compatibility with customer environments and using RollForward=Major to ensure future compatibility. Finally I provided some general tips about using .NET tools, particularly for when you're building and testing .NET tools in CI. In the next post we'll look at some of the new features coming to .NET tools in .NET 10!

Andrew Lock | .NET Escapades Andrew Lock