In this post I discuss the new TUnit testing framework, why I ported one of my libraries to use it instead of xUnit, how I did that, and related issues I had to deal with.

What is TUnit

TUnit is a new testing library, similar to xUnit, NUnit, or MSTest, that you can use to write test suites for your .NET applications. As per the TUnit project's README:

TUnit is a next-generation testing framework for C# that outpaces traditional frameworks with source-generated tests, parallel execution by default, and Native AOT support. Built on the modern Microsoft.Testing.Platform, TUnit delivers faster test runs, better developer experience, and unmatched flexibility.

The project is still pretty new, with the very first alpha releases first appearing in 2024, and without a v1.0.0 release yet. That said, according to the current status, "The API is mostly stable", and it certainly seems well developed both from a feature point of view, but also in terms of documentation and guides. More on that later!

Another important aspect is that it uses the newer Microsoft.Testing.Platform experience (instead of the old VSTest runner), so it requires using a recent version of the .NET SDK. Even though it's new, Visual Studio, VS Code, and Rider all support the newer lightweight platform, and hence support TUnit (though you may need to enabled the option in your IDE). Consequently, you'll see your TUnit tests show up in your IDE's test explorer, just as you are used to with other frameworks.

One of the selling points of TUnit is that many of the decisions made in the design mean that it can be fast. That includes source-generation based test discovery and even NativeAOT support. The benchmarks on the TUnit README describe various scenarios, but the one that really stands out is:

A test that takes 50ms to execute, repeated 100 times (including spawning a new process and initialising the test framework)

The first set of benchmarks for this scenario run on an Apple M1 mac, and show a very impressive speed up versus other frameworks (which I think is ultimately due to the increased parallelization used by TUnit):

For me the speed up is a nice benefit, but ultimately it's not the main reason I looked at TUnit or wondered if it was worth converting my existing xUnit-based project to use TUnit…

Why change test frameworks?

In general, I wouldn't recommend changing your test framework. Yes, there are differences between the different frameworks, but many of those differences are cosmetic; it is really worth the hassle for minor changes? How much are you going to gain?

And yet, I considered it 😅 This was actually first driven by an issue I found at work, working on the DataDog .NET Tracer, when we tried to update to using the .NET 10 preview 5 SDK and our xUnit test libraries were failing to discover any tests. Not failing, just failing to run any tests 😬. That issue was ultimately an unrelated distraction, but it made us wonder; what is the chance that we will hit an issue with xUnit at some point and be completely stuck?

That might seem a bit irrational, if it wasn't for the "xUnit v3" issue…

xUnit v2 uses the "xUnit" NuGet package, and is the framework that most people think about when they are talking about xUnit (at least, historically). xUnit v2 supports basically all target frameworks; it supports .NET standard 1.1+ (so all .NET Core) and .NET Framework 4.5.2+.

xUnit.v3 is in some ways an evolution, and in others is a rebuilding of xUnit. It uses a different NuGet package, and only supports .NET Framework 4.7.2 and .NET 8+ target frameworks.

It's obviously entirely up to the maintainers to decide what to support, and they've settled on supporting only the frameworks that Microsoft supports, which is totally reasonable on the face of it. But it gives us a tricky question at work, given that we need to test on a wide variety of older frameworks which are used by customers. Even in my OSS projects I normally favour supporting a larger number of target frameworks for ease of use by consumers, and that range is always going to be larger than xUnit.v3's matrix

All that means that it's unlikely I'll ever adopt xUnit.v3, either at work, or in my personal projects. Which means I'm essentially stuck on xUnit v2 forever. And that makes me nervous, given that xUnit v2 isn't receiving updates any more…

I have some other minor gripes with xUnit which also contribute to the uneasiness, not least is the fact that xUnit.v3 has had 3 major versions in the space of 6 months. That level of churn is something I personally don't want to have to deal with in a test framework.

All of which made me wonder if TUnit could be a potential alternative, so I decided to try it out on one of my OSS projects.

How is TUnit different to xUnit?

One of the nice things about the TUnit docs is that they explicitly try to address the differences between TUnit and other test frameworks. For xUnit, the TUnit docs highlight 5 main points of difficulty:

• Async tests parallel limit. xUnit allows limiting the thread count, but not the number of concurrent tests, because with async tests, those two things are not the same. With TUnit, you can explicitly choose the number of tests to run in parallel.
• Set up and tear downs. xUnit mostly relies on constructors and IDisposable to handle test setup and shutdown. If you want async setup and teardown, you have to implement IAsyncLifetime, and there are some complexities around inheritance. With TUnit you can use all these approaches if you want, but you have an additional option, [BeforeTest] and [AfterTest] attributes.
• Assembly level hooks. xUnit doesn't have an easy way to run something before executing the tests for an assembly, TUnit has [BeforeAssembly] and [AfterAssembly].
• TestContext. xUnit doesn't track the state of the "current" test in things like tear down methods. With TUnit you can inject a context object into tear down methods or use the static TestContext.Current.
• Assertions. xUnit assertions use a somewhat "legacy" Assert.Equal(x, y) format, which makes it hard to know if the x or y is the actual or expected value. TUnit assertions use a fluent style like FluentAssertions or Shouldly.

Now, to be fair to xUnit, many of these complaints are directed towards the v2 version of xUnit. For example, xUnit v3 has a TestContext type similar to TUnit's, and an [AssemblyFixture] attribute for providing assembly-level hooks.

That said, TUnit brings a bunch of extra features to the table too:

• Source generation. Instead of discovering tests at runtime, TUnit relies on source generators to drive test discovery, which makes things much faster at runtime.
• Native AOT support. One benefit to using Native AOT is that you can actually use Native AOT to publish your test projects. For most projects that's likely overkill, but if you're testing an app you're going to deploy with NativeAOT, that could be handy.
• Test ordering. TUnit provides a [DependsOn] attribute that allows providing an implicit ordering to tests, without having to disable parallelization in general or set ordinal Order values.
• Console capture. With xUnit v2, nothing you write to Console is captured in the test output, but TUnit automatically intercepts these logs and correlates them with the current test. For the record, xUnit v3 has an option for this too.

Of course, the really important difference from my point of view is the supported frameworks. As I mentioned before, xUnit v3 only supports .NET Framework 4.7.2 and .NET 8+. TUnit, on the other hand, supports .NET 8+ but also .NET Standard 2.0, which means .NET Framework 4.7.2, but more importantly also .NET Core 2.0+ which is the really killer feature from my point of view.

So with that in mind, I set about seeing what it would be like to convert one of my OSS projects to use TUnit instead of xUnit.

Converting an XUnit project to TUnit

Further kudos is due to the TUnit project: they have a step-by-step guide describing migrating from xUnit to TUnit. And it worked amazingly well! So this section is mostly a rehash of those steps, with examples applied to my repo, and some caveats.

1. Add the TUnit packages

First of all, add the TUnit package to your test projects, for example:

dotnet add package TUnit

At this point you'll have both xUnit and TUnit references in your project.

2. Remove the automatically added global usings

TUnit adds implicit global using statements to your project, however having these as well as xUnit using statements may cause some namespaces clashes. It's important that the projects are building successfully for the next step, so disable the using statements in the test projects:

<PropertyGroup>
    <TUnitImplicitUsings>false</TUnitImplicitUsings>
    <TUnitAssertionsImplicitUsings>false</TUnitAssertionsImplicitUsings>
</PropertyGroup>

You should now be able to build the project, which will also make sure all the TUnit analyzers are running and available.

3. Convert the xUnit usages to TUnit

The next step is the really neat part. TUnit includes an analyzer that helps convert from xUnit to TUnit. And the best bit is that you can run the analyzer and associated code fixer from the command line, and it will convert the project to TUnit for you:

dotnet format analyzers --severity info --diagnostics TUXU0001

I ran this on the two test projects in my library and it did all the grunt work of conversion. It removed the using xunit, changed [Fact] to [Test], and [InlineData] to [Arguments], among various other things:

This wasn't completely successful, in that it left a few xUnit asserts untouched, but as I was generally converting to FluentAssertions anyway, I just went ahead and completed that conversion first.

Another thing to note is that the analyzer performed some general reformatting of some lines (you can see removed blank lines in the screenshot above, for example). This wasn't a big deal for me, but it's something worth bearing in mind.

4. Reinstate the global usings

Now that all the usages are converted, you need to re-enable the TUnit namespaces, so the easiest thing to do is to revert the TUnitImplicitUsings and TUnitAssertionsImplicitUsings properties.

5. Remove unneeded packages

We're almost done. Finally, you can remove all the xUnit packages, including the runner packages, and importantly also the Microsoft.NET.Test.Sdk package. TUnit uses the newer, lightweight Microsoft.Testing.Platform package instead, and references the package transitively, so the only package you need is TUnit.

At this point, you should be able to build and test your project, and if all has gone well, you can test your project with dotnet test, or however you prefer!

Related issues

There were a few minor issues I ran into.

The first one I've already mentioned, in that I had to fix some Assert calls that weren't converted correctly, primarily Assert.Throws<> calls. As described previously, I resolved the issue by converting to FluentAssertions instead, before converting to TUnit.

Another issue I had was with TRX report generation. TUnit and Microsoft.Testing.Platform support TRX report generation by way of an extension package, Microsoft.Testing.Extensions.TrxReport. Theoretically you just need to remove the --logger trx call used with Microsoft.NET.Test.Sdk, and add the --report-trx version required by Microsoft.Testing.Platform.

However, I found that I needed to add --report-trx (and the --results-directory argument) after a -- argument. And related to this, I had to work around the fact that Nuke, my preferred build system, doesn't have first-class support for the Microsoft.Testing.Platform library. All in all I had to make the following changes to the Test stage in my Nuke script:

DotNetTest(s => s
    .SetProjectFile(Solution)
    .SetConfiguration(Configuration)
    .SetProperty("Version", Version)
    .When(IsServerBuild, x => x
-       .SetLoggers("trx")
-       .SetResultsDirectory(TestResultsDirectory))
+       .SetProcessArgumentConfigurator(x=>x
+           .Add("--")
+           .Add("--report-trx")
+           .Add("--results-directory")
+           .Add(TestResultsDirectory)));

The final issue I had was with my use of Verify for snapshot testing. Verify has plugins for multiple test frameworks, including TUnit. However, unfortunately for me, even the earliest version of Verify.TUnit only supports .NET 8+, not earlier versions of .NET Core, which makes it a deal breaker for me.

Verify is another library where I'm indefinitely stuck on old versions (both in OSS and at work) due to the dropping of older frameworks. That, plus a somewhat aggressive approach to breaking changes, means I'm about 12-16 major versions behind the latest, and unable to upgrade 😅

So this left me a little stuck. I considered just removing Verify entirely, seeing as I was only using it in a single test. However I also wanted to consider moving to TUnit for other projects, and converting those projects could end up tricky due to the Verify dependency…

So I did something moderately drastic. I forked Verify at the highest version that I could add a .NET Core 2.1 target to (21.3.0) and uploaded my own version. I don't envisage this going any further or necessarily being a long-term solution, but it may make it simpler for me to convert some of my other libraries to TUnit without having to also swap to a different snapshot testing library!

Summary

In this post I discussed the new TUnit testing framework, looking at some of it's features, and some of its differences from other frameworks. Then I described how the move for xUnit to create xUnit.v3 had prompted me to consider changing the test framework for my open source libraries to TUnit. This isn't something you should take lightly, but the fact that xUnit.v3 on works on .NET 8+ (and .NET Framework) was a blocker for me to use it in my libraries, so I tried a sample conversion for one of my libraries.

For the remainder of the post I described the process of converting an xUnit project to TUnit. I found this to be incredibly smooth, thanks to the TUnit project providing a migration tool, by way of Roslyn analyzers. This process worked smoothly overall, with only relatively minor changes required on my side after the conversion. Overall TUnit seems very interesting, and I shall be watching it with interest, and potentially converting more of my projects across!