Creating a .NET CLR profiler using C# and NativeAOT with Silhouette

In this post I take Kevin Gosse's Silhouette library for a spin, to see how easy it is to build a basic .NET CLR profiler. And when I say basic, I mean basic—in this post we'll simply log when an assembly is loaded and that's it. I was mostly interested in seeing how easy it was to get up and running.

Kevin already has a 5 part series on writing a .NET profiler in C# in which he describes his Silhouette library, as well as follow up posts using the library to do real work, like measuring UI responsiveness in Resharper and using profiler function hooks. This post is going to be much more basic than that, just a demonstration of the library in action.

I'm also not going to go into great details about the profiling APIs themselves. Kevin covers some of this in his above posts, or alternatively you can see Christophe Nasarre's series of posts on the profiling APIs. In this post we're really not diving in deep like that, we're just going to dip a toe in and see what's possible!

So before we get started, we should first recap: what are the .NET profiling APIs?

What are the .NET profiling APIs?

For 99% of people, working with .NET means staying in a nice managed runtime and never having to worry about native code, other than perhaps the occasional P/Invoke. However, behind the scenes, the .NET base class libraries often interact with native libraries, and the .NET runtime itself is a native application. What's more, both .NET Core and .NET Framework expose a whole suite of unmanaged APIs that you can invoke from native code.

.NET Core documents three main categories of unmanaged APIs:

Debugging APIs for debugging code that runs in the common language runtime (CLR) environment.
Metadata APIs for reading or generating details about modules and types without loading them in the CLR.
Profiling APIs for monitoring a program's execution by the CLR.

In this post we're primarily looking at the final category, the profiling APIs, and will use the metadata APIs in a supporting role.

When you think of "profiling", you probably think about the profiling tools built into Visual Studio, JetBrain's dotTrace, or viewing dotnet-trace traces in PerfView. These "traditional" profilers can use the profiling APIs for their functionality (though there are other approaches too), but the profiling APIs are far more general than that. For example, we use them in the Datadog .NET client library to rewrite methods to include our instrumentation.

The profiling APIs are very powerful, but the real difficulty is that they're unmanaged APIs, which typically means writing C/C++ code to use them. Yuk. Well, that's where Silhouette comes in.

Who needs C when you have NativeAOT?

Microsoft have been working on NativeAOT for many releases, and with each new version of .NET it gets a little bit better. With NativeAOT, you can compile your .NET application to a native, standalone binary. And a native standalone binary is all you need to write a .NET profiler!

The key thing with a NativeAOT binary is that it's fully self-contained. That means that when your profiling binary is loaded, it's running a completely separate .NET runtime from the application being profiled. Yes, that technically means there's two .NET runtimes loaded in the process!

Of course, just compiling .NET as a native binary isn't the only requirement. You also need to make sure your native binary exposes all the correct entrypoints and interfaces such that the .NET runtime can load your library as though it was built using C++. To quote from Kevin's article on Silhouette:

In a nutshell, we need to expose a DllGetClassObject method that will return an instance of IClassFactory. The .NET runtime will call the CreateInstance method on the class factory, which will return an instance of ICorProfilerCallback (or ICorProfilerCallback2, ICorProfilerCallback3, …, depending on which version of the profiling API we want to support). Last but not least, the runtime will call the Initialize method on that instance with an IUnknown parameter that we can use to fetch an instance of ICorProfilerInfo (or ICorProfilerInfo2, ICorProfilerInfo3, …) that we will need to query the profiling API.

If that went right over your head, that's normal😅 These are APIs that very few .NET engineers ever need to work with, and seeing as they're C++ APIs, that's even worse! But that's the point; with Native AOT and the Silhouette library, you don't need to understand all these interactions. The Silhouette library handles the messy work of setting up your entrypoint and exposing .NET types as C++ interfaces.

Of course, Silhouette doesn't let you completely off the hook—you still need to know what the unmanaged APIs are for, how to use them, and how to chain them together. But Silhouette makes it easier to get started, and means you can write your logic in C#, the language you know best, instead of having to wrestle with C++.

Writing a .NET profiler in C#

As an example of how easy Silhouette makes getting started, for the rest of the post we're going to write a simple profiler using .NET that simply prints out the assemblies that are loaded to the console.

Creating the profiler and test projects

We'll start by creating a simple solution. This will consist of two projects: a class library which is our profiler, and a "hello world" test app, which we will profile:

# Create the two projects
dotnet new classlib -o SilhouetteProf
dotnet new console -o TestApp

# Add the projects to a sln file
dotnet new sln
dotnet sln add .\SilhouetteProf\
dotnet sln add .\TestApp\

This gives us our basic project structure. Now we'll add the Silhouette library to our profiler project:

dotnet add package Silhouette --project SilhouetteProf

Next we need to ensure we publish our application using NativeAOT and allow unsafe code. We're not actually going to write any unsafe code ourselves in this test, but Silhouette includes a source generator which does use unsafe.

Open up SilhoutteProj.csproj, and add the two properties

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <RootNamespace>SilhouetteProf</RootNamespace>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <!-- 👇 Add these two  -->
    <PublishAot>true</PublishAot>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Silhouette" Version="3.2.0" />
  </ItemGroup>

</Project>

Now we have our prerequisites, we can start creating our profiler.

Creating the basic profiler

To create a .NET profiler with Silhouette, you create a class that derives from the Silhouette-provided CorProfilerCallbackBase (or CorProfilerCallback2Base, CorProfilerCallback3Base etc, depending on which functionality you need). You then decorate this class with a [Profiler] attribute and provide a unique Guid:

using Silhouette;

namespace SilhouetteProf;

// 👇 Use a new random Guid, don't just use this one! 
[Profiler("9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6")]
internal partial class MyCorProfilerCallback : CorProfilerCallback5Base
{
}

In the example above I chose a random Guid for my profiler, and derived from CorProfilerCallback5Base. Nothing in this post actually needs the "5" from ICorProfilerInfo5, I'm just including it here to demonstrate the pattern.

The [Profiler] attribute above drives a source generator included with Silhouette which generates the boilerplate necessary required by the .NET runtime for creating an IClassFactory. You don't have to use this generated code (if, for example, you need additional logic in your DllGetClassObject method); if you don't want this code, just omit the [Profiler] attribute. The generated code looks like this:

namespace Silhouette._Generated
{
    using System;
    using System.Runtime.InteropServices;

    file static class DllMain
    {
        [UnmanagedCallersOnly(EntryPoint = "DllGetClassObject")]
        public static unsafe HResult DllGetClassObject(Guid* rclsid, Guid* riid, nint* ppv)
        {
            if (*rclsid != new Guid("9fd62131-bf21-47c1-a4d4-3aef5d7c75c6"))
            {
                return HResult.CORPROF_E_PROFILER_CANCEL_ACTIVATION;
            }

            *ppv = ClassFactory.For(new global::SilhouetteProf.MyCorProfilerCallback());
            return HResult.S_OK;
        }
    }
}

We have the skeleton of our profiler, but before we can compile it, we need to implement the Initialize method:

using Silhouette;

namespace SilhouetteProf;

[Profiler("9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6")]
internal partial class MyCorProfilerCallback : CorProfilerCallback5Base
{
    protected override HResult Initialize(int iCorProfilerInfoVersion)
    {
        Console.WriteLine("[SilhouetteProf] Initialize");
        if (iCorProfilerInfoVersion < 5)
        {
            // we need at least ICorProfilerInfo5 and we got < 5
            return HResult.E_FAIL;
        }

        // Call SetEventMask to tell the .NET runtime which events we're interested in
        return ICorProfilerInfo5.SetEventMask(COR_PRF_MONITOR.COR_PRF_MONITOR_ALL);
    }
}

The above code is about the most simple version Initialize method we can write. Silhouette takes care of working out which version of ICorProfilerInfo is available, and passes this as an int to the method. In the above code, we're making sure that we have at least ICorProfilerInfo5 available, so we can call any methods exposed by ICorProfilerInfo5, ICorProfilerInfo4, ICorProfilerInfo3 etc.

You'll notice a lot of HResult values used as return values. Returning error codes is the predominant error handling approach with the native APIs, so you'll be dealing with these a lot. Luckily, Silhouette exposes this as a handy enum for you to use.

Once we've confirmed that the current .NET runtime supports the features we need, we need to tell the runtime which events we're interested in using the COR_PRF_MONITOR enum and the SetEventMask() or SetEventMask2() methods. For simplicity I used ICorProfilerInfo5.SetEventMask() and just enabled all the features.

The ICorProfilerInfo5 field is initialized prior to Initialize being called, based on the available interface version. For example, if iCorProfilerInfoVersion is 7, then all the ICorProfilerInfo* fields up to ICorProfilerInfo7 will be initialized. It's very important you only call interface versions that have been initialized. So if iCorProfilerInfoVersion is 7, don't call ICorProfilerInfo8 or higher!

At this point we could test our profiler, but I'm going to continue with the implementation a bit before we come to testing.

Adding functionality to our profiler

Responding to events with a Silhouette profiler is as easy as overriding a method in the base class. For example we could override the Shutdown method, called when the runtime is shutting down:

protected override HResult Shutdown()
{
    Console.WriteLine("[SilhouetteProf] Shutdown");
    return HResult.S_OK;
}

To add a bit of interest, we're going to override the AssemblyLoadFinished method which is called when an assembly has finished loading (shocking, I know):

protected override HResult AssemblyLoadFinished(AssemblyId assemblyId, HResult hrStatus)
{
    // ...
}

The AssemblyLoadFinished method provides an AssemblyId which we can use to retrieve the name of the assembly by calling another method on ICorProfilerInfo5, GetAssemblyInfo(AssemblyId).

The AssemblyId type is a very thin wrapper around an IntPtr, acting as strongly-typed wrappers around the otherwise-ubiquitous IntPtrs used in the profiling APIs. I'm a big fan of this approach as it eliminates a whole class of mistakes that you could otherwise make of passing a "Class ID" IntPtr to a method expecting an "Assembly ID" IntPtr (for example).

We can call GetAssemblyInfo() easily enough using the ICorProfilerInfo5 field, but this is a good opportunity to look at a common pattern in the Silhouette library, the use of HResult<T>:

protected override HResult AssemblyLoadFinished(AssemblyId assemblyId, HResult hrStatus)
{
     HResult<AssemblyInfoWithName> assemblyInfo = ICorProfilerInfo5.GetAssemblyInfo(assemblyId)
    // ...
}

HResult<T> is effectively a simple result pattern discriminated union. It contains both an HResult and, if the HResult represents success, an object T. This approach is a way of avoiding the common pattern in the profiling APIs of having multiple "out" parameters, and an HRESULT to indicate whether it's valid to use those values, e.g.

HRESULT GetAssemblyInfo(  
    [in]  AssemblyID  assemblyId,  
    [in]  ULONG       cchName,  
    [out] ULONG       *pcchName,  
    [out, size_is(cchName), length_is(*pcchName)]  
          WCHAR       szName[] ,  
    [out] AppDomainID *pAppDomainId,  
    [out] ModuleID    *pModuleId);

The typical pattern when working with the profiling APIs directly is to make the call, check the return value, and then decide whether to continue or not. HResult<T> allows that pattern too, but you can also go YOLO mode. Instead of doing all the checks yourself, you can instead call HResult<T>.ThrowIfFailed() which returns the T if the call was successful, and throws a Win32Exception otherwise. This can make for some dramatically simpler code to read and write, so it's a real win.

Of course, whether you would want to do this with a production grade profiler is a whole other thing. But then, should you really be using anything from this post for production? Probably not 😉

Using the ThrowIfFailed() approach gives us the code below. We try to get the assembly name, and if it's available, print it:

protected override HResult AssemblyLoadFinished(AssemblyId assemblyId, HResult hrStatus)
{
    try
    {
        // Try to get the AssemblyInfoWithName, and if the HResult returns non-success, throw
        AssemblyInfoWithName assemblyInfo = ICorProfilerInfo5.GetAssemblyInfo(assemblyId).ThrowIfFailed();

        Console.WriteLine($"[SilhouetteProf] AssemblyLoadFinished: {assemblyInfo.AssemblyName}");
        return HResult.S_OK;
    }
    catch (Win32Exception ex)
    {
        // GetAssemblyInfo() failed for some reason, weird.
        Console.WriteLine($"[SilhouetteProf] AssemblyLoadFinished failed: {ex}");
        return ex.NativeErrorCode;
    }
}

The gains of ThrowIfFailed() aren't particularly obvious if you just have a single call. Where it really shines is when you want to chain multiple calls. For example, if we wanted to implement ClassLoadStarted, we would need to chain multiple calls, and that's where ThrowIfFailed() comes into its own:

protected override HResult ClassLoadStarted(ClassId classId)
{
    try
    {
        ClassIdInfo classIdInfo = ICorProfilerInfo.GetClassIdInfo(classId).ThrowIfFailed();

        using ComPtr<IMetaDataImport>? metaDataImport = ICorProfilerInfo2
                                                            .GetModuleMetaDataImport(classIdInfo.ModuleId, CorOpenFlags.ofRead)
                                                            .ThrowIfFailed()
                                                            .Wrap();
        TypeDefPropsWithName classProps = metaDataImport.Value.GetTypeDefProps(classIdInfo.TypeDef).ThrowIfFailed();

        Console.WriteLine($"[SilhouetteProf] ClassLoadStarted: {classProps.TypeName}");
        return HResult.S_OK;
    }
    catch (Win32Exception ex)
    {
        Console.WriteLine($"[SilhouetteProf] ClassLoadStarted failed: {ex}");
        return ex.NativeErrorCode;
    }
}

We have three ThrowIfFailed() calls in the above code, which keeps the nice, procedural, flow. We could instead have added three additional if (result != HResult.S_OK) in the code, but that's harder to follow, particularly if you're writing something similar or just prototyping.

OK, we now have enough functionality to take our profiler for a spin!

Testing our new profiler

To test our profiler, we need to do three things

Publish our test app.
Publish our profiler.
Set the required profiling environment variables.

Publish the test app

We'll startup by publishing the test app. This isn't technically required, we could just run the app using dotnet run for example. The difficulty is that this invokes the .NET SDK, which is itself a .NET app, which means we'd end up profiling that too. Which is fine, it's just not what we're trying to do.

We can publish our hello world app using a simple dotnet publish:

❯ dotnet publish .\TestApp\ -c Release 
Restore complete (0.6s)
  TestApp net10.0 succeeded (0.9s) → TestApp\bin\Release\net10.0\publish\

Publish our profiler

Publishing our profiler is similar, but as we're using NativeAOT, we also need to provide a runtime ID. In .NET 10, you can also use the --use-current-runtime option to publish for "whatever runtime you're currently using". As you can see below, the SDK used win-x64 as I'm running on Windows:

❯ dotnet publish .\SilhouetteProf\ -c Release --use-current-runtime
Restore complete (0.6s)
  SilhouetteProf net10.0 win-x64 succeeded (4.2s) → SilhouetteProf\bin\Release\net10.0\win-x64\publish\

Build succeeded in 5.5s

As we're using NativeAOT, the result is a single, self contained dll (plus separate debug symbols). This is our .NET app, compiled as a NativeAOT .NET profiler!

The profiler dll

Setting the profiling environment variables

To attach a profiler to the .NET runtime, you need to set some environment variables. These are different depending on whether you're profiling a .NET Framework or .NET Core app. There are three different variables to set:

For profiling a .NET Framework app:

COR_ENABLE_PROFILING=1—Enable profiling
COR_PROFILER={9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6}—Set to the value of the GUID from the [Profiler] attribute.
COR_PROFILER_PATH=c:\path\to\profiler—Path to the profiler dll

For profiling a .NET Core/.NET 5+ app:

CORECLR_ENABLE_PROFILING=1—Enable profiling
CORECLR_PROFILER={9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6}—Set to the value of the GUID from the [Profiler] attribute.
CORECLR_PROFILER_PATH=c:\path\to\profiler—Path to the profiler dll

There are additional platform-specific versions of the path variable you can set if you need to support multiple platforms.

After publishing the profiler and the app, I copied the absolute path to the profiler dll, and set the required environment variables using powershell.

You technically don't have to use an absolute path for the dll, you can use a relative path, but is that relative to the target app? To the working directory? I prefer to use absolute paths as they're are unambiguous!

$env:CORECLR_ENABLE_PROFILING=1
$env:CORECLR_PROFILER="{9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6}"
$env:CORECLR_PROFILER_PATH="D:\repos\temp\silouette-prof\SilhouetteProf\bin\Release\net10.0\win-x64\publish\SilhouetteProf.dll"

Note that the GUID variable does include the {} surrounding braces. Once the variables are set, we can take our profiler for a spin!

Testing our app with our NativeAOT profiler

When we run our app, the .NET runtime checks the CORECLR_ variables, and loads our NativeAOT profiler, emitting events as the application executes. As each event is raised, we write to the console, and we can see all the assemblies being loaded as the "Hello World!" application runs!

❯ .\TestApp.exe
[SilhouetteProf] Initialize
[SilhouetteProf] AssemblyLoadFinished: System.Private.CoreLib
[SilhouetteProf] AssemblyLoadFinished: TestApp
[SilhouetteProf] AssemblyLoadFinished: System.Runtime
[SilhouetteProf] AssemblyLoadFinished: System.Console
[SilhouetteProf] AssemblyLoadFinished: System.Threading
[SilhouetteProf] AssemblyLoadFinished: System.Text.Encoding.Extensions
[SilhouetteProf] AssemblyLoadFinished: System.Runtime.InteropServices
Hello, World!
[SilhouetteProf] Shutdown

And there we have it, our .NET profiler, written in .NET, works as expected!🎉 Now, this was obviously a very simple implementation, but it showed me how easy it is to use the Silhouette library to get something up and running vastly quicker than if I had to mess with C++.

One thing to bear in mind is that while Silhouette helps with the mechanics of listening to events and interoperating with the C++ interfaces, you still need to know how to use the native APIs. Silhouette helps with the learning curve there, but you'll likely still need to do research for how to achieve what you want.

From my point of view, Silhouette is clearly a handy tool for fulfilling a specific need. You won't necessarily want to use it to produce a production-grade profiler, but for proof of concept or development work, it seems invaluable. Especially if Kevin continues posting practical examples of using Silhouette himself!

Summary

In this post I gave a brief introduction to the unmanaged .NET profiling APIs, and how you would typically interact with these APIs using C++. I then described how you can use .NET to produce a binary that can interact with these APIs instead, giving all the benefits of working in .NET, while still being able to call native APIs.

I then introduced Kevin Gosse's Silhouette library, and showed how this library makes producing a profiler with NativeAOT simple, by deriving from a base class, and overriding the methods you're interested in. I produced a simple profiler, published it, and used it to show all the assemblies loaded by a hello world console application. Overall I was impressed with how simple it was to Silhouette and will likely explore it much more in the future too!

Andrew Lock | .NET Escapades Andrew Lock