This post describes how to work around an issue I ran into when logging exceptions that occur inside a scope block in ASP.NET Core. I'll provide a brief background on logging in ASP.NET Core, structured logging, and the concept of scopes. Then I'll show how exceptions can cause you to lose an associated scope, and how to get round this using a neat trick with exception filters.

tl;dr; Exception filters are executed in the same scope as the original exception, so you can use them to write logs in the original context, before the using scope blocks are disposed.

Logging in ASP.NET Core

ASP.NET Core includes logging infrastructure that makes it easy to write logs to a variety of different outputs, such as the console, a file, or the Windows EventLog. The logging abstractions are used through the ASP.NET Core framework libraries, so you can even get log messages from deep inside the infrastructure libraries like Kestrel and EF Core if you like.

The logging abstractions include common features like different event levels, applying unique ids to specific logs, and event categories for tracking which class created the log message, as well as the ability to use structured logging for easier parsing of logs.

Structured logging is especially useful, as it makes finding and diagnosing issues so much easier in production. I'd go as far as to say that it should be absolutely required if you're running an app in production.

Introduction to structured logging

Structured logging basically involves associating key-value pairs with each log entry, instead of a simple string "message". For example, a non-structured log message might look something like:

info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:51919/

This message contains a lot of information, but if it's just stored as a string like this, then it's not easy to search or filter the messages. For example, what if you wanted to find all of the error messages generated by the WebHost class? You could probably put together a regex to extract all the information, but that's a lot of work.

The same method stored as a structured log would essentially be stored as a JSON object making it easily searchable, as something like:

{
    "eventLevel" : "Information",
    "category" : "Microsoft.AspNetCore.Hosting.Internal.WebHost",
    "eventId" : 1,
    "message" : "Request starting HTTP/1.1 GET http://localhost:51919/",
    "protocol" : "HTTP/1.1",
    "method" : "GET",
    "url" : "http://localhost:51919/"
}

The complete message is still there, but you also have each of the associated properties available without having to do any messy string processing. Nicholas Blumhardt has a great explanation of the benefits in this stack overflow answer.

Now, as these logs are no longer simple strings, they can't just be written to the console, or stored in a file - they need dedicated storage. Some of the most popular options are to store the logs in Elastic Search with a Kibana front end, or to use Seq. The Serilog logging provider also supports structured logging, and is typically used to write to both of these destinations.

Nicholas Blumhardt is behind both the Serilog provider and Seq, so I highly recommend checking out his blog if you're interested in structured logging. In particular, he recently wrote a post on how to easily integrate Serilog into ASP.NET Core 2.0 applications.

Adding additional properties using scopes

In some situations, you might like to add the same values to every log message that you write. For example, you might want to add a database transaction id to every log message until that transaction is committed.

You could manually add the id to every relevant message, but ASP.NET Core also provides the concept of scopes. You can create a new scope in a using block, passing in some state you want to log, and it will be written to each log message inside the using block.

You don't have to be using structured logging to use scopes - you can add them to the console logger for example - but they make the most sense in terms of structured logging.

For example, the following sample taken from the serilog-aspnetcore package (the recommended package for easily adding Serilog to ASP.NET Core 2.0 apps) demonstrates multiple nested scopes in the Get() method. Calling _logger.BeginScope<T>(T state) creates a new scope with the provided state.

[Route("api/[controller]")]
public class ScopesController : Controller
{
    ILogger<ScopesController> _logger;

    public ScopesController(ILogger<ScopesController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public IEnumerable<string> Get()
    {
        _logger.LogInformation("Before");

        using (_logger.BeginScope("Some name"))
        using (_logger.BeginScope(42))
        using (_logger.BeginScope("Formatted {WithValue}", 12345))
        using (_logger.BeginScope(new Dictionary<string, object> { ["ViaDictionary"] = 100 }))
        {
            _logger.LogInformation("Hello from the Index!");
            _logger.LogDebug("Hello is done");
        }

        _logger.LogInformation("After");

        return new string[] { "value1", "value2" };
    }
}

Running this application and hitting the action method produces logs similar to the following in Seq:

As you can see, you can store anything as the state parameter T - a string, an integer, or a Dictionary<string, object> of values. Seq handles these scope state values in too different ways:

• integers, strings and formatted strings are added to an array of objects on the Scope property
• Parameters and values from formatted strings, and Dictionary<string, object> are added directly to the log entry as key-value pairs.

Surprise surprise, Nicholas Blumhardt also has a post on what to make of these values, how logging providers should handle them, and how to use them!

Exceptions inside scope blocks lose the scope

Scopes work well for this situation when you want to attach additional values to every log message, but there's a problem. What if an exception occurs inside the scope using block? The scope probably contains some very useful information for debugging the problem, so naturally you'd like to include it in the error logs.

If you can include a try-catch block inside the scope block, then you're fine - you can log the errors and the scope will be included as you'd expect.

But what if the try-catch block surrounds the using blocks? For example, imagine the previous example, but this time we have a try-catch block in the method, and an exception is thrown inside the using blocks:

[Route("api/[controller]")]
public class ScopesController : Controller
{
    ILogger<ScopesController> _logger;

    public ScopesController(ILogger<ScopesController> logger)
    {
        _logger = logger;
    }

    // GET api/scopes
    [HttpGet]
    public IEnumerable<string> Get()
    {
        _logger.LogInformation("Before");
        try
        {
            using (_logger.BeginScope("Some name"))
            using (_logger.BeginScope(42))
            using (_logger.BeginScope("Formatted {WithValue}", 12345))
            using (_logger.BeginScope(new Dictionary<string, object> { ["ViaDictionary"] = 100 }))
            {
                // An unexpected problem!
                throw new Exception("Oops, something went wrong!");
                _logger.LogInformation("Hello from the Index!");
                _logger.LogDebug("Hello is done");
            }

            _logger.LogInformation("After");

            return new string[] { "value1", "value2" };
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "An unexpected exception occured");
            return new string[] { };
        }
    }
}

Obviously this is a trivial example, you could easily put the try-catch block inside the using blocks, but in reality the scope blocks and exception could occur several layers deep inside some service.

Unfortunately, if you look at the error logged in Seq, you can see that the scopes have all been lost. There's no Scope, WithValue, or ViaDictionary properties:

At the point the exception is logged, the using blocks have all been disposed, and so the scopes have been lost. Far from ideal, especially if the scopes contained information that would help debug why the exception occurred!

Using exception filters to capture scopes

So how can we get the best of both worlds, and record the scope both for successful logs and errors? The answer was buried in an issue in the Serilog repo, and uses a "common and accepted form of 'abuse'" by using an exception filter for side effects.

Exception filters are a C# 6 feature that lets you conditionally catch exceptions in a try-catch block:

try
{
  // Something throws an exception
}
catch(MyException ex) when (ex.MyValue == 3)
{
  // Only caught if the expression filter evaluates
  // to true, i.e. if ex.MyValue == 3
}

If the filter evaluates to true, the catch block executes; if it evaluates to false, the catch block is ignored, and the exception continues to bubble up the call stack until it is handled.

There is a lesser known "feature" of exception filters that we can make of here - the code in an exception filter runs in the same context in which the original exception occurred - the stack in unharmed, and is only dumped if the exception filter evaluates to true.

We can use this feature to allow recording the scopes at the location the exception occurs. The helper method LogError(exception) simply writes the exception to the logger when it is called as part of an exception filter using when (LogError(ex)). Returning true means the catch block is executed too, but only after the exception has been logged with its scopes.

[Route("api/[controller]")]
public class ScopesController : Controller
{
    ILogger<ScopesController> _logger;

    public ScopesController(ILogger<ScopesController> logger)
    {
        _logger = logger;
    }

    // GET api/scopes
    [HttpGet]
    public IEnumerable<string> Get()
    {
        _logger.LogInformation("Before");
        try
        {
            using (_logger.BeginScope("Some name"))
            using (_logger.BeginScope(42))
            using (_logger.BeginScope("Formatted {WithValue}", 12345))
            using (_logger.BeginScope(new Dictionary<string, object> { ["ViaDictionary"] = 100 }))
            {
                throw new Exception("Oops, something went wrong!");
                _logger.LogInformation("Hello from the Index!");
                _logger.LogDebug("Hello is done");
            }

            _logger.LogInformation("After");

            return new string[] { "value1", "value2" };
        }
        catch (Exception ex) when (LogError(ex))
        {
            return new string[] { };
        }
    }

    bool LogError(Exception ex)
    {
        _logger.LogError(ex, "An unexpected exception occured");
        return true;
    }
}

Now when the exception occurs, it's logged with all the active scopes at the point the exception occurred (Scope, WithValue, or ViaDictionary), instead of the active scopes inside the catch block.

Summary

Structured logging is a great approach that makes filtering and searching logs after the fact much easier by storing key-value pairs of properties. You can add extra properties to each log by using scopes inside a using block. Every log written inside the using block will include the scope properties, but if an exception occurs, those scope values will be lost.

To work around this, you can use the C# 6 exception filters feature. Exception filters are executed in the same context as the original exception, so you can use them to capture the logging scope at the point the exception occurred, instead of the logging scope inside the catch block.