blog post image
Andrew Lock avatar

Andrew Lock

~13 min read

Adding support for collection expressions to your own types

Behind the scenes of collection expressions - Part 5

Share on:

I started this series by providing an introduction to collection expressions. Over the next 3 posts I showed the code that's generated by the compiler when you use collection expressions. In this final post in the series, I show how you can make your own types compatible with collection expressions, and tie up a few loose ends.

Adding collection expression support by supporting collection initializers

I've already discussed in this series that collection expressions are automatically compatible with any concrete type that supports collection initializers. For example, there's no special code in Roslyn to support HashSet<T>, but you can use it with collection expressions:

HashSet<int> values = [1, 2, 3, 4]

behind the scenes, the collection expression generates code that looks a bit like this:

HashSet<int> values = new HashSet<int>();
values.Add(1);
values.Add(2);
values.Add(3);
values.Add(4);
values.Add(5);

Which is the same code as gets generated by the collection initializer version:

HashSet<int> values = new(){ 1, 2, 3, 4 }

This applies to any type that supports a collection initializer and has a public parameterless constructor—the type automatically supports collection expressions too.

Note that I'm only talking about "list-style" collection initializers here; collection expressions don't support dictionary-style collections yet

The minimum requirements to support collection initializers are that you implement IEnumerable, have a public parameterless constructor, and have a public Add(T value) instance method where T is the correct type (or can be generically typed as such).

For example, the following shows a minimum implementation of a custom collection which can be used with collection initializers:

public class MyCollection: IEnumerable // Implementing the non-generic IEnumerable
{
    // Backing collection that contains the data
    private readonly List<int> _list = new();

    // Implement the required member of IEnumerable
    public IEnumerator GetEnumerator() => _list.GetEnumerator();

    // The required Add() method for collection initializers
    public void Add(int val)
    {
        _list.Add(val);
    } 
}

Note that I implemented the non-generic IEnumerable (instead of IEnumerable<T> or IEnumerable<int> for example) but that's only because it's the minimum required; you certainly can (and arguably should) implement the generic version.

With the code above, you can now create collections using the collection initializer syntax:

MyCollection<int> myCollection = new() { 1, 2, 3, 4 };

Which means you can now also use collection expressions:

MyCollection<int> myCollection = [1, 2, 3, 4];

That's pretty neat, but there are some trade-offs here. For example, even though with collection expressions we know ahead of time how many elements there are, there's no way for the compiler (or the MyCollection type) to use that information to optimize the code. Instead you have to call Add() repeatedly.

Also, using a collection initializers means you're forced to use a mutable type, because the type is created using new() and then mutated by calling Add repeatedly. It would be nice if you had the option to not do that.

Unfortunately, for collection initializers, you're stuck, there's no way to satisfy the required API to meet these requirements. But for collection expressions we have a chance!

Using [CollectionBuilder] to create collections

The [CollectionBuilder] attribute was introduced as part of the C#12 collection expressions feature and provides an efficient mechanism for supporting collection expressions in your types. It's easiest to show a minimal implemention of a collection builder and then discuss the limitations, requirements, and ways we can extend it.

The simplest implementation of MyCollection, updated to support [CollectionBuilder], might look something like this:

using System;
using System.Linq;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.CompilerServices;

// Decorate the type with the [CollectionBuilder] type, pointing to 
// a method that the collection expression should call to create the type
[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection
{
    // 👇 This is the method the collection expression calls
    // It must take a ReadOnlySpan<> of the values and return an instance
    // of the collection
    public static MyCollection Create(ReadOnlySpan<int> values) => new(values);

    private readonly int[] _values;
    public MyCollection(ReadOnlySpan<int> values)
    {
        // Because all the values are provided in the constructor, we can
        // use an array backing type instead of a list, which is more efficient
        // to create, and doesn't need to expose a mutation Add() method
        _values = values.ToArray();
    }

    // Must have a GetEnumerator() method that returns an IEnumerator implementation
    public IEnumerator<int> GetEnumerator() => _values.AsEnumerable().GetEnumerator();
}

There's several important points here:

  • The [CollectionBuilder] attribute points to a Type and a named method that will be invoked to create the decorated collection.
  • The collection must be an "iteration type", i.e. it should have a GetEnumerator() method that returns an IEnumerator (or IEnumerator<T>).
    • Normally you would satisfy this by implementation IEnumerable<T>, but I went for the minimal example above!
    • The type T of the IEnumerator<T> returned by GetEnumerator() defines the "element type" for the collection type.
    • If you return a non-generic IEnumerator, the "element type" is object
  • The method pointed to by the [CollectionBuilder] attribute must be static, accessible (e.g. public or internal) and take a single parameter of type ReadOnlySpan<T>, where T is the "element type" of the collection.

By following these rules and adding a [CollectionBuilder], the collection can likely be created much more efficiently. We saw in a previous post that the compiler is able to significantly optimize collection expressions when you're creating a ReadOnlySpan<T>, and ultimately that's what happens here.

To see it in action, if we create a new instance of our newly implemented collection:

MyCollection myCollection = [1, 2, 3, 4];

Then the generated code is much more efficient than when we had with the collection initializer version.

MyCollection myCollection = MyCollection.Create(RuntimeHelpers.CreateSpan<int>((RuntimeFieldHandle)/*OpCode not supported: LdMemberToken*/));

internal sealed class <PrivateImplementationDetails>
{
    [StructLayout(LayoutKind.Explicit, Pack = 4, Size = 16)]
    internal struct __StaticArrayInitTypeSize=16_Align=4
    {
    }

    internal static readonly __StaticArrayInitTypeSize=16_Align=4 CF97ADEEDB59E05BFD73A2B4C2A8885708C4F4F70C84C64B27120E72AB733B724/* Not supported: data(01 00 00 00 02 00 00 00 03 00 00 00 04 00 00 00) */;
}

I'm not going to go into the details in this post, as I discussed it in depth in my previous posts, but ultimately this is very efficient compared to what we had previously. And on top of that we didn't need to expose an Add() mutation method (if we don't want to), and instead we're given all the elements when the type is constructed.

Handling generic collections with [CollectionBuilder]

The MyCollection example I've shown above is a bit contrived as it only supports creating collections of int. Most of the collections we use these days are generic, with a T type parameter. If the collection you want to decorate with [CollectionBuilder] is generic, there's a few changes you need to make.

  • Move the Create() method to a separate, non-generic type. The type referenced in [CollectionBuilder] must not be generic.
  • Update the Create() method to be a generic method that takes a ReadOnlySpan<T>.
  • Update the GetEnumerator() call to return IEnumerator<T> (or better yet, implement IEnumerable<T>)

The resulting updated minimal code might look something like this:

public static class MyCollectionBuilder
{
    // The builder method must be generic, 
    // but must _not_ be in a generic type
    public static MyCollection<T> Create<T>(ReadOnlySpan<T> values) => new(values);
}

// Update [CollectionBuilder] to point to the other type
[CollectionBuilder(typeof(MyCollectionBuilder), nameof(MyCollectionBuilder.Create))]
public class MyCollection<T>(ReadOnlySpan<T> values) // It's a generic type now
{
    // Initializing the generic T[] using primary constructors for brevity
    private readonly T[] _values = values.ToArray();

    // Returning IEnumerator<T>
    public IEnumerator<T> GetEnumerator() => _values.AsEnumerable().GetEnumerator();
}

Usage-wise, the code looks roughly the same, we have just changed the type to be generic, so we've moved from MyCollection to MyCollection<T>:

MyCollection<int> myCollection = [1, 2, 3, 4];

The generated code is essentially identical to the previous non-generic implementation, but now the type works with any element type!

Adding [CollectionBuilder] to interfaces

So far we've looked at adding [CollectionBuilder] to concrete types, but one of the nice features of collection expressions is that you can also use them with interfaces, for example:

IList<int> = [1, 2, 3, 4];
ICollection<int> = [1, 2, 3, 4];

Well you can also do the same thing with your own interfaces if you apply the [CollectionBuilder] attribute to them! Lets extend the previous example to use an interface too. First we'll make our MyCollection type implement an interface IMyCollection<T>:

[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection<T>: IMyCollection<T> // implement an interface
{
    // ... as before
}

I've made the interface generic in this case, but it doesn't need to be. The definition of the interface is shown below. It's just a marker interface for now, the only requirements are:

  • It's decorated with a [CollectionBuilder] attribute that points to a valid constructor method (where the return type of the method is an IMyCollection).
  • It implements IEnumerable<T> (giving an element type of T) or IEnumerable (giving an element type of object).

So adding support to the IMyCollection<T> type, is as simple as this:

[CollectionBuilder(typeof(MyCollectionBuilder), nameof(MyCollectionBuilder.Create))]
public interface IMyCollection: IEnumerable<T>
{
}

Now you can use collection expressions with the interface type, just as you would the concrete type!

IMyCollection myCollection = [1, 2, 3, 4];

Using [CollectionBuilder] in earlier framework versions

The CollectionBuilderAttribute type was added in C#12 along with .NET 8, but what if you're targeting an earlier version of .NET, or you're multi-targeting multiple runtimes? Well the good news is that you can just create your own version of the CollectionBuilderAttribute in your project, and that's good enough to satisfy the compiler!

Lots of C# features that depend on types can be pollyfilled in this way. I typically like to use Simon Cropp's Polyfill library to handle all this for me automatically!

Simply add the following file to your project, and suddenly your collection expressions will work in earlier versions of .NET, even in .NET Framework!

#if !NET8_OR_GREATER
namespace System.Runtime.CompilerServices;

sealed class CollectionBuilderAttribute : Attribute
{
    public CollectionBuilderAttribute(Type builderType, string methodName)
    {
        BuilderType = builderType;
        MethodName = methodName;
    }

    public Type BuilderType { get; }
    public string MethodName { get; }
}
#endif

Note that if you're targeting old versions of .NET Framework, you'll also need to add a reference to System.Memory so that you can use ReadOnlySpan<T>.

And that pretty much covers everything about creating your own collection expressions, but before we go we'll take a short trip of the errors you might encounter while trying to implement a collection builder.

When things don't work quite right…

When I was initially exploring the minimal requirements for collection expressions, I got myself into a bit of a mess. I was exploring the minimal (non-generic) version, but I could not get my code to compile. In case it's useful for other people, I'll walk through all the ways I got it wrong initially! 😅

This is the code I started with:

using System;
using System.Linq;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.CompilerServices;

// create the collection
MyCollection myCollection = [1, 2, 3, 4];

// The definition
[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection(ReadOnlySpan<int> values)
{
    // The builder method
    public static MyCollection Create(ReadOnlySpan<int> values) => new(values);

    private readonly int[] _values = values.ToArray(); // intialize from primary constructor
}

but I was getting the error

error CS9188: 'MyCollection' has a CollectionBuilderAttribute but no element type.

I started digging around on CollectionBuilderAttribute trying to find an element parameter I could pass or something, but there was nothing🤷‍♂️ Hopefully after reading the previous section, the answer will be apparent. I found a hint towards the answer after searching for the error here:

The collection type must have an iteration type. In other words, you can foreach the type as a collection.

I knew that foreach works using duck-typing for the GetEnumerator() method (as opposed to requiring that you implement a specific method) so I tried adding the GetEnumerator() to the type (I'll omit the using and invocations now for brevity):

[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection(ReadOnlySpan<int> values)
{
    public static MyCollection Create(ReadOnlySpan<int> values) => new(values);
    private readonly int[] _values = values.ToArray();

    // Make the type an "iteration type" so it can be used in a foreach
    public IEnumerator GetEnumerator() => _values.GetEnumerator();
}

This solved the CS9188 type, but now I was getting a new error:

error CS9187: Could not find an accessible 'Create' method with the expected signature: a static method with a single parameter of type 'ReadOnlySpan<object>' and return type 'MyCollection'.

This one confused me initially, why was it insisting that Create() take a ReadOnlySpan<object> instead of a ReadOnlySpan<int>?

I got there eventually—it's because I was returning the non-generic IEnumerator instead of an IEnumerator<int>, so the compiler was treating the "element type" as object instead of int.

To solve this one, I decided to just implement IEnumerable<int> so that I didn't get anything else wrong:

[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection(ReadOnlySpan<int> values): IEnumerable<int> // implement IEnumerable<T>
{
    public static MyCollection Create(ReadOnlySpan<int> values) => new(values);
    private readonly int[] _values = values.ToArray();

    // Implement IEnumerable<T>
    IEnumerator<int> IEnumerable<int>.GetEnumerator() => ((IEnumerable<int>)_values).GetEnumerator();
    public IEnumerator GetEnumerator() => _values.GetEnumerator();
}

But this still didn't work 🤦‍♂️ I was still getting the same CS9187 error requiring ReadOnlySpan<object> 😕

After a lot of staring, I finally realised what I'd done. IEnumerable<T> also implements IEnumerable, so you have to implement two GetEnumerator() methods:

  • IEnumerator GetEnumerator() is required by the IEnumerable interface
  • IEnumerator<T> GetEnumerator() is required by the IEnumerable<T> interface

As both methods have the same signature, you have to explicitly implement at least one of them in your type. In my code I had explicitly implemented the IEnumerable<int> interface, which is the opposite of what you'd typically do. By doing so, the collection expression was only "discovering" the IEnumerable version and setting the element type as object. To fix it, I just needed to flip which is explicitly implemented:

[CollectionBuilder(typeof(MyCollection), nameof(Create))]
public class MyCollection(ReadOnlySpan<int> values): IEnumerable<int> // implement IEnumerable<T>
{
    public static MyCollection Create(ReadOnlySpan<int> values) => new(values);
    private readonly int[] _values = values.ToArray();

    // 👇 Need to implicitly implement the IEnumerable<T> implementation...
    public IEnumerator<int> GetEnumerator() => ((IEnumerable<int>)_values).GetEnumerator();
    // 👇 ... and explicitly implement the IEnumerable implementation
    IEnumerator IEnumerable.GetEnumerator() => _values.GetEnumerator();
}

After making that change, finally my code compiled! 🎉 It's also worth noting that the following two options would also have worked:

  • Explicitly implementing both the IEnumerable and IEnumerable<T> methods. The collection expression automatically chooses the more specific IEnumerable<T> version to determine the element type in this case
  • Implement the IEnumerator<int> GetEnumerator() method without implementing IEnumerable<T> at all, as I showed in the minimal example in the previous section.

You might be interested to know that the [CollectionBuilder] attribute is also how the runtime adds support for collection expressions to some built-in types like the Immutable* types such as ImmutableArray and ImmutableList.

But not all collections have been given this love. As I was researching this series I found a number of collections which don't work with collection expressions. In the final section of this series I'll briefly discuss the collections I found that don't work:

Built-in types that don't support collection expressions

The most obvious collections that don't support collection expressions are dictionaries, even though these support collection initializers. The good news is that these will hopefully arrive in a future version of C#, it's mostly a question of deciding on syntax.

So as of now, all the various dictionary types Dictionary<,>, ImmutableDictionary<,>, SortedDictionary<,> etc don't support collection expressions. However, there are also a bunch of other types that you might expect to support collection expressions, but which don't compile.

  • ISet<T>—Many interface types like IList<T> and IReadOnlyCollection<T> can be used directly with collection expressions, but you can't use ISet<T>. I haven't dug into whether there's a specific reason why!
  • FrozenSet<T>—ImmutableSet<T> works with collection expressions, so why doesn't FrozenSet<T>? Well in .NET 9, it does!.
  • SortedList<TKey, TValue>—This one is a bit of an odd one, in that while it is a list, it's effectively initialised like a dictionary, with two generic parameters, and an Add(key, value) instead of value. So we may get this in .NET 9 when we get dictionary expressions.
  • PriorityQueue<TElement, TPriority>—I discussed priority queues in a recent series, and showed that these behave rather differently to "standard" lists. They also don't support collection initializers so I'm not entirely surprised they don't support collection expressions.
  • ConcurrentQueue<T>—I have to say, I expected this to work with collection expressions, especially because ConcurrentBag<T> supports them. But ConcurrentQueue<T> also doesn't support the collection initializer syntax, because it doesn't have an Add() method (just an Enqueue() method). So the "default" collection expression mode of falling back to collection initializers means the code doesn't compile.
  • ConcurrentStack<T>—Just like ConcurrentQueue<T>, ConcurrentStack<T> doesn't have an Add() method (it has Push()), so it doesn't support collection initializers, and hence also doesn't support collection expressions.

The final three types there PriorityQueue<>, ConcurrentQueue<> and ConcurrentStack<> don't support collection initializers but they certainly feel like they could if they used [CollectionBuilder]. So who knows, maybe they'll get support in the future if the demand is high enough!

Summary

In this post I described how to add support for collection expressions to your own types using the [CollectionBuilder] attribute. I showed how to implement the type for both non-generic and generic types, for interfaces, and for earlier versions of .NET, including .NET Framework. Finally, I described some of the collections available in .NET which don't support collection expressions.

That's the end of this series on collection expressions. I hope you've enjoyed this peek behind the curtains to see how collection expressions work, and why you should use them wherever possible in your applications!

  • Buy Me A Coffee
  • Donate with PayPal
Andrew Lock | .Net Escapades
Want an email when
there's new posts?