[API Proposal]: IBitwiseEquatable<T>

[stephentoub]

Background and motivation

Most of our vectorized implementations on MemoryExtensions check RuntimeHelpers.IsBitwiseEquatable to determine whether vectorization is feasible. Today, IsBitwiseEquatable is hardcoded to a finite list of primitive types; #75640 extends that further, but overriding Equals / implementing IEquatable<T> opts you out again. Developers can still get the vectorization benefits of these implementations, but it's awkward and unsafe, e.g. with a type like:

struct MyColor : IEquatable<T> { ... }

instead of:

ReadOnlySpan<MyColor> colors = ...;
MyColor value = ...;
return colors.IndexOf(value);

it's something more like:

ReadOnlySpan<MyColor> colors = ...;
MyColor value = ...;
return MemoryMarshal.Cast<MyColor, int>(colors).IndexOf(Unsafe.As<MyColor, int>(ref value));

and most importantly, it's something someone has to write code for on each use rather than it "just work"ing.

If we instead expose a way for a type to be annotated as "You can trust that my Equals override and/or IEquatable implementation are identical to bitwise equality semantics", then we can special-case IsBitwiseEquatable to recognize this annotation, and everything else just lights up.

API Proposal

namespace System;

public interface IBitwiseEquatable<T> : IEquatable<T>
{
    // no additional methods, it's just a marker interface
}

API Usage

struct MyColor : IBitwiseEquatable<MyColor>
{
    private byte R, G, B, A;

    public bool Equals(MyColor other) => R == other.R && G == other.G && B == other.B && A == other.A;

    public override bool Equals(object obj) => obj is MyColor c && Equals(c);

    public override int GetHashCode() => HashCode.Combine(R, G, B, A);
}

Alternative Designs

It could also be an attribute, e.g.

namespace System;

[AttributeUsage(AttributeTargets.Struct, Inherited = false, AllowMultiple = false)]
public sealed class BitwiseEquatableAttribute : Attribute
{
    public BitwiseEquatableAttribute() { }
}

that could then be applied to the type, e.g.

[BitwiseEquatable]
struct MyColor : IEquatable<MyColor>
{
    private byte R, G, B, A;

    public bool Equals(MyColor other) => R == other.R && G == other.G && B == other.B && A == other.A;

    public override bool Equals(object obj) => obj is MyColor c && Equals(c);

    public override int GetHashCode() => HashCode.Combine(R, G, B, A);
}

Risks

It's yet one more thing a developer implementing equality would need to think about. We'd probably want the C# compiler to emit it in some situations for records, and anything we do around #48733 should also factor it in. We might also want analyzers that try to flag types which implement IBitwiseEquality but don't integrate all fields into its Equals, don't do a simple field-by-field comparison or unsafe-cast-memcmp in their Equals, etc.

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Most of our vectorized implementations on MemoryExtensions check RuntimeHelpers.IsBitwiseEquatable to determine whether vectorization is feasible. Today, IsBitwiseEquatable is hardcoded to a finite list of primitive types; #75640 extends that further, but overriding Equals / implementing IEquatable<T> opts you out again. Developers can still get the vectorization benefits of these implementations, but it's awkward and unsafe, e.g. with a type like:

struct MyColor : IEquatable<T> { ... }
...
instead of:
```C#
ReadOnlySpan<MyColor> colors = ...;
MyColor value = ...;
return colors.IndexOf(value);

it's something more like:

ReadOnlySpan<MyColor> colors = ...;
MyColor value = ...;
return MemoryMarshal.Cast<MyColor, int>(colors).IndexOf(Unsafe.As<MyColor, int>(ref value));

and most importantly, it's something someone has to write code for on each use rather than it "just work"ing.

If we instead expose a way for a type to be annotated as "You can trust that my Equals override and/or IEquatable implementation are identical to bitwise equality semantics", then we can special-case IsBitwiseEquatable to recognize this annotation, and everything else just lights up.

API Proposal

namespace System;

public interface IBitwiseEquatable<T> : IEquatable<T>
{
    // no additional methods, it's just a marker interface
}

API Usage

struct MyColor : IBitwiseEquatable<MyColor>
{
    private byte R, G, B, A;

    public bool Equals(MyColor other) => R == other.R && G == other.G && B == other.B && A == other.A;

    public override bool Equals(object obj) => obj is MyColor c && Equals(c);

    public override int GetHashCode() => HashCode.Combine(R, G, B, A);
}

Alternative Designs

No response

Risks

It's yet one more thing a developer implementing equality would need to think about. We'd probably want the C# compiler to emit it in some situations for records, and anything we do around #48733 should also factor it in.

Author:	stephentoub
Assignees:	-
Labels:	api-suggestion, area-System.Memory
Milestone:	8.0.0

[MichalPetryka]

The biggest issue is that bitwise equality is not just checking all fields, floating point data blocks it too but the thing that'd break it the most is padding between struct fields. With it being platform specific, it's impossible to verify it with an analyzer reliably.

I could rather see the VM being able to check it at runtime or going the other way around with the interfaces and having ISequentialEquatable that'd contain statics operating on spans of the type.

[stephentoub]

With it being platform specific, it's impossible to verify it with an analyzer reliably.

It wouldn't need to verify perfectly. It could flag provably wrong usage.

[MichalPetryka]

An attribute could force some sort of Auto layout with Pack 1 for it, although it'd be a bit awkward and wouldn't play nice with platforms that don't support unaligned loads.

[neon-sunset]

Would it be viable for this interface or attribute to indirectly but publicly expose https://github.com/dotnet/runtime/blob/main/src/coreclr/vm/comutilnative.cpp#L1642 ? So that violating CanCompareBits condition would be a compile time error and require user maybe do some additional work with analyzer hints instead while allowing for custom layout as long as it satisfies the condition?

[stephentoub]

To be clear, my intent wasn't to say "dear runtime, please always use memcmp regardless of whether it's valid", but rather "dear runtime, i'm ok if you use memcmp instead of my Equals". The latter doesn't require the runtime to use memcmp but rather just let's it, and it wouldn't if the type wasn't ammenable, just as it doesn't today in ValueType.Equals if the type's layout isn't ammenable. The analyzer I mention would be to help flag cases where they'd obviously have different semantics, e.g. Equals was only comparing one of multiple fields.

[jkotas]

Alternative Designs

Pattern-match the IL in the Equals method. No public APIs needed, just works, even for existing code.

[stephentoub]

Pattern-match the IL in the Equals method. No public APIs needed, just works, even for existing code.

That sounds fairly fragile, but if we can make it work well enough on enough cases, sure. Do we have existing examples outside of the JIT/compiler where we do that?

[jkotas]

E.g. ILLinker has quite a bit of IL pattern matching.

[stephentoub]

E.g. ILLinker has quite a bit of IL pattern matching.

Thanks, sure, I meant examples in the VM...?

[jkotas]

I do not see anything like this in the VM currently. We used to have it in the past (e.g. for CER). It is not that hard to add ad-hoc IL parser.

Or if we want to have all (fragile) optimizations under src\clr\jit, I guess we can figure out a way how to teach the JIT to do this sort of analysis.

[ericwj]

Pattern matching would be awesome. Especially for countless structs with just one field.

But how will this translate to C#? The following is still not out of the box available:

new ConsoleColor[0].AsSpan().IndexOfAnyExcept(ConsoleColor.Red)

Would it mean the type constraints where T : IEquatable<T>? on the members of MemoryExtensions should be dropped?

I found the current version of C# is actually able to distinguish between MemoryMarshal.IndexOf<T>(..) where T : IEquatable<T>? and MyEnumExtensions.IndexOf<E>(..) where E : struct, Enum which I believe was a problem before.

If MemoryExtensions.IndexOf<T> keeps its type constraint, it'd be possible to put some 600 lines of code like the following in another class and ship it:

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static int IndexOf<E>(this ReadOnlySpan<E> span, E other) where E : unmanaged, Enum => Unsafe.SizeOf<E>() switch {
    1 => MemoryMarshal.Cast<E, byte  >(span).IndexOf(Unsafe.As<E, byte  >(ref other)),
    2 => MemoryMarshal.Cast<E, ushort>(span).IndexOf(Unsafe.As<E, ushort>(ref other)),
    4 => MemoryMarshal.Cast<E, uint  >(span).IndexOf(Unsafe.As<E, uint  >(ref other)),
    8 => MemoryMarshal.Cast<E, ulong >(span).IndexOf(Unsafe.As<E, ulong >(ref other)),
    _ => throw new InvalidCastException()
};

Poke me if you want a copy of mine.