.Net 11 Everything

Yesterday, I updated all of the libraries in json-everything to the newly-released .Net 11. And then I ran the tests…

Okay, it wasn’t that dire. There were only four tests that failed, and they all centered around the same problem: numeric conversions.

The upgrade

Every time a new .Net major version drops, everyone races to get their applications and packages supporting it. I’m no different. I want my users to be able to take advantage of the performance increases and abilities the new update enables.

So after adding net11.0 all of the projects, and after updating all of the Nuget packages, I ran the tests as a sanity check.

Four tests failed, only in .Net 11, and they all had to do with expecting a number that they didn’t get. Puzzlingly, they all failed with error output like

Expected: 4.789
Actual: 4.789

I don’t know about you, but that looks like a pass to me.

I dug a little bit and couldn’t find anything on my own. I debugged, and the debugger appeared to show the same value in each variable. After I couldn’t find anything, I resorted to asking AI, and it pointed me to a breaking change in the framework itself.

The breaking change

Despite efforts by the .Net team to avoid introducing breaking changes, the latest release does have some. They’ve been acknowledged here.

The only one that affected these JSON libraries is a change in how doubles and floats are converted to decimals. The short story is that the conversion is more precise now, and that’s the problem.

When assigning a value to a float or a double, say 10.1, the internal representation of that value carries only a certain level of precision, dictated not only by the number of bits used to represent the value, but also by that value’s best approximation in binary. So that simple 10.1 is actually 10.1000003814697265625 as a float and 10.099999999999999644728632119949907064437866 as a double. Not quite exact.

In .Net 10, when you cast that double approximation to a decimal,

double r = 10.1;
decimal v = (decimal)r;

the double is rounded to 15 significant digits and then converted. Because of this rounding, the conversion lost precision, so you end up with 10.1 in the decimal, which is actually what we’d expect looking at the code. But when you think about it, this feels strange: decimal is supposed to be more precise than double, so the value should transfer as precisely as possible. We should get that approximation.

dotnet/runtime #130566 fixed this by changing the conversion algorithm. In linked issue dotnet/runtime #72135, @dakersnar noted that fixing this will break any code expecting the legacy behavior.

However, because this behavior has existed for so long, it is important to recognize that users might have incorrect expectations that we will be breaking with this fix. For example, a test case I found was assuming the double literal 1.23 could be represented exactly and casting it to decimal only validated their false assumption. With the fix applied, the more precise decimal we now get is not what they were expecting.

using System;

double doubleValue = 1.23;
Console.WriteLine("Actually stored as:");
Console.WriteLine(doubleValue.ToString("G99")); // 1.229999999999999982236431605997495353221893310546875

decimal decimalValue = (decimal)doubleValue;

var builder = new System.Text.StringBuilder("Hello");
builder.Append(decimalValue);
Console.WriteLine(builder.ToString()); 
// This currently (incorrectly) prints "Hello1.23"
// With the fix it prints "Hello1.22299999999999999822364316059"
// User should have used decimal literal 1.23m

The impact

When I first saw the PR, I wanted to send the .Net team a strongly worded issue about how this change broke things. How it broke me. However after reading the linked issue and seeing that some had already called it out, I figured there wasn’t really anything I could do about it. I wasn’t going to persuade them to revert this change. So I decided to see what I could do about it on my end. (I think that’s called “growth”.)

At a minimum, I wanted to make landing .Net 11 a little bit softer for my users.

I started with some testing. I wanted to see what my users will experience around updating JsonSchema.Net and/or .Net.

JsonSchema.Net	.Net Runtime	Test Result
v9.6.0 (.Net 8/9/10 build)	.Net 10	passes
v9.6.0 (.Net 8/9/10 build)	.Net 11	fails
v9.7.0 (.Net 8/9/10/11 build)	.Net 10	passes
v9.7.0 (.Net 8/9/10/11 build)	.Net 11	fails

So very clearly, it’s the .Net upgrade that produces the break, not my support of it.

This is the reason the new package bumps are only minor versions. Since the break happens when you update .Net regardless of whether you upgrade the package, the break isn’t in the package.

The remedy

The tests that had failed came in two flavors. First, there was building a JsonNode directly from a value.

JsonNode node = 4.789; // cannot be represented exactly with double

The internals of the JsonValue held on to the double as the underlying value type, and when later comparing to the exact decimal 4.789 that was represented in the schema, the assertion failed. Updating the code to use decimal literals, e.g. 4.789m, fixed these tests. Simple, but I had three of these: two in the JsonSchema.Net tests, and one in the JsonE.Net tests. I imagine this will likely trip other people up as well, even outside of the context of these libraries, but it’s an easy fix.

The second was a bit more hidden. To support runtime (reflective) schema generation from C# types, there are several attributes that take numeric values. For example [Minimum(10)] adds a "minimum": 10 into the generated schema for the property it decorates. However, if we want 4.789, for instance, [Minimum(4.789m)] doesn’t work: decimal is disallowed as an attribute value. (You can read about why in Jon Skeet’s fantastic blog post.) Due to this limitation, the numeric attributes take double, which is then converted to decimal. You can see where this led.

So I’m stuck with double, but in .Net 11, it doesn’t give me the right values. What to do?

At first, I considered just letting the framework do the conversion and letting runtime generation just break, like everyone else’s code was going to do anyway. That could push more people to use the Roslyn source generative approach (the parameter is read as typed in the code rather than converted to a runtime type). But for the runtime generation, there wouldn’t be a way to say [Minimum(4.789)]. It just wouldn’t work because that would round into the double’s precision before being exactly converted to decimal, yielding 4.7889999999999997015720509808. That’s not going to work.

So then I thought maybe I could just reimplement the legacy conversion. But then that would force the old behavior on people who might actually expect the new .Net 11 behavior. That’s not going to work, either.

The best approach I could think of was to let you decide.

And now all of the numeric attributes have a new optional bool parameter (or property) called useNet10CompatibleRounding, which allows you to specify whether you want the old algorithm or the new. I left the default as false so that the framework still decides its own default behavior.

public class Foo
{
    [Minimum(4.789)] // let the framework decide
    public decimal Bar { get; set; }

    [Minimum(4.789, true)] // always use legacy conversion
    public decimal Baz { get; set; }
}

The outcome

I think the approach I landed on is the best of both worlds. If you’re still using the runtime reflective generation, you’re supported moving up to .Net 11 with a way to keep the existing behavior. You still have to change your code, which technically is a breaking behavior, but as stated before: .Net broke it, not me.

I hope to do a performance analysis soon to see what improvements there are with the new framework version. When I have that, I’ll post it here.

If you aren’t generating revenue, you like the work I put out, and you would still like to support the project, please consider becoming a sponsor!