Home
json-everything
Cancel

JSON-e Expressions

JSON-e is a data-structure parameterization system for embedding context in JSON objects. At least that’s how they describe it. My take would be that it’s something of an amalgamation between JSONLogic and Jsonnet. It supports expressions, through which it can do a lot of the logic things that JSON Logic gives you, and it can perform templating and data transformations, giving you a lot of what Jsonnet can do. Their docs are really great, and I recommend reading through those. It’s not long, but it still does a good job of covering what JSON-e can do. I’ve also written some docs for how you can use JSON-e in .Net. This post is going to highlight some of the interesting aspects of the expression syntax that I discovered while implementing it. It’s going to take a bit of setup, which is why this post is a bit longer than some of my others. So grab a drink and get comfy because it’s gonna get fun. A brief introduction to JSON-e To start, let’s cover how JSON-e works at a high level. The idea is pretty simple: you have a template and a context. The context is just a JSON object which contains data that may be referenced from the template. The template is a JSON value which contains instructions. Within those instructions can be expressions stored in JSON strings. These expressions are the focus for this post. JSON-e takes the template and the context (JSON in) and gives you a new value (JSON out). What are expressions? Before we get too deep into the weeds, some basic understanding of expressions is warranted. JSON-e expressions are similar to what you might find in most programming languages, but specifically JS or Python. They take some values and perform some operations on those values in order to get a result. The value space follows the basic JSON type system: objects, arrays, numbers, strings, booleans, and null. You get the basic math operators (+, -, *, /, and ** for exponentiation), comparators (<= and friends), and boolean operators (&& and ||). You also get in for checking the contents of arrays and strings, + can concatenate strings, and you get JSON-Path-like value access (.-properties and [] indexers that can take integers, strings, and slices). Operands which are not values are treated as context accessors. That is, symbols that access data contained in the context you provide. This allows expressions like a.b + c[1] where an expected context object might be something like 1 2 3 4 { "a": { "b": 1 }, "c": [ 4, 5, 6 ] } The context While the context that you initially provide to JSON-e is a mere JSON object, as shown above, during processing the context is so much more. There are some other keys that have default values, and they can be overridden by the object you provide. For instance, the property now is assumed to be the ISO 8601 string of the date/time when evaluation begins. This property is used by the $fromNow operator. The effect is that this property is automatically added to the context so that if the template were to reference it directly, e.g. { "$eval": "now" }, the result would just be the date/time string. However, if you were to include a now property in your context, it would override the implicit value. 1 2 3 4 5 { "a": { "b": 1 }, "c": [ 4, 5, 6 ], "now": "2010-08-12T20:35:40+0000" } Furthermore, other operations, e.g. $let, provide their own context data that can override data in your context. But again, this is only within the scope of the operation. Once you leave that operation, its overrides no longer apply. The net effect of all of this is that the context is actually a stack of JSON objects. Looking up a value starts at the top and works its way down until the value is found. In this way, you can think of that default now value as being a low-level context object with just the now key/value. Function support Expressions also support functions, and you get some handy built-in ones, like min() and uppercase(). Each function declares what it expects for parameters and what its output is. And just like operands for the expression operators, arguments to functions can be just about anything, even full expressions. This enables composing functions and passing context values into functions. 1 { "$eval": "min(a + 1, b * 2)" } with the context 1 { "a": 4, "b": 2 } will result in 4. Functions as values This is where it gets really cool. I lied a little above when I said the value space is the JSON data types. Functions are also valid values. This enables being able to pass functions around as data. Many languages have this built in, but it’s not part of JSON. Every implementation will likely be a bit different in how it makes this happen due to the constraints of how JSON is handled in that language, but JSON-e regards this as a very important feature. For example, I could have the template 1 { "$eval": "x(1, 2, 3)" } In this case, x isn’t defined, and it’s expecting the user to supply the function that should run. The only requirement is that the function must take several numbers as parameters. A context for this template could be something like 1 { "x": "min" } min is recognized as the function of the same name, and so that’s what’s called. You can also do this 1 { "$eval": "[min,max][x](1, 2, 3)" } with the context 1 { "x": 1 } This would run the max function from the array of functions that starts the expression, giving 3 as the result. Note that arrays and objects inside expressions aren’t JSON/YAML values, even though it may look like they are. Because their values can be functions or reference the context, they need to be treated as their own thing: expression arrays and expression objects. In .Net But, you may think, json-everything is built on top of the System.Text.Json namespace, specifically focusing on JsonNode, and surely you can’t just put a function in a JsonObject, right? Wrong! You can wrap anything you want in a JsonValue using its static .Create() method, which means you can absolutely add a function to a JsonObject! JSON-e functions are pretty simple: they take a number of JSON parameters and output a single JSON value. They also need to have access to the context. That gives us a signature: 1 JsonNode? Invoke(JsonNode?[] arguments, EvaluationContext context) In order to get this stored in a JsonValue, you could just store the delegate directly, but I found that it was more beneficial to create a base class from which each built-in function could derive. Also, in the base class I could define an implicit cast to JsonValue, which enables easily adding functions directly to nodes! 1 2 3 4 var obj = new JsonObject { ["foo"] = new MinFunction() } At certain points in the implementation, when I need to check to see if a value is a function, I do it just like I’m checking for a string or a number: 1 2 3 4 if (node is JsonValue val && val.TryGetValue(out FunctionDefinition? func)) { // ... } Embedding functions as data was such a neat idea! JSON-e has a requirement that a function MUST NOT be included as a value in the final output. It can be passed around between operators during evaluation; it just can’t come out into the final result. Also, kudos to the System.Text.Json.Nodes design team for allowing JsonValue to wrap anything! I don’t think I’d have been able to support this with my older Manatee.Json models. Custom functions What’s more, JSON-e allows custom functions! That is, you can provide your own functions in the context and call those functions from within expressions! You want a modulus function? JSON-e doesn’t provide that out of the box, but it does let you provide it. In this library, it means providing an instance of JsonFunction (following the naming scheme of JsonValue, JsonArray, and JsonObject) along with a delegate that matches the signature from above. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 var context = new JsonObject { ["mod"] = JsonFunction.Create((parameters, context) => { var a = parameters[0]?.AsValue().GetNumber(); var b = parameters[1]?.AsValue().GetNumber(); return a % b; }) }; var template = new JsonObject { ["$eval"] = "mod(10, 4)" }; var result = JsonE.Evaluate(template, context); // 2 Bringing it all together And finally, the three aspects of JSON-e that I’ve discussed in this post come together in the most beautiful way. The context is a stack of JSON objects. Functions are values. Custom functions can be conveyed via the context. JsonPath.Net also supports custom functions in its expressions. To manage custom functions there, the static FunctionRepository class is used. At first, I wanted to use this same approach for JSON-e. But once I figured out how to embed functions in data, I realized that I could just pre-load all of the functions into another layer of the context. Then the context lookup does all of the work for me! So now, when you begin the evaluation, the context actually looks like this: 1 2 3 4 // top of stack - <user provided context> - { "now": "<evaluation start time>" } - { "min": <min func as a value>, "max": <max func as a value>, ... } Figuring this out was the key that unlocked everything else in my mind. How to include functions in a JSON object was the hard part. Once I realized that, the rest just kinda wrote itself. Introducing JsonE.Net All of this is to say that I’ve had a fun time bringing JSON-e to .Net and the json-everything project. I’ve learned a lot while building it, including aspects of functional programming, the whole putting-anything-into-JsonValue thing, and new ideas around expression parsing. I’ll definitely be revisiting some of the other libs to see where I can apply my new understanding. It’s also been great working with the JSON-e folks, specifically Dustin Mitchell, who has been very accommodating and responsive. He’s done well to create an environment where questions, feedback, and contributions are welcome. This library is now available on Nuget! If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

.Net Decimals are Weird

I’ve discovered another odd consequence of what is probably fully intentional code: 4m != 4.0m. Okay, that’s not strictly true, but it does seem so if you’re comparing the values in JSON. 1 2 3 4 5 6 7 8 var a = 4m; var b = 4.0m; JsonNode jsonA = a; JsonNOde jsonB = b; // use .IsEquivalentTo() from Json.More.Net Assert.True(jsonA.IsEquivalentTo(jsonB)); // fails! What?! This took me so long to find… What’s happening (brother) The main insight is contained in this StackOverflow answer. decimal has the ability to retain significant digits! Even if those digits are expressed in code!! So when we type 4.0m in C# code, the compiler tells System.Decimal that the .0 is important. When the value is printed (e.g. via .ToString()), even without specifying a format, you get 4.0 back. And this includes when serializing to JSON. If you debug the code above, you’ll see that a has a value of 4 while b has a value of 4.0. Even before it gets to the JsonNode assignments. While this doesn’t affect numeric equality, it could affect equality that relies on the string representation of the number (like in JSON). How this bit me In developing a new library for JSON-e support (spoiler, I guess), I found a test that was failing, and I couldn’t understand why. I won’t go into the full details here, but JSON-e supports expressions, and one of the tests has the expression 4 == 3.2 + 0.8. Simple enough, right? So why was I failing this? When getting numbers from JSON throughout all of my libraries, I chose to use decimal because I felt it was more important to support JSON’s arbitrary precision with decimal’s higher precision rather than using double for a bit more range. So when parsing the above expression, I get a tree that looks like this: 1 2 3 4 5 == / \ 4 + / \ 3.2 0.8 where each of the numbers are represented as JsonNodes with decimals underneath. When the system processes 3.2 + 0.8, it gives me 4.0. As I said before, numeric comparisons between decimals work fine. But in these expressions, == doesn’t compare just numbers; it compares JsonNodes. And it does so using my .IsEquivalentTo() extension method, found in Json.More.Net. What’s wrong with the extension? When I built the extension method, I already had one for JsonElement. (It handles everything correctly, too.) However JsonNode doesn’t always store JsonElement underneath. It can also store the raw value. This has an interesting nuance to the problem in that if the JsonNodes are parsed: 1 2 3 4 var jsonA = JsonNode.Parse("4"); var jsonB = JsonNode.Parse("4.0"); Assert.True(jsonA.IsEquivalentTo(jsonB)); the assertion passes because parsing into JsonNode just stores JsonElement, and the comparison works for that. So instead of rehashing all of the possibilities of checking strings, booleans, and all of the various numeric types, I figured it’d be simple enough to just .ToString() the node and compare the output. And it worked… until I tried the expression above. For 18 months it’s worked without any problems. Such is software development, I suppose. It’s fixed now So now I check explicitly for numeric equality by calling .GetNumber(), which checks all of the various .Net number types returns a decimal? (null if it’s not a number). There’s a new Json.More.Net package available for those impacted by this (I didn’t receive any reports). And that’s the story of how creating a new package to support a new JSON functionality showed me how 4 is not always 4. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

Interpreting JSON Schema Output

Cross-posting from the JSON Schema Blog. I’ve received a lot of questions (and purported bugs) and had quite a few discussions over the past few years regarding JSON Schema output, and by far the most common is, “Why does my passing validation contain errors?” Let’s dig in. No Problem Before we get into where the output may be confusing, let’s have a review of a happy path, where either all of the child nodes are valid, so the overall validation is valid, or one or more of the child nodes is invalid, so the overall validation is invalid. These cases are pretty easy to understand, so it serves as a good place to start. 1 2 3 4 5 6 7 8 9 10 { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://json-schema.org/blog/interpreting-output/example1", "type": "object", "properties": { "foo": { "type": "boolean" }, "bar": { "type": "integer" } }, "required": [ "foo" ] } This is a pretty basic schema, where this is a passing instance: 1 { "foo": true, "bar": 1 } with the output: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 { "valid": true, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#", "instanceLocation": "", "annotations": { "properties": [ "foo", "bar" ] }, "details": [ { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo", "instanceLocation": "/foo" }, { "valid": true, "evaluationPath": "/properties/bar", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar", "instanceLocation": "/bar" } ] } All of the subschema output nodes in /details are valid, and the root is valid, and everyone’s happy. Similarly, this is a failing instance (because bar is a string): 1 { "foo": true, "bar": "value" } with the output: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 { "valid": false, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#", "instanceLocation": "", "details": [ { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo", "instanceLocation": "/foo" }, { "valid": false, "evaluationPath": "/properties/bar", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar", "instanceLocation": "/bar", "errors": { "type": "Value is \"string\" but should be \"integer\"" } } ] } The subschema output at /details/1 is invalid, and the root is invalid, and while we may be a bit less happy because it failed, we at least understand why. So is that always the case? Can a subschema that passes validation have failed subschemas? Absolutely! More Complexity There are limitless ways that we can create a schema and an instance that pass it while outputting a failed node. Pretty much all of them have to do with keywords that present multiple options (anyOf or oneOf) or conditionals (if, then, and else). These cases, specifically, have subschemas that are designed to fail while still producing a successful validation outcome. For this post, I’m going to focus on the conditional schema below, but the same ideas pertain to schemas that contain “multiple option” keywords. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://json-schema.org/blog/interpreting-output/exmaple2", "type": "object", "properties": { "foo": { "type": "boolean" } }, "required": ["foo"], "if": { "properties": { "foo": { "const": "true" } } }, "then": { "required": ["bar"] }, "else": { "required": ["baz"] } } This schema says that if foo is true, we also need a bar property, otherwise we need a baz property. Thus, both of the following are valid: 1 { "foo": true, "bar": 1 } 1 { "foo": false, "baz": 1 } When we look at the validation output for the first instance, we get output that resembles the happy path from the previous section: all of the output nodes have valid: true, and everything makes sense. However, looking at the validation output for the second instance (below), we notice that the output node for the /if subschema has valid: false. But the overall validation passed. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 { "valid": true, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#", "instanceLocation": "", "annotations": { "properties": [ "foo" ] }, "details": [ { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/properties/foo", "instanceLocation": "/foo" }, { "valid": false, "evaluationPath": "/if", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if", "instanceLocation": "", "details": [ { "valid": false, "evaluationPath": "/if/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if/properties/foo", "instanceLocation": "/foo", "errors": { "const": "Expected \"\\\"true\\\"\"" } } ] }, { "valid": true, "evaluationPath": "/else", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/else", "instanceLocation": "" } ] } How can this be? Output Includes Why Often more important than the simple result that an instance passed validation is why it passed validation, especially if it’s not the expected outcome. In order to support this, it’s necessary to include all relevant output nodes. If we exclude the failed output nodes from the result, 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 { "valid": true, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#", "instanceLocation": "", "annotations": { "properties": [ "foo" ] }, "details": [ { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/properties/foo", "instanceLocation": "/foo" }, { "valid": true, "evaluationPath": "/else", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/else", "instanceLocation": "" } ] } we see that the /else subschema was evaluated, from which we can infer that the /if subschema MUST have failed. However, we have no information as to why it failed because that subschema’s output was omitted. But looking back at the full output, it’s clear that the /if subschema failed because it expected foo to be true. For this reason, the output must retain the nodes for all evaluated subschemas. It’s also important to note that the specification states that the if keyword doesn’t directly affect the overall validation result. A Note About Format Before we finish up, there is one other aspect of reading output that can be important: format. All of the above examples use the Hierarchical format (formerly Verbose). However, depending on your needs and preferences, you may want to use the List format (formerly Basic). Here’s the output from the simple schema in List format: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 { "valid": false, "details": [ { "valid": false, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#", "instanceLocation": "" }, { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo", "instanceLocation": "/foo" }, { "valid": false, "evaluationPath": "/properties/bar", "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar", "instanceLocation": "/bar", "errors": { "type": "Value is \"string\" but should be \"integer\"" } } ] } This is easy to read and process because all of the output nodes are on a single level. To find errors, you just need to scan the nodes in /details for any that contain errors. Here’s the output from the conditional schema in List format: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 { "valid": true, "details": [ { "valid": true, "evaluationPath": "", "schemaLocation": "https://json-schema.org/blog/interpreting-output/exmaple2#", "instanceLocation": "", "annotations": { "properties": [ "foo" ] } }, { "valid": true, "evaluationPath": "/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/exmaple2#/properties/foo", "instanceLocation": "/foo" }, { "valid": false, "evaluationPath": "/if", "schemaLocation": "https://json-schema.org/blog/interpreting-output/exmaple2#/if", "instanceLocation": "" }, { "valid": true, "evaluationPath": "/else", "schemaLocation": "https://json-schema.org/blog/interpreting-output/exmaple2#/else", "instanceLocation": "" }, { "valid": false, "evaluationPath": "/if/properties/foo", "schemaLocation": "https://json-schema.org/blog/interpreting-output/exmaple2#/if/properties/foo", "instanceLocation": "/foo", "errors": { "const": "Expected \"\\\"true\\\"\"" } } ] } Here, it becomes obvious that we can’t just scan for errors because we have to consider where those errors are coming from. The error in the last output node only pertains to the /if subschema, which (as mentioned before) doesn’t affect the validation result. Wrap-up JSON Schema output gives you all of the information that you need in order to know what the validation result is and how an evaluator came to that result. Knowing how to read it, though, takes understanding of why all the pieces are there. If you have any questions, feel free to ask on the JSON Schema Slack workspace or open a discussion. All output was generated using my online evaluator https://json-everyting.net/json-schema. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

Exploring Code Generation with JsonSchema.Net.CodeGeneration

About a month ago, my first foray into the world of code generation was published with the extension library JsonSchema.Net.CodeGeneration. For this post, I’d like to dive into the process a little to show how it works. Hopefully, this will give better insight on how to use it as well. This library currently serves as an exploration platform for the JSON Scheam IDL Vocab work, which aims to create a new vocabulary designed to help support translating between code and schemas (both ways). Extracting type information The first step in the code generation process is determining what the schema is trying to model. This library uses a complex set of mini-meta-schemas to identify supported patterns. A meta-schema is just a schema that validates another schema. For example, in most languages, enumerations are basically just named constants. The ideal JSON Schema representation of this would be a schema with an enum. So .Net’s System.DayOfWeek enum could be modelled like this: 1 2 3 4 { "title": "DayOfWeek", "enum": [ "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday" ] } To identify this schema as defining an enumeration, we’d need a meta-schema that looks like this: 1 2 3 4 5 6 7 8 9 10 { "type": "object", "title": "MyEnum", "properties": { "enum": { "type": "array" } }, "required": [ "enum" ] } However, in JSON Schema, an enum item can be any JSON value, whereas most languages require strings. So, we also want to ensure that the values of that enum are strings. 1 2 3 4 5 6 7 8 9 10 { "type": "object", "title": "MyEnum", "properties": { "enum": { "items": { "type": "string" } } }, "required": [ "enum" ] } We don’t need to include type or uniqueItems because we know the data is a schema, and its meta-schema (e.g. Draft 2020-12) already has those constraints. We only need to define constraints on top of what the schema’s meta-schema defines. Now that we have the idea, we can expand this by defining mini-meta-schemas for all of the patterns we want to support. There are some that are pretty easy, only needing the type keyword: string number integer boolean And there are some that are a bit more complex: arrays dictionaries custom objects (inheritable and non-inheritable) And we also want to be able to handle references. The actual schemas that were used are listed in the docs. As with any documentation, I hope to keep these up-to-date, but short of that, you can always look at the source. Building type models Now that we have the different kinds of schemas that we want to support, we need to represent them in a sort of type model from which we can generate code. The idea behind the library was to be able to generate multiple code writers that could support just about any language, so .Net’s type system (i.e. System.Type) isn’t quite the right model. The type model as it stands has the following: TypeModel - Serves as a base class for the others while also supporting our simple types. This basically just exposes a type name property. EnumModel - Each value has a name and an integer value derived from the item’s index. ArrayModel - Exposes a property to track the item type. DictionaryModel - Exposes properties to track key and item types. ObjectModel - Handles both open and closed varieties. Each property has a name, a type, and whether it can read/write. Whenever we encounter a subschema or a reference, that represents a new type for us to generate. Lastly, in order to avoid duplication, we set up some equality for type models. With this all of the types supported by this library can be modelled. As more patterns are identified, this modelling system can be expanded as needed. Writing code The final step for code generation is the part everyone cares about: actually writing code. The library defines ICodeWriter which exposes two methods: TransformName() - Takes a JSON string and transforms it into a language-compatible nme. Write() - Renders a type model into a type declaration in the language. There’s really quite a bit of freedom in how this is implemented. The built-in C# writer branches on the different model types and has private methods to handle each one. One aspect to writing types that I hadn’t thought about when I started writing the library was that there’s a difference between writing the usage of a type and writing the declaration of a type. Before, when I thought about code generation, I typically thought it was about writing type declarations: you have a schema, and you generate a class for it. But what I found was that if the properties of an object also use any of the generated types, only the type name needs to be written. For example, for the DayOfWeek enumeration we discussed before, the declaration is 1 2 3 4 5 6 7 8 9 10 public enum DayOfWeek { Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday } But if I have an array of them, I need to generate DayOfWeek[], which only really needs the type name. So my writer needs to be smart enough to write the declaration once and write just the name any time it’s used. There are a couple of other little nuance behaviors that I added in, and I encourage you to read the docs on the capabilities. Generating a conclusion Overall, writing this was an enjoyable experience. I found a simple architecture that seems to work well and is also extensible. My hope is that this library will help inform the IDL Vocab effort back in JSON Schema Land. It’s useful having a place to test things. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

The New JsonSchema.Net

Some changes are coming to JsonSchema.Net: faster validation and fewer memory allocations thanks to a new keyword architecture. The best part: unless you’ve built your own keywords, this probably won’t require any changes in your code. A new keyword architecture? For about the past year or so, I’ve had an idea that I’ve tried and failed to implement several times: by performing static analysis of a schema, some work can be performed before ever getting an instance to validate. That work can then be saved and reused across multiple evaluations. For example, with this schema 1 2 3 4 5 6 7 { "type": "object", "properties": { "foo": { "type": "string" }, "bar": { "type": "number" } } } we know: that the instance must be an object if that object has a foo property, its value must be a string if that object has a bar property, its value must be a number These are the constraints that this schema applies to any instance that it validates. Each constraint is comprised of an instance location and a requirement for the corresponding value. What’s more, most of the time, we don’t need the instance to identify these constraints. This is the basic idea behind the upcoming JsonSchema.Net v5 release. If I can capture these constraints and save them, then I only have to perform this analysis once. After that, it’s just applying the constraints to the instance. Architecture overview With the upcoming changes, evaluating an instance against a schema occurs in two phases: gathering constraints, and processing individual evaluations. For the purposes of this post, I’m going to refer to the evaluation of an individual constraint as simply an “evaluation.” Collecting constraints There are two kinds of constraints: schema and keyword. A schema constraint is basically a collection of keyword constraints, but it also needs to contain some things that are either specific to schemas, such as the schema’s base URI, or common to all the local constraints, like the instance location. A keyword constraint, in turn, will hold the keyword it represents, any sibling keywords it may have dependencies on, schema constraints for any subschemas the keyword defines, and the actual evaluation function. I started with just the idea of a generic “constraint” object, but I soon found that the two had very different roles, so it made sense to separate them. I think this was probably the key distinction from previous attempts that allowed me to finally make this approach work. So for constraints we have this recursive definition that really just mirrors the structural definition represented by JsonSchema and the different keyword classes. The primary difference between the constraints and the structural definition is that the constraints are more generic (implemented by two types) and evaluation-focused, whereas the structural definition is the more object-oriented model and is used for serialization and other things. Building a schema constraint consists of building constraints for all of the keywords that (sub)schema contains. Each keyword class knows how to build the constraint that should represent it, including getting constraints for subschemas and identifying keyword dependencies. Once we have the full constraint tree, we can save that in the JsonSchema object and reuse that work for each evaluation. Evaluation Each constraint object produces an associated evaluation object. Again, there are two kinds: one for each kind of constraint. When constructing a schema evaluation, we need the instance (of course), the evaluation path, and any options to apply during this evaluation. It’s important to recognize that options can change between evaluations; for example, sometimes you may or may not want to validate format. A results object for this subschema will automatically be created. Creating a schema evaluation will also call on the contained keyword constraints to build their evaluations. To build a keyword evaluation, the keyword constraint is given the schema constraint that’s requesting it, the instance location, and the evaluation path. From that, it can look at the instance, determine if the evaluation even needs to run (e.g. is there a value at that instance location?), and create an evaluation object if it does. It will also create schema evaluations for any subschemas. In this way, we get another tree: one built for evaluating a specific instance. The structure of this tree may (and often will) differ from the structure of the constraint tree. For example, when building constraints, we don’t know what properties additionalProperties will need to cover, so we build a template from which we can later create multiple evaluations: one for each property. Or maybe properties contains a property that doesn’t exist in the instance; no evaluation is created because there’s nothing to evaluate. While building constraints only happens once, building evaluations occurs every time JsonSchema.Evaluate() is called. And there was much rejoicing This a lot, and it’s a significant departure from the more procedural approach of previous versions. But I think it’s a good change overall because this new design encapsulates forethought and theory present within JSON Schema and uses that to improve performance. If you find you’re in the expectedly small group of users writing your own keywords, I’m also updating the docs, so you’ll have some help there. If you still have questions, feel free to open an issue or you can find me in Slack (link on the repo readme). I’m also planning a post for the JSON Schema blog which looks at a bit more of the theory of JSON Schema static analysis separately from the context of JsonSchema.Net, so watch for that as well. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

JsonNode's Odd API

1 2 3 4 5 6 var array = new JsonArray { ["a"] = 1, ["b"] = 2, ["c"] = 3, }; This compiles. Why does this compile?! Today we’re going to explore that. What’s wrong? In case you didn’t see it, we’re creating a JsonArray instance and initializing using key/value pairs. But arrays don’t contain key/value pairs; they contain values. Objects contain key/value pairs. 1 2 3 4 5 6 var list = new List<int> { ["a"] = 1, ["b"] = 2, ["c"] = 3, }; This doesn’t compile, as one would expect. So why does JsonArray allow this? Is the collection initializer broken? Collection initializers Microsoft actually has some really good documentation on collection initializers so I’m not going to dive into it here. Have a read through that if you like. The crux of it comes down to when collection initializers are allowed. First, you need to implement IEnumerable<T> and an .Add(T) method (apparently it also works as an extension method). This will enable the basic collection initializer syntax, like 1 var list = new List<int> { 1, 2, 3 }; But you can also enable direct-indexing initialization by adding an indexer. This lets us do thing like 1 2 3 4 5 6 var list = new List<int>(10) { [2] = 1, [5] = 2, [6] = 3 }; More commonly, you may see this used for Dictionary<TKey, TValue> initialization: 1 2 3 4 5 6 var dict = new Dictionary<string, int> { ["a"] = 1, ["b"] = 2, ["c"] = 3, } But, wait… does that mean that JsonArray has a string indexer? JsonArray has a string indexer! It sure does! You can see it in the documentation, right there under Properties. Why?! Why would you define a string indexer on an array type? Well, they didn’t. They defined it and the integer indexer on the base type, JsonNode, as a convenience for people working directly with the base type without having to cast it to a JsonArray or JsonObject first. But now, all of the JsonNode-derived types have both an integer indexer and a string indexer, and it’s really weird. It makes all of this code completely valid: 1 2 3 4 5 6 7 8 9 10 11 JsonValue number = ((JsonNode)5).AsValue(); // can't cast directly to JsonValue _ = number[5]; // compiles but will explode _ = number["five"]; // compiles but will explode JsonArray array = new() { 0, 1, 2, 3, 4, 5, 6 }; _ = array[5]; // fine _ = array["five"]; // compiles but will explode JsonObject obj = new() { ["five"] = 1 }; _ = obj[5]; // compiles but will explode _ = obj["five"]; // fine Is this useful? This seems like a very strange API design decision to me. I don’t think I’d ever trust a JsonNode enough to confidently attempt to index it before checking to see if it can be indexed. Furthermore, the process of checking whether it can be indexed can easily result in a correctly-typed variable. 1 2 if (node is JsonArray array) Console.WriteLine(array[5]); This will probably explode because I didn’t check bounds, but from a type safety point of view, this is SO much better. I have no need to access indexed values directly from a JsonNode. I think this API enables programming techniques that are dangerously close to using the dynamic keyword, which should be avoided at all costs. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

Correction: JSON Path vs JSON Pointer

In my post comparing JSON Path and JSON Pointer, I made the following claim: A JSON Pointer can be expressed as a JSON Path only when all of its segments are non-numeric keys. Thinking about this a bit more in the context of the upcoming JSON Path specification, I realized that this only considers JSON Path segments that have one selector. If we allow for multiple selectors, and the specification does, then we can write /foo/2/bar as: $.foo[2,'2'].bar Why this works The /2 segment in the JSON Pointer says If the value is an array, choose the item at index 2. If the value is an object, choose the value under property “2”. So to write this as a path, we just need to consider both of these options. If the value is an array, we need a [2] to select the item at index 2. If the value is an object, we need a ["2"] to select the value under property “2”. Since the value cannot be both an array and an object, having both of these selectors in a segment [2,"2"] is guaranteed not to cause duplicate selection, and we’re still guaranteed to get a single value. Caveat While this path is guaranteed to yield a single value, it’s still not considered a “Singular Path” according to the syntax definition in the specification. I raised this to the team, and we ended up adding a note to clarify. Summary A thing that I previously considered impossible turned out to be possible. I’ve added a note to the original post summarizing this as well as linking here. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

Parallel Processing in JsonSchema.Net

This post wraps up (for now) the adventure of updating JsonSchema.Net to run in an async context by exploring parallel processing. First, let’s cover the concepts in JSON Schema that allow parallel processing. Then, we’ll look at what that means for JsonSchema.Net as well as my experience trying to make it work. Part of the reason I’m writing this is sharing my experience. I’m also writing this to have something to point at when someone asks why I don’t take advantage of a multi-threaded approach. Parallelization in JSON Schema There are two aspects of evaluating a schema that can be parallelized. The first is by subschema (within the context of a single keyword). For those keywords which contain multiple subschemas, e.g. anyOf, properties, etc, their subschemas are independent from each other, and so evaluating them simultaneously won’t affect the others’ outcomes. These keywords then aggregate the results from their subschemas in some way: anyOf ensures that at least one of the subschemas passed (logical OR). This can be short-circuited to a passing validation when any subschema passes. allOf ensures that all of the subschemas passed (logical AND). This can be short-circuited to a failing validation when any subschema fails. properties and patternProperties map subschemas to object-instance values by key and ensures that those values match the associated subschemas (logical AND, but only for those keys which match). These can also be short-circuited to a failing validation when any subschema fails. The other way schema evaluation can be parallelized is by keyword within a (sub)schema. A schema is built using a collection of keywords, each of which define a constraint. Those constraints are usually independent (e.g. type, minimum, properties, etc.), however some keywords have dependencies on other keywords (e.g. additionalProperties, contains, else, etc.). Organizing the keywords into dependency groups, and then sorting those groups so that each group’s dependencies are run before the group, we find that the keywords in each group can be run in parallel. 1. Keywords with no dependencies We start with keywords which have no dependencies. type minimum/maximum allOf/anyOf/not properties patternProperties if minContains/maxContains None of these keywords (among others) have any impact on the evaluation of the others within this group. Running them in parallel is fine. Interestingly, though, some of these, like properties, patternProperties, and if, are themselves dependencies of keywords not in this set. 2. Keywords with only dependencies on independent keywords Once we have all of the independent keywords processed, we can evaluate the next set of keywords: ones that only depend on the first set. additionalProperties (depends on properties and patternProperties) then/else (depends on if) contains (depends on minContains/maxContains) Technically, if we don’t mind processing some keywords multiple times, we can run all of the keywords in parallel. For example, we can process then and else in the first set if we process if for each of them. JsonSchema.Net seeks to process each keyword once, so it performs this dependency grouping. This then repeats, processing only those keywords which have dependencies that have already been processed. In each iteration, all of the keywords in that iteration can be processed in parallel because their dependencies have completed. The last keywords to run are unevaluatedItems and unevaluatedProperties. These keywords are special in that they consider the results of subschemas in any adjacent keywords, such as allOf. That means any keyword, including keywords defined in third-party vocabularies, are dependencies of these two. Running them last ensures that all dependencies are met. Parallelization in JsonSchema.Net For those who wish to see what this ended up looking like, the issue where I tracking this process is here and the final result of the branch is here. (Maybe someone looking at the changes can find somewhere I went wrong. Additional eyes are always welcome.) Once I moved everything over to async function calls, I started on the parallelization journey by updating AllOfKeyword for subschema parallelization. In doing this, I ran into my first conundrum. The evaluation context Quite a long time ago, in response to a report of high allocations, I updated the evaluation process so that it re-used the evaluation context. Before this change, each subschema evaluation (and each keyword evaluation) would create a new context object based on information in the “current” context, and then the results from that evaluation would be copied back into the “current” context as necessary. The update changed this processes so that there was a single context that maintained a series of stacks to track where it was in the evaluation process. A consequence of this change, however, was that I could only process serially because the context indicated one specific evaluation path at a time. The only way to move into a parallel process (in which I needed to track multiple evaluation paths simultaneously) was to revert at least some of that allocation management, which meant more memory usage again. I think I figured out a good way to do it without causing too many additional allocations by only creating a new context when multiple branches were possible. So that means any keywords that have one a single subschema would continue to use the single context, but any place where the process could branch would create new contexts that only held the top layer of the stacks from the parent context. I updated all of the keywords to use this branching strategy, and it passed the test suite, but for some reason it ran slower. Sync Method optimized Mean Error StdDev Gen0 Gen1 Gen2 Allocated RunSuite False 874.0 ms 13.53 ms 12.65 ms 80000.0000 19000.0000 6000.0000 178.93 MB RunSuite True 837.3 ms 15.76 ms 14.74 ms 70000.0000 22000.0000 8000.0000 161.82 MB Async Method optimized Mean Error StdDev Gen0 Gen1 Gen2 Allocated RunSuite False 1.080 s 0.0210 s 0.0206 s 99000.0000 29000.0000 9000.0000 240.26 MB RunSuite True 1.050 s 0.0204 s 0.0201 s 96000.0000 29000.0000 9000.0000 246.53 MB Investigating this led to some interesting discoveries. Async is not always parallel My first thought was to check whether evaluation was utilizing all of the processor’s cores. So I started up my Task Manager and re-ran the benchmark. Performance tab of the Task Manager during a benchmark run. One core is pegged out completely, and the others are unaffected. That’s not parallel. A little research later, and it seems that unless you explicitly call Task.Run(), a task will be run on the same thread that spawned it. Task.Run() tells .Net to run the code on a new thread. So I updated all of the keywords again to create new threads. Things get weird Before I ran the benchmark again, I wanted to run the test suite to make sure that the changes I made still actually evaluated schemas properly. After all, what good is running really fast if you’re going the wrong direction? Of the 7,898 tests that I run from the official JSON Schema Test Suite, about 15 failed. That’s not bad, and it usually means that I have some data mixed up somewhere, a copy/paste error, or something like that. Running each test on its own, though, they all passed. Running the whole suite again, and 17 would fail. Running all of the failed tests together, and they would all pass. Running the the suite again… 12 failed. Each time I ran the full, it was a different group of less than 20 tests that would fail. And every time, they’d pass if I ran them in isolation or in a smaller group. This was definitely a parallelization problem. I added some debug logging to see what the context was holding. Eventually, I found that for the failed tests, the instance would inexplicably delete all of its data. Here’s some of that logging: 1 2 3 4 5 6 starting /properties - instance root: {"foo":[1,2,3,4]} (31859421) starting /patternProperties - instance root: {"foo":[1,2,3,4]} (31859421) returning /patternProperties - instance root: {} (31859421) returning /properties - instance root: {} (31859421) starting /additionalProperties - instance root: {} (31859421) returning /additionalProperties - instance root: {} (31859421) The “starting” line was printed immediately before calling into a keyword’s .Evaluate() method, and the “returning” line was called immediately afterward. The parenthetical numbers afterward are the hash code (i.e. .GetHashCode()) of the JsonNode object, so you can see that it’s the same object, only the contents are missing. None of my code edits the instance: all access is read only. So I have no idea how this is happening. A few days ago, just by happenstance, this dotnet/runtime PR was merged, which finished off changes in this PR from last year, which resolved multi-threading issues in JsonNode… that I reported! I’m not sure how that slipped by me while working on this. This fix is slated to be included in .Net 8. I finally figure out that if I access the instance before (or immediately after) entering each thread, then it seems to work, so I set about making edits to do that. If the instance is a JsonObject or JsonArray, I simply access the .Count property. This is the simplest and quickest thing I could think to do. That got all of the tests working. Back to our regularly scheduled program With the entire test suite now passing every time I ran it, I wanted to see how we were doing on speed. I once again set up the benchmark and ran it with the Task Manager open. Performance tab of the Task Manager during a benchmark run with proper multi-threading. The good news is that we’re actually multi-threading now. The bad news is that the benchmark is reporting that the test takes twice as long as synchronous processing and uses a lot more memory. Method optimized Mean Error StdDev Gen0 Gen1 Allocated RunSuite False 1.581 s 0.0128 s 0.0120 s 130000.0000 39000.0000 299.3 MB RunSuite True 1.681 s 0.0152 s 0.0135 s 134000.0000 37000.0000 309.65 MB I don’t know how this could be. Maybe touching the instance causes a re-initialization that’s more expensive than I expect. Maybe spawning and managing all of those threads takes more time than the time saved by running the evaluations in parallel. Maybe I’m just doing it wrong. The really shocking result is that it’s actually slower when “optimized.” That is, taking advantage of short-circuiting when possible by checking for the first task that completed with a result that matched a predicate, and then cancelling the others. (My code for this was basically re-inventing this SO answer.) Given this result, I just can’t see this library moving into parallelism anytime soon. Maybe once .Net Framework is out of support, and I move it into the newer .Net era (which contains the threading fix) and out of .Net Standard (which won’t ever contain the fix), I can revisit this. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

The "Try" Pattern in an Async World

Something I ran across while converting JsonSchema.Net from synchronous to asynchronous is that the “try” method pattern doesn’t work in an async context. This post explores the pattern and attempts to explain what happens when we try make the ​method async. What is the “try” method pattern? We’ve all seen various TryParse() methods. In .Net, they’re on pretty much any data type that has a natural representation as a string, typically numbers, dates, and other simple types. When we want to parse that string into the type, we might go for a static parsing method which returns the parsed value. For example, 1 static int Parse(string s) { /* ... */ } The trouble with these methods is that they throw exceptions when the string doesn’t represent the type we want. If we don’t want the exception, we could wrap the Parse() call in a try/catch, but that will incur exception handling costs that we’d like to avoid. The answer is to use another static method that has a slightly different form: 1 static bool TryParse(string s, out int i) { /* ... */ } Here, the return value is a success indicator, and the parsed value is passed as an out parameter. If the parse was unsuccessful, the value in the out parameter can’t be trusted (it will still have a value, though, usually the default for the type). Ideally, this method does more than just wrapping Parse() in a try/catch for you. Instead, it should reimplemented the parsing logic to not throw an exception in the first place. However, calling TryParse() from Parse() and throwing on a failure is the ideal setup for this pair of methods if you want to re-use logic. This pattern is very common for parsing, but it can be used for other operations as well. For example, JsonPointer.Net uses this pattern for evaluating JsonNode instances because of .Net’s decision to unify .Net-null and JSON-null. There needs to be a distinction between “the value doesn’t exist” and “the value was found and is null,” and a .TryEvaluate() method allows this. Why would I need to make this pattern async? As I mentioned in the intro, I came across this when I was converting JsonSchema.Net to async. Specifically, the data keyword implementation uses a set of resolvers to locate the data that is being referenced. Those resolvers implement an interface that defines a .TryResolve() method. 1 bool TryResolve(EvaluationContext context, out JsonNode? node); I have a resolver for JSON Pointers, Relative JSON Pointers, and URIs. Since the entire point of this change was to make URI resolution async, I now have to make this “try” method async. Let’s make the pattern async To make any method support async calls, its return type needs to be a Task. In the case of .TryParse() it needs to return Task<bool>. 1 Task<bool> TryResolve(EvaluationContext context, out JsonNode? node); No problems yet. Let’s go to one of the resolvers and tag it with async so that we can use await for the resolution calls. Oh… that’s not going to work. Since we can’t have out parameters for async methods, we have two options: Implement the method without using async and await. Get the value out another way. I went with the second solution. 1 async Task<(bool, JsonNode?)> TryResolve(EvaluationContext context) { /* ... */ } This works perfectly fine: it gives a success output and a value output. Hooray for tuples in .Net! Later, I started thinking about why out parameters are forbidden in async methods. Why are out parameters forbidden in async methods? Without going into too much detail, when you have an async method, the compiler is actually doing a few transformations for you. Specifically it has to transform your method that looks like it’s returning a bool into one that returns a Task<bool>. This async method 1 2 3 4 5 6 7 async Task<bool> SomeAsyncMethod() { // some stuff await AnotherAsyncMethod(); // some other stuff return true; } essentially becomes 1 2 3 4 5 6 7 8 9 10 Task<bool> SomeAsyncMethod() { // some stuff return Task.Run(AnotherAsyncMethod) .ContinueWith(result => { // some other stuff return true; }); } There are a few other changes and optimizations that happen, but this is the general idea. So when we add an out parameter, 1 2 3 4 5 6 7 8 9 10 Task<bool> SomeAsyncMethod(out int value) { // some stuff return Task.Run(AnotherAsyncMethod) .ContinueWith(result => { // some other stuff return true; }); } it needs to be set before the method returns. That means it can only be set as part of // some stuff. But in the async version, it’s not apparent that value has to be set before anything awaits, so they just forbid having the out parameter in async methods altogether. In the context of my .TryResolve() method, I’d have to set the out parameter before I fetch the URI content, but I can’t do that because the URI content is what goes in the out parameter. Given this new information, it seems the first option of implementing the async method without async/await really isn’t an option. A new pattern While I found musing over the consequences of out parameters in async methods interesting, I think the more significant outcome from this experience is finding a new version of the “try” pattern. 1 2 3 4 Task<(bool, ResultType)> TrySomethingAsync(InputType input) { // ... } It’s probably a pretty niche need, but I hope having this in your toolbox helps you at some point. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!

JSON Schema, But It's Async

The one thing I don’t like about how I’ve set up JsonSchema.Net is that SchemaRegistry.Fetch only supports synchronous methods. Today, I tried to remedy that. This post is a review of those prospective changes. For those who’d like to follow along, take a look at the commit that is the fallout of this change. Just about every line in this diff is a required, direct consequence of just making SchemaRegistry.Fetch async. What is SchemaRegistry? Before we get into the specific change and why we need it, we need to cover some aspects of dereferencing the URI values of keywords like $ref. The JSON Schema specification states … implementations SHOULD NOT assume they should perform a network operation when they encounter a network-addressable URI. That means that, to be compliant with the specification, we need some sort of registry to preload any documents that are externally referenced by schemas. This text addresses the specification’s responsibility around the many security concerns that arise as soon as you require implementations to reach out to the network. By recommending against this activity, the specification avoids those concerns and passes them onto the implementations that, on their own, wish to provide that functionality. JsonSchema.Net is one of a number of implementations that can be configured to perform these “network operations” when they encounter a URI they don’t recognize. This is acceptable to the specification because it is opt-in. In JsonSchema.Net this is accomplished using the SchemaRegistry.Fetch property. By not actually defining a method in the library, I’m passing on those security responsibilities to the user. I actually used to use it to run the test suite. Several of the tests reference external documents through a $ref value that starts with http://localhost:1234/. The referenced documents, however, are just files stored in a dedicated directory in the suite. So in my function, I replaced that URI prefix with the directory, loaded the file, and returned the deserialized schema. Now I just pre-load them all to help the suite run a bit faster. SchemaRegistry.Fetch is declared as an instance property of type Func<Uri, IBaseDocument?>. Really, this acts as a method that fetches documents that haven’t been pre-registered. Declaring it as a property allows the user to define their own method to perform this lookup. As this function returns an IBaseDocument?, it’s synchronous. Why would we want this to be async? The way to perform a network operation in .Net is by creating an HttpClient and calling one of its methods. Funnily, though, all of those methods are… async. One could create a quasi-synchronous method that makes the call and waits for it. 1 2 3 4 5 6 7 8 9 IBaseDocument? Download(Uri uri) { using var client = new HttpClient(); var text = client.GetAsStringAsync(uri).Result; if (text == null) return null; return JsonSchema.FromText(text); } but that isn’t ideal, and, in some contexts, it’s actively disallowed. Attempting to access a task’s .Result in Blazor Web Assembly throws an UnsupportedException, which is why json-everything.net doesn’t yet support fetching referenced schemas, despite it being online, where fetching such documents automatically might be expected. So we need the SchemaRegistry.Fetch property to support an async method. We need it to be of type Func<Uri, Task<IBaseDocument?>>. Then our method can look like this 1 2 3 4 5 6 7 8 9 async Task<IBaseDocument?> Download(Uri uri) { using var client = new HttpClient(); var text = await client.GetAsStringAsync(uri); if (text == null) return null; return JsonSchema.FromText(text); } Making the change Changing the type of the property is simple enough. However this means that everywhere that the function is called now needs to be within an async method… and those methods also need to be within async methods… and so on. Async propagation is real! In the end, the following public methods needed to be changed to async: JsonSchema.Evaluate() IJsonSchemaKeyword.Evaluate() and all of its implementations, which is every keyword, including the ones in the vocabulary extensions SchemaRegistry.Register() SchemaRegistry.Get() IBaseDocument.FindSubschema() The list doesn’t seem that long like this, but there were a lot of keywords and internal methods. The main thing that doesn’t make this list, though, is the tests. Oh my god, there were so many changes in the tests! Even with the vast majority of the over 10,000 tests being part of the JSON Schema Test Suite (which really just has some loading code and a single method), there were still a lot of .Evaluate() calls to update. Another unexpected impact of this change was in the validating JSON converter from a few posts ago. JsonConverter’s methods are synchronous, and I can’t change them. That means I had to use .Result inside the .Read() method. That means the converter can’t be used in a context where that doesn’t work. It’s ready… … but it may be a while before this goes in. All of the tests pass, and I don’t see any problems with it, but it’s a rather large change. I’ll definitely bump major versions for any of the packages that are affected, which is effectively all of the JSON Schema packages. I’ll continue exploring a bit to see what advantages an async context will bring. Maybe I can incorporate some parallelism into schema evaluation. We’ll see. But really I want to get some input from users. Is this something you’d like to see? Does it feel weird at all to have a schema evaluation be async, even if you know you’re not making network calls? How does this impact your code? Leave some comments below or on this issue with your thoughts. If you like the work I put out, and would like to help ensure that I keep it up, please consider becoming a sponsor!