Error identification on complex schemas

[franksl]

Hello,
I have some problems when trying to identify errors on complex schemas.
Suppose you have a "anyOf" clause with many items, for example you can look at the AWS Cloud Formation schema example here https://raw.githubusercontent.com/awslabs/goformation/master/schema/cloudformation.schema.json
.
The "Resources" property is an "anyOf" with many items. If you create a json document where Resources entry (ie "AWS::AmazonMQ::Broker") has a mystyped property (ie "AutoMinorVersionUpgrade" not a boolean) it's very difficult to find where the error is because the library outputs (correctly) a "no valid results for anyOf" error and in the subErrors property you will find a list of all the possible errors.
This list is difficult to parse so finding the "right" error is difficult.

I know this is strictly related to the specific schema so a generic method can be difficult to implement, a suggestion could be to add a parseable property to the errors so that, for example, it's possible to add application specific code to restrict the errors list.
Referring to the example above I could filter all the Resources subschemas that are not "AWS::AmazonMQ::Broker" by inspecting error type, dataPointer and processingPath.

Thanks,
Frank

[vearutop]

I'm sorry, I think I need a simpler example to get your idea. Could you make a sample schema/data/filtering condition?

[vearutop]

Error readability is a relevant topic for JSON Schema community, here is a relevant hot issue fyi: json-schema-org/json-schema-spec#396

Hopefully with next draft we will come up with usable standard solution, and your case might contribute.

[franksl]

Here's a short example:
Schema:

<?php

use Swaggest\JsonSchema\Schema;

require('vendor/autoload.php');

$schema = <<<EOF
{
    "type": "object",
    "properties": {
      "test": {
      	"anyOf": [
        	{
              "type": "object" ,
          	  "properties": {
                "testA": {
                  "type": "object",
                  "properties": {
                    "prop1": {"type": "string"},
                    "prop2": {"type": "string"}
                  },
                  "required": ["prop1", "prop2"],
              	  "additionalProperties": false
                }
              },
              "required": ["testA"],
              "additionalProperties": false
        	},
          {
              "type": "object" ,
          	  "properties": {
                "testB": {
                  "type": "object",
                  "properties": {
                    "prop1": {"type": "string"},
                    "prop2": {"type": "string"}
                  },
                  "required": ["prop1", "prop2"],
              	  "additionalProperties": false
                }
              },
              "required": ["testB"],
              "additionalProperties": false
        	},
          {
              "type": "object" ,
          	  "properties": {
                "testC": {
                  "type": "object",
                  "properties": {
                    "prop1": {"type": "string"},
                    "prop2": {"type": "string"}
                  },
                  "required": ["prop2"],
              	  "additionalProperties": false
                }
              },
              "required": ["testC"],
              "additionalProperties": false
        	}
      	]
      }
    }
}
EOF;
$data = <<<EOF
{
	"test": {
      "testC": {
        "propERR": "a",
        "prop2": "b"
      }
    }
}
EOF;
$schema = Schema::import(json_decode($schema));
$schema->in(json_decode($data));

The error returned by the script is:

PHP Fatal error:  Uncaught Swaggest\JsonSchema\Exception\LogicException: No valid results for anyOf {
  0: Required property missing: testA, data: {"testC":{"propERR":"a","prop2":"b"}} at #->properties:test->anyOf[0]
  1: Required property missing: testB, data: {"testC":{"propERR":"a","prop2":"b"}} at #->properties:test->anyOf[1]
  2: Additional properties not allowed: propERR at #->properties:test->anyOf[2]->properties:testC
} at #->properties:test in /var/www/html/testschema/vendor/swaggest/json-schema/src/Schema.php:475

There are 2 'required property missing' and one 'additional properties not allowed' errors. One idea could be to filter out all the 'required property missing' errors where the path is '#->properties:test->anyOf[*'.
Hope this clarifies the idea.
Thanks for the link, with complex schemas I find very difficult in general to identify the errors, this is indeed not a problem specific of this library.

[franksl]

As a matter of fact in this specific example it should be possible to identify the "testC" schema as the source of the error and output just that reference, am I wrong?

[vearutop]

Thank you for example, I agree flexible filtering mechanism can be indeed helpful for advanced user. I'll try to implement that.