Provide minor validation for essential schema properties(data,properties)
Schema properties
and data
are essential for records indexing:
{
"schemaInfo": {
"schemaIdentity": {
"authority": "{{data-partition-id}}",
"source": "TestSchema",
"entityType": "testing",
"schemaVersionMajor": 0,
"schemaVersionMinor": 1,
"schemaVersionPatch": 0,
"id": "{{data-partition-id}}:testing:0.1.0"
},
"createdBy": "OSDU Data Definition Group",
"scope": "SHARED",
"status": "DEVELOPMENT"
},
"schema": {
"",
"type": "object",
"properties": {
...............
},
"data": {
"allOf": [
..............
}
And they will be validated by the Indexer service since it's not possible to index records without them. : https://community.opengroup.org/osdu/platform/system/indexer-service/-/blob/master/indexer-core/src/main/java/org/opengroup/osdu/indexer/schema/converter/SchemaToStorageFormatImpl.java#L94
if (schemaServiceSchema.getProperties() == null) {
throw new SchemaProcessingException(String.format("Schema doesn't have data section, kind: %s", kind));
}
PropertiesData schemaData = schemaServiceSchema.getProperties().getData();
if (Objects.isNull(schemaData)) {
throw new SchemaProcessingException(String.format("Schema doesn't have properties section, kind: %s", kind));
}
Similar topics were raised several times, for example, #82
But unlike previous issues, the current one suggests adding only minor validation, similar to Indexer, to verify that at least essential blocks are present in the schema, which are properties
and data
.
Decision
- Add validation to Schema service, it could be done in SchemaResolver.java
Consequences
- Schema service will reject such malformed schemas
- It's possible that some test cases from community collections must be reworked
- The number of unprocessable indexing events will be reduced.