Schema issueshttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues2024-01-08T13:43:28Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/144Schema Bootstrap failure issue2024-01-08T13:43:28ZIsha KumariSchema Bootstrap failure issueThe system schema has experienced substantial growth, expanding from an initial range of 200-300 to approximately 900. Consequently, I've been facing failures in the schema during script execution. Notably, the script is designed to rei...The system schema has experienced substantial growth, expanding from an initial range of 200-300 to approximately 900. Consequently, I've been facing failures in the schema during script execution. Notably, the script is designed to reinitiate the schema upon encountering failures
Additionally, the Bootstrap process is showing failures randomly due to the schema variations. To address this, I have implemented error-catching mechanisms and introduced a retry strategy to mitigate the impact of these intermittent failures.M22 - Release 0.25Isha KumariIsha Kumarihttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/143OSDU-DD-Delivery-M22 (v0.25.0)2023-12-11T06:12:12ZThomas Gehrmann [slb]OSDU-DD-Delivery-M22 (v0.25.0)- [x] Update to the M22 deliverables from OSDU Data Definitions
CC @chad- [x] Update to the M22 deliverables from OSDU Data Definitions
CC @chadThomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/139Add /liveness_check2023-11-21T19:08:35ZRiabokon Stanislav(EPAM)[GCP]Add /liveness_checkNeed to add the endpoint '/liveness_check' in order to verify the operational status of the Schema Service.Need to add the endpoint '/liveness_check' in order to verify the operational status of the Schema Service.M22 - Release 0.25Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/138OSDU-DD-Delivery-M21 (v0.24.0)2023-11-24T08:44:39ZChad LeongOSDU-DD-Delivery-M21 (v0.24.0)- [x] Update to the M21 deliverables from OSDU Data Definitions- [x] Update to the M21 deliverables from OSDU Data DefinitionsM21 - Release 0.24https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/134OSDU-DD-Delivery-M20 (v0.23.0)2023-08-21T19:12:47ZThomas Gehrmann [slb]OSDU-DD-Delivery-M20 (v0.23.0)- [x] Update to the M20 deliverables from OSDU Data Definitions- [x] Update to the M20 deliverables from OSDU Data DefinitionsM20 - Release 0.23Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/131OSDU-DD-Delivery-M19 (v0.22.0)2023-08-17T08:30:47ZThomas Gehrmann [slb]OSDU-DD-Delivery-M19 (v0.22.0)- [x] Update to the M19 deliverables from OSDU Data Definitions- [x] Update to the M19 deliverables from OSDU Data DefinitionsM19 - Release 0.22Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/127OSDU-DD-Delivery-M18 (v0.21.0)2023-05-19T06:04:20ZThomas Gehrmann [slb]OSDU-DD-Delivery-M18 (v0.21.0)- [x] Update to the M18 deliverables from OSDU Data Definitions- [x] Update to the M18 deliverables from OSDU Data DefinitionsM18 - Release 0.21Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/125OSDU-DD-Delivery-M17 (v0.20.0)2023-03-29T14:04:40ZThomas Gehrmann [slb]OSDU-DD-Delivery-M17 (v0.20.0)- [x] Update to the M17 deliverables from OSDU Data Definitions- [x] Update to the M17 deliverables from OSDU Data DefinitionsM17 - Release 0.20Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/122OSDU-DD-Delivery-M16 (v0.19.0)2023-02-15T12:59:00ZThomas Gehrmann [slb]OSDU-DD-Delivery-M16 (v0.19.0)- [x] Publish the OSDU Data Definitions schema registration resources for M16 v0.19.0- [x] Publish the OSDU Data Definitions schema registration resources for M16 v0.19.0M16 - Release 0.19Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/118OSDU-DD-Delivery-M15 (v0.18.0)2023-01-26T07:15:21ZThomas Gehrmann [slb]OSDU-DD-Delivery-M15 (v0.18.0)- [x] Publish the OSDU Data Definitions schema registration resources for M15 v0.18.0- [x] Publish the OSDU Data Definitions schema registration resources for M15 v0.18.0M15 - Release 0.18Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/114Failing to register WellboreMarkerSet.1.2.1.json schema2022-10-18T05:47:03ZDanylo Vanin (EPAM)Failing to register WellboreMarkerSet.1.2.1.json schemaWhile trying to register shared `WellboreMarkerSet.1.2.1.json` [(link)](https://community.opengroup.org/osdu/platform/system/schema-service/-/blob/master/deployments/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json...While trying to register shared `WellboreMarkerSet.1.2.1.json` [(link)](https://community.opengroup.org/osdu/platform/system/schema-service/-/blob/master/deployments/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json) schema using [DeploySharedSchemas.py](https://community.opengroup.org/osdu/platform/system/schema-service/-/blob/master/deployments/scripts/DeploySharedSchemas.py) - registration fails with the following error:
```
Error with kind osdu:wks:work-product-component--WellboreMarkerSet:1.2.1: Message: Patch version validation failed. Changes requiring a minor or major version increment were found; analysed version: 1.2.1 and 1.2.0. Updating the schema version to a higher minor or major version is required.
```
For the same request application logs the following:
```
schema.app: Failed to resolve the schema and find breaking changes. Reason :Patch version validation failed. Changes requiring a minor or major version increment were found; analysed version: {0} and {1}. Updating the schema version to a higher minor or major version is required.
schema.app: Patch version validation failed. Changes requiring a minor or major version increment were found; analysed version: 1.2.0 and 1.2.1. Updating the schema version to a higher minor or major version is required.
```https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/113OSDU-DD-Delivery-M142022-12-13T00:18:55ZThomas Gehrmann [slb]OSDU-DD-Delivery-M14Publish the shared OSDU schemas for M14 deployment.Publish the shared OSDU schemas for M14 deployment.M14 - Release 0.17Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/80Refresh DDSC schemas2022-01-05T11:34:34ZThomas Gehrmann [slb]Refresh DDSC schemasCreate a MR to update the shared OSDU standard schemas based on the DDSC master state November 23rd.
- [x] VirtualProperty definitions in schemas
- [x] Updated `x-osdu-indexing` tag values to improvde searchCreate a MR to update the shared OSDU standard schemas based on the DDSC master state November 23rd.
- [x] VirtualProperty definitions in schemas
- [x] Updated `x-osdu-indexing` tag values to improvde searchM10 - Release 0.13Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/68SchemaResolver class is not testable for resolution of `https://` related refs2022-11-24T11:45:44ZAman VermaSchemaResolver class is not testable for resolution of `https://` related refsProblem:
===
The SchemaResolver class performs the task of resolving all kinds of references provided in request. The references are provided with `$ref` keyword. The references could be internal one, pointing to a schema within the syst...Problem:
===
The SchemaResolver class performs the task of resolving all kinds of references provided in request. The references are provided with `$ref` keyword. The references could be internal one, pointing to a schema within the system or it could be external reference, like this one '"$ref": "https://geojson.org/schema/FeatureCollection.json"'. These are resolved by making https calls to given URL, like this:
```
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpGet getRequest = new HttpGet(url);
getRequest.addHeader("accept", "application/json");
HttpResponse response = httpClient.execute(getRequest);
```
Now, this code works absolutely fine, but it's not testable in unit tests.
Ask:
===
Move out this logic to a resolve external schemas to a new class, something like `ExternalSchemaResolver` and use it in `has-a` relationship with `SchemaResolver`.
@pbehede assigning this to you. Please reassign accordingly.Paresh BehedeParesh Behedehttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/65M8 Data Definitions content2021-09-22T20:31:55ZThomas Gehrmann [slb]M8 Data Definitions content- [ ] Publish the new types and (decorative) changes from the data Definitions member GitLab to the bootstrap resources.- [ ] Publish the new types and (decorative) changes from the data Definitions member GitLab to the bootstrap resources.M9 - Release 0.12Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/63White source issue2022-11-24T11:10:20ZAbhishek Kumar (SLB)White source issueSecurity Vulnerability
General Details Name Severity CVSS 3 Score CVSS 2 Score Date Modified
CVE-2021-22118 High 7.8 4.6 27-05-2021 21-07-2021
In Spring Framework, versions 5.2.x prior to 5.2.15 and versions 5.3.x prior to 5.3.7, a Web...Security Vulnerability
General Details Name Severity CVSS 3 Score CVSS 2 Score Date Modified
CVE-2021-22118 High 7.8 4.6 27-05-2021 21-07-2021
In Spring Framework, versions 5.2.x prior to 5.2.15 and versions 5.3.x prior to 5.3.7, a WebFlux application is vulnerable to a privilege escalation: by (re)creating the temporary storage directory, a locally authenticated malicious user can read or modify files that have been uploaded to the WebFlux application, or overwrite arbitrary files with multipart request data.Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/61Support for version upgrade during schema-validation2022-02-25T14:13:08ZAbhishek Kumar (SLB)Support for version upgrade during schema-validationThere is intensive use of `$ref` to schema fragments, which are incoming as schema-IDs like e.g. `"$ref": "osdu:wks:AbstractSpatialLocation:1.0.0"`. These fragments use semantic versioning as well.
As a consequence, these ids should be v...There is intensive use of `$ref` to schema fragments, which are incoming as schema-IDs like e.g. `"$ref": "osdu:wks:AbstractSpatialLocation:1.0.0"`. These fragments use semantic versioning as well.
As a consequence, these ids should be validated also during schema-validation:
- $ref target versions for patch increments can only refer to higher patch versions - if they refer to a higher minor (or even major version) validation must fail.
<br>**Example:**
| Sl No| Type | Initial Version | New Version | Status |
| ------ | ------ | ------ | ------ |------ |
| 1| Base Schema | osdu:wks:AbstractSpatialLocation:1.0.0 | osdu:wks:AbstractSpatialLocation:1.0.1 | |
| 2| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.2 | valid |
| 3| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.2.1 | invalid |
| 4| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.0.1 | invalid |
| 5| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.0 | invalid |
| 6| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:3.1.1 | invalid |
| 7| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources-Changed:2.1.1 | invalid |
- Similarly $ref target versions for higher minor versions can only refer to higher minor versions - higher major version references must be rejected by the validation.
<br>**Example:**
| Sl No| Type | Initial Version | New Version | Status |
| ------ | ------ | ------ | ------ |------ |
| 1| Base Schema | osdu:wks:AbstractSpatialLocation:1.0.0 | osdu:wks:AbstractSpatialLocation:1.1.0 | |
| 2| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.2 | valid |
| 3| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.2.1 | valid |
| 4| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.0.1 | invalid |
| 5| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.0 | invalid |
| 6| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:3.1.1 | invalid |
| 7| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources-Changed:2.1.1 | invalid |M9 - Release 0.12Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/59Correct Internal Versioning2022-02-11T21:58:57ZDavid Diederichd.diederich@opengroup.orgCorrect Internal VersioningDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/55Upgrade Core Common Dependency2022-02-11T21:59:15ZDavid Diederichd.diederich@opengroup.orgUpgrade Core Common DependencyDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/45Schema is getting created even if invalid def ref is passed in payload2022-11-24T12:50:10ZTsvetelina IvanovaSchema is getting created even if invalid def ref is passed in payloadWhen there is an internal `$ref` value in schema service, for example, “$ref”: “#/definitions/example”, but the “example” is missing in the `definitions` section we should throw an exception, because this should be invalid input. Current...When there is an internal `$ref` value in schema service, for example, “$ref”: “#/definitions/example”, but the “example” is missing in the `definitions` section we should throw an exception, because this should be invalid input. Currently, we allow schema to be created even if the definition for an internal reference is missing.
State after the fix - this should not be allowed, because 'person' is missing in the `definitions` section:
```
{
"$schema": "http://json-schema.org/draft-07/schema#",
"x-os-lifecycle-state": "published",
"description": "The entity well.",
"title": "Well",
"type": "object",
"definitions": { },
"properties": {
"person": {
"description": "Information for person",
"title": "Information for person",
"$ref": "#/definitions/person"
},
"locationOriginalCRS": {
"description": "The well's original location as AnyCrsFeatureCollection - a structure similar to but distinct from GeoJSON.",
"title": "Original CRS Location",
"$ref": "os:wks:anyCrsFeatureCollection.1.0"
},
"locationWGS84": {
"description": "The well's location as GeoJSON FeatureCollection.",
"title": "WGS84 Location",
"$ref": "https://geojson.org/schema/FeatureCollection.json",
"example": {
"features": [
{
"geometry": {
"coordinates": [
-92.11569999999999,
29.8823,
153.4779442519685
],
"type": "Point"
},
"type": "Feature",
"properties": {
"name": "Newton 2-31"
}
}
],
"type": "FeatureCollection"
}
}
}
}
```