Schema issueshttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues2022-05-11T07:58:44Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/70GCP Integration test is taking too much time2022-05-11T07:58:44ZAbhishek Kumar (SLB)GCP Integration test is taking too much timeGCP integration is taking too long to complete.
One thing, I prominently observed is that there are too many logging of huge messages which is possibly slowing down the entire process.
![image](/uploads/77ae3ba57eb81626cc37d73972a1b166/...GCP integration is taking too long to complete.
One thing, I prominently observed is that there are too many logging of huge messages which is possibly slowing down the entire process.
![image](/uploads/77ae3ba57eb81626cc37d73972a1b166/image.png)
https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/590392
Please find below time taken by other CSPs:
- azure_test = Duration: 14 minutes 20 seconds
- aws-test-java = Duration: 8 minutes 18 seconds
- ibm-test = Duration: 11 minutes 55 seconds
- osdu-gcp-test = Duration: **59 minutes 36 seconds**Oleksandr Kosse (EPAM)Riabokon Stanislav(EPAM)[GCP]Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/76Azure IT is taking approximately 1 hour to finish2022-05-11T07:58:44ZAbhishek Kumar (SLB)Azure IT is taking approximately 1 hour to finishhttps://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/683575https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/683575Neha SardaNeha Sardahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/83Log4J CVE-2021-442282022-04-12T09:59:19ZKhilesh SahuLog4J CVE-2021-44228Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters...Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. In previous releases (>2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or it can be mitigated in prior releases (<2.10) by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class).https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/90New attribute added at patch level should not be allowed2022-04-12T09:58:27ZAbhishek Kumar (SLB)New attribute added at patch level should not be allowedAt present adding a new attribute at patch level is allowed. This, however, should not be the case.
User should be forced to increment minor version if he intend to add a new attribute to the schema body.At present adding a new attribute at patch level is allowed. This, however, should not be the case.
User should be forced to increment minor version if he intend to add a new attribute to the schema body.Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/92Update to M11 schemas2022-03-28T12:35:16ZThomas Gehrmann [slb]Update to M11 schemasPublish the M11 version of the OSDU WKS schemas as delivered by OSDU Data Definitions.
- [x] Update shared OSDU schemas in this repository in `./deployments/shared-schemas/osdu`Publish the M11 version of the OSDU WKS schemas as delivered by OSDU Data Definitions.
- [x] Update shared OSDU schemas in this repository in `./deployments/shared-schemas/osdu`M11 - Release 0.14Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/61Support for version upgrade during schema-validation2022-02-25T14:13:08ZAbhishek Kumar (SLB)Support for version upgrade during schema-validationThere is intensive use of `$ref` to schema fragments, which are incoming as schema-IDs like e.g. `"$ref": "osdu:wks:AbstractSpatialLocation:1.0.0"`. These fragments use semantic versioning as well.
As a consequence, these ids should be v...There is intensive use of `$ref` to schema fragments, which are incoming as schema-IDs like e.g. `"$ref": "osdu:wks:AbstractSpatialLocation:1.0.0"`. These fragments use semantic versioning as well.
As a consequence, these ids should be validated also during schema-validation:
- $ref target versions for patch increments can only refer to higher patch versions - if they refer to a higher minor (or even major version) validation must fail.
<br>**Example:**
| Sl No| Type | Initial Version | New Version | Status |
| ------ | ------ | ------ | ------ |------ |
| 1| Base Schema | osdu:wks:AbstractSpatialLocation:1.0.0 | osdu:wks:AbstractSpatialLocation:1.0.1 | |
| 2| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.2 | valid |
| 3| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.2.1 | invalid |
| 4| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.0.1 | invalid |
| 5| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.0 | invalid |
| 6| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:3.1.1 | invalid |
| 7| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources-Changed:2.1.1 | invalid |
- Similarly $ref target versions for higher minor versions can only refer to higher minor versions - higher major version references must be rejected by the validation.
<br>**Example:**
| Sl No| Type | Initial Version | New Version | Status |
| ------ | ------ | ------ | ------ |------ |
| 1| Base Schema | osdu:wks:AbstractSpatialLocation:1.0.0 | osdu:wks:AbstractSpatialLocation:1.1.0 | |
| 2| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.2 | valid |
| 3| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.2.1 | valid |
| 4| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.0.1 | invalid |
| 5| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:2.1.0 | invalid |
| 6| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources:3.1.1 | invalid |
| 7| $ref | osdu:wks:AbstractCommonResources:2.1.1 | osdu:wks:AbstractCommonResources-Changed:2.1.1 | invalid |M9 - Release 0.12Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/2[Schema Service] Integration tests2022-02-14T04:23:54ZRucha Deshpande[Schema Service] Integration testsDid not find the integration tests as part of the Schema Service source code on Gitlab.
Is it housed in a separate project?Did not find the integration tests as part of the Schema Service source code on Gitlab.
Is it housed in a separate project?M1 - Release 0.1Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/88Indexing hints missing for AbstractBusinessRule2022-02-13T09:01:45ZMandar KulkarniIndexing hints missing for AbstractBusinessRuleAs per the Indexer hints [report](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/blob/master/E-R/IndexerHintsReport.md#abstract), AbstractBusinessRule schema doesn't have indexing hints.
![image](/upload...As per the Indexer hints [report](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/blob/master/E-R/IndexerHintsReport.md#abstract), AbstractBusinessRule schema doesn't have indexing hints.
![image](/uploads/2f272360c802c1a3b4b39a28cec728cf/image.png)
Because of this search queries like below don't work
```
{
"kind": "*:*:work-product-component--DataQuality:1.0.0",
"query": "data.BusinessRules.DataRuleSets.DataRuleSetID:\"rulesetID1""
}
```
We need to add indexing hints (type : nested) in the AbstractBusinessRule schema for `DataRuleSets` and `DataRules` properties to support queries like below:
"query":"nested(data.BusinessRules.DataRuleSets, (DataRuleSetID:\"rulesetID1" AND DataRuleSetRunStatus:true))"
"query":"nested(data.BusinessRules.DataRules, (DataRuleID:\"ruleID1" AND DataRuleRunStatus:true))"
Similarly, the QualityDataRuleSet doesn't have indexer hint for DataRules block
![image](/uploads/8bbd5369d79f25b323175e2ba1f275ba/image.png)
We need to add indexing hints (type : flattened) for querying the rulesets containing specific rule.https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/55Upgrade Core Common Dependency2022-02-11T21:59:15ZDavid Diederichd.diederich@opengroup.orgUpgrade Core Common DependencyDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/56Upgrade Core AWS Dependency2022-02-11T21:59:09ZDavid Diederichd.diederich@opengroup.orgUpgrade Core AWS Dependencyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/57Upgrade Core IBM Dependency2022-02-11T21:59:05ZDavid Diederichd.diederich@opengroup.orgUpgrade Core IBM Dependencyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/58Upgrade Core Azure Dependency2022-02-11T21:59:01ZDavid Diederichd.diederich@opengroup.orgUpgrade Core Azure DependencyDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/59Correct Internal Versioning2022-02-11T21:58:57ZDavid Diederichd.diederich@opengroup.orgCorrect Internal VersioningDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/86M10 Data Definitions Content2022-01-13T06:26:13ZThomas Gehrmann [slb]M10 Data Definitions ContentM10 - Release 0.13Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/48GET API fails if SupersededBy is present in SHARED partition2022-01-10T04:03:36ZAbhishek Kumar (SLB)GET API fails if SupersededBy is present in SHARED partitionGET API call fails if the call is made from the PRIVATE tenant and it includes schema from SHARED partition.
If there are schemas in a shared partition with SupersededBy reference, then it tries to fetch the schema from the private tena...GET API call fails if the call is made from the PRIVATE tenant and it includes schema from SHARED partition.
If there are schemas in a shared partition with SupersededBy reference, then it tries to fetch the schema from the private tenant.
GET call returns records from both SHARED and PRIVATE Partitions.Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/80Refresh DDSC schemas2022-01-05T11:34:34ZThomas Gehrmann [slb]Refresh DDSC schemasCreate a MR to update the shared OSDU standard schemas based on the DDSC master state November 23rd.
- [x] VirtualProperty definitions in schemas
- [x] Updated `x-osdu-indexing` tag values to improvde searchCreate a MR to update the shared OSDU standard schemas based on the DDSC master state November 23rd.
- [x] VirtualProperty definitions in schemas
- [x] Updated `x-osdu-indexing` tag values to improvde searchM10 - Release 0.13Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/85Upgrade to Log4J 2.172021-12-21T00:48:28ZDavid Diederichd.diederich@opengroup.orgUpgrade to Log4J 2.17The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/84Log4J Expedient Updates and Patches2021-12-16T00:29:57ZDavid Diederichd.diederich@opengroup.orgLog4J Expedient Updates and PatchesThis issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.This issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/64offset parameter is not working2021-12-14T12:32:46ZAbhishek Kumar (SLB)offset parameter is not workingThe Offset parameter is not working when searching the schemaInfo repository. If I specify the data-partition-id opendes, set the limit to 1 and do two queries, one with an offset of 0 and one with an offset of 99, I am getting the same ...The Offset parameter is not working when searching the schemaInfo repository. If I specify the data-partition-id opendes, set the limit to 1 and do two queries, one with an offset of 0 and one with an offset of 99, I am getting the same record.
To summarize, by changing the offset value there is no impact on the result list. Ideally, if there are multiple schemas in the list the behaviour should be:
S1,S2,S3,S4
Offset=0,Limit=1 ==> Should return S1
Offset=1,Limit=1 ==> Should return S2
Offset=2,Limit=1 ==> Should return S3M9 - Release 0.12Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/66Job Failed #5608922021-12-14T12:31:12ZAbhishek Kumar (SLB)Job Failed #560892Please help us with to investigate this error.
Job [#560892](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/560892) failed for 709d197fd10fce98c14576feb2789608fe451893:Please help us with to investigate this error.
Job [#560892](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/560892) failed for 709d197fd10fce98c14576feb2789608fe451893:Aman VermaAman Verma