Schema merge requestshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests2023-08-18T21:56:43Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/208Code coverage2023-08-18T21:56:43ZAalekh JainCode coverageTasks accomplished as part of this MR
1. Added the support for generating code coverage reports using `Jacoco`
2. Updated the settings for CI/CD pipelines (Under section General Pipelines) to capture the code coverage using the Test Cov...Tasks accomplished as part of this MR
1. Added the support for generating code coverage reports using `Jacoco`
2. Updated the settings for CI/CD pipelines (Under section General Pipelines) to capture the code coverage using the Test Coverage Parsing regex as: `Total.*?([0-9]{1,3})%`
3. Added code coverage badge for azure in the `README.md` corresponding to azure cloud provider.
Attached below are the screenshots of the code coverage captured through bade and in the pipeline stages -
1. ![image](/uploads/0c7917ca04e2b26de08e49c34df8ce5d/image.png)
2. ![image](/uploads/ccb5584e6be2ef6e43b8e3ad7210883a/image.png)M10 - Release 0.13https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/207core lib upgrade to rc62023-08-18T21:56:45Zshitij singhcore lib upgrade to rc6IBM core lib upgrade to rc6IBM core lib upgrade to rc6M10 - Release 0.13shitij singhshitij singhhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/204Upgrading library versions to incorporate the latest Log4j fix2023-08-18T21:56:46ZDavid Diederichd.diederich@opengroup.orgUpgrading library versions to incorporate the latest Log4j fixThese updates apply version 2.17, addressing CVE-2021-45105
Part of the #85 seriesThese updates apply version 2.17, addressing CVE-2021-45105
Part of the #85 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/203remove offset concept from IBM service2023-08-18T21:56:48ZAshwani Pandeyremove offset concept from IBM serviceoffset filter is not require in IBM service to get schema records based on offset field.offset filter is not require in IBM service to get schema records based on offset field.M10 - Release 0.13Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/202Disables unsafe subst for log4j2023-08-18T21:56:49ZOleksii Tsyganov (EPAM)Disables unsafe subst for log4jM10 - Release 0.13Oleksii Tsyganov (EPAM)Oleksii Tsyganov (EPAM)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/197Upgrading versions of the core libraries to fix log4j issues2023-08-18T21:56:51ZDavid Diederichd.diederich@opengroup.orgUpgrading versions of the core libraries to fix log4j issuesPart of the #84 seriesPart of the #84 seriesM10 - Release 0.13David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/196Ibm schema filter check2023-08-18T21:56:53ZAshwani PandeyIbm schema filter checkadded filter check for getting schema recordsadded filter check for getting schema recordsM10 - Release 0.13Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/195Vuln fix netty codec2023-08-18T21:56:54ZGokul NagareVuln fix netty codecM10 - Release 0.13Anuj GuptaShrikant GargAnuj Guptahttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/194M10 preview shared schemas2023-08-18T22:40:47ZThomas Gehrmann [slb]M10 preview shared schemasCloses #80
DDSC commit 1b05145a18bde08ef16900909cb026fe0b1e8a6e
# Summary of changes
The links refer to the OSDU member GitLab.
1. **VirtualProperties**<br>
Based on the community [search-service ADR #69](https://community.opengr...Closes #80
DDSC commit 1b05145a18bde08ef16900909cb026fe0b1e8a6e
# Summary of changes
The links refer to the OSDU member GitLab.
1. **VirtualProperties**<br>
Based on the community [search-service ADR #69](https://community.opengroup.org/osdu/platform/system/search-service/-/issues/69)
the schemas for storable records were extended to carry a JSON extension tag defining
virtual properties to resolve the following two challenges:
1. Ambiguous Location information<br>
A number of schemas offer more than one property with map coordinates. The `data.VirtualProperties.DefaultLocation`
list disambiguates the locations by providing a priority. This way a deterministic map location be achieved.
2. Default Name<br>
There is no standard name property for all schemas. While `Name` is present in most of the OSDU schemas there are
notable exceptions, e.g. Well and Wellbore with `FacilityName` instead. The `data.VirtualProperties.DefaultName`
definition refers to where the default name property is to be derived from.
2. **VirtualProperties Support**<br>
1. The script to create new proposal workbooks or proposal workbooks from existing schemas has been modified
to populate a new fixed sheet 'VirtualPropertiesSpecification'.
2. The script to parse schema proposals to JSON schema definitions has been updated to create the data block
under the `x-osdu-virtual-properties` tag.
3. An overview report about the VirtualProperties has been provided in
[VirtualPropertiesReport.md](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/VirtualPropertiesReport.md).
3. **Indexing Hint Changes**<br>
The following schema fragments were changed such that queries into arrays of objects frequently used return more
scoped responses. This causes a massive re-indexing of existing records since virtually all storable osdu kinds
include the following fragments - see more details in the following Migration Guides:
1. [`osdu:wks:AbstractFacility:1.0.0`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master//Guides/MigrationGuides/M10/AbstractFacility.1.0.0.md)
2. [`osdu:wks:AbstractMaster:1.0.0`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master//Guides/MigrationGuides/M10/AbstractMaster.1.0.0.md)
3. [`osdu:wks:AbstractProject:1.0.0`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master//Guides/MigrationGuides/M10/AbstractProject.1.0.0.md)
4. [`osdu:wks:AbstractReferenceType:1.0.0`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master//Guides/MigrationGuides/M10/AbstractReferenceType.1.0.0.md)
5. [`osdu:wks:AbstractWorkProductComponent:1.0.0`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master//Guides/MigrationGuides/M10/AbstractWorkProductComponent.1.0.0.md)
4. **LogCurveType** and **LogCurveFamily** augmented with explicit relationships; the associated OSDU reference value
manifests for LogCurveType and LogCurveFamily are updated with the new relationships.<br>
1. `LogCurveType:1.1.0` records now relate to other reference-data types, such as:
1. → [`PropertyType`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/reference-data/PropertyType.1.0.0.md), via
[AbstractPropertyType](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/abstract/AbstractPropertyType.1.0.0.md) `data.PropertyType.PropertyTypeID` and `data.PropertyType.Name`
2. → [`UnitQuantity`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/reference-data/UnitQuantity.1.0.0.md), via `data.UnitQuantityID`
3. → [`StandardsOrganisation`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/reference-data/StandardsOrganisation.1.0.0.md) via `data.AcquisitionCompanyID`
2. `LogCurveFamily:1.1.0`
1. → [`PropertyType`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/reference-data/PropertyType.1.0.0.md), via
[AbstractPropertyType](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/abstract/AbstractPropertyType.1.0.0.md) `data.PropertyType.PropertyTypeID` and `data.PropertyType.Name`
2. → [`UnitQuantity`](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/E-R/reference-data/UnitQuantity.1.0.0.md), via `data.UnitQuantityID`M10 - Release 0.13Thomas Gehrmann [slb]Thomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/193disabled spring web security2023-08-18T21:56:56ZBhushan Radedisabled spring web securityM10 - Release 0.13Anuj GuptaBhushan RadeAnuj Guptahttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/192Renaming schema's OpenAPI file2023-08-18T21:56:57ZDavid Diederichd.diederich@opengroup.orgRenaming schema's OpenAPI fileGitLab renders this based on the filenameGitLab renders this based on the filenameM10 - Release 0.13Chris ZhangChris Zhanghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/187Schema pointing to cicd ibm-java-git.yml2023-08-18T21:56:59ZWalter DSchema pointing to cicd ibm-java-git.ymlM10 - Release 0.13Walter DWalter Dhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/185Fyre bootstrap2023-08-18T21:57:01ZSuman GhoshFyre bootstrapWe have change the logic in Token.py based on the argument which user will pass during the post installation for the certificate.We have change the logic in Token.py based on the argument which user will pass during the post installation for the certificate.M10 - Release 0.13Anuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/184Stabilize Schema Service (GONRG-3575)2021-11-08T14:18:31ZDmitrii Novikov (EPAM)Stabilize Schema Service (GONRG-3575)## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] I...## Type of change
- [x] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [x] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
- Fixed filter
- Fixed date default value
- Fixed not found exceptionM10 - Release 0.13Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/177Gcp test fixes2023-08-18T21:57:03ZAliaksandr Ramanovich (EPAM)Gcp test fixesFix for osdu-gcp-test jobFix for osdu-gcp-test jobM10 - Release 0.13https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/142Schema ref validation changes2022-09-16T08:44:03ZAbhishek Kumar (SLB)Schema ref validation changesThis MR introduces validation rule for `$ref` fragments. Basically, if there are minor level changes in the schema then all the $ref components of that schema are allowed to have incremental changes in their $ref versioning (incremental ...This MR introduces validation rule for `$ref` fragments. Basically, if there are minor level changes in the schema then all the $ref components of that schema are allowed to have incremental changes in their $ref versioning (incremental change in minor and patch level versioning). In other words, the schemas pointed by $ref can also have all permissible changes that are allowed at the Minor level (for more details please refer here [https://community.opengroup.org/osdu/platform/system/schema-service/-/blob/master/docs/SchemaService-OSDU.md#schema-validation].
Likewise, at the Patch level, $ref values are permitted to have incremented patch version.
For more example, refere to the issue https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/61
## Code refactor
![image](/uploads/9d8017340e5455def4ff3a2af4e009e8/image.png)
![image](/uploads/6da1920a8e1eef46726fd90a60bfe645/image.png)
Along with the above state changes, there is code refactoring done to make components manageable and reusable. For example, earlier there were below classes:
- `schema-core/src/main/java/org/opengroup/osdu/schema/validation/SchemaMinorVersionValidator.java`
- `schema-core/src/main/java/org/opengroup/osdu/schema/validation/SchemaPatchVersionValidator.java`
which now, has evolved into different Handlers depending upon the kind of tags or operations they validate:
* AdditionalPropertiesHandler
* RemoveOperationHandler
* RequiredAttributeHandler
* TypeOperationHandler
* CompositionPropertiesHandler
* ReferenceAttributeHandler
## Testing
There are comprehensive set of Unit and IT test cases added/updated. To have a good test coverage, we have included several JSON files at below locations:
- https://community.opengroup.org/osdu/platform/system/schema-service/-/tree/schema-ref-validation/schema-core/src/test/resources/schema_compare [61 files]
- https://community.opengroup.org/osdu/platform/system/schema-service/-/tree/master/testing/schema-test-core/src/test/resources/input_payloads [88 Files]M10 - Release 0.13Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/262Update bootstrap.sh2022-03-28T10:58:23ZShrikant GargUpdate bootstrap.shFix for IBM bootstrap errorFix for IBM bootstrap errorM10 Patch - Release 0.13 patchShrikant GargShrikant Garghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/273Bugfix: Storage Exception is not being processed by Schema2022-04-12T16:04:13ZArtem Dobrynin (EPAM)Bugfix: Storage Exception is not being processed by Schema* Fixed case when Google Cloud Storage exception cannot be parsed correctly* Fixed case when Google Cloud Storage exception cannot be parsed correctlyM11 - Release 0.14Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/260IBM bootstrap fix-012022-03-28T08:31:04ZAshwani PandeyIBM bootstrap fix-01added delete command for common DB to fix IBM bootstrap pipelineadded delete command for common DB to fix IBM bootstrap pipelineM11 - Release 0.14Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/257Describe Blob Storage settings in separate READMEs for GCP and Anthos2022-03-22T09:53:30ZRostislav Dublin (EPAM)Describe Blob Storage settings in separate READMEs for GCP and Anthos## Type of change
- [ ] Bug Fix
- [Х] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://jiraeu.epam.com/browse/GONRG-4518
## Does this introduce a change in the core logic?
- [NO]
## Does ...## Type of change
- [ ] Bug Fix
- [Х] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
https://jiraeu.epam.com/browse/GONRG-4518
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [Х] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
No sufficient info in READMEs for GCP and Anthos
## What is the new/expected behavior?
Added needed info in GCP and Anthos READMEs
## Have you added/updated Unit Tests and Integration Tests?
NO
## Any other useful information
NOM11 - Release 0.14Andrei Dalhikh [EPAM/GC]Andrei Dalhikh [EPAM/GC]