Schema merge requestshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests2023-05-12T19:14:36Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/490UserAgent Changes2023-05-12T19:14:36ZAbhay JoshiUserAgent Changes(cherry picked from commit d4ebc88569f88fa73a5ce563b8025d0f952d839c)(cherry picked from commit d4ebc88569f88fa73a5ce563b8025d0f952d839c)M18 - Release 0.21Okoun-Ola Fabien HouetoAbhay JoshiYong ZengOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/489Schema Resiliency Enhacements2023-05-18T11:44:08ZChahat KumarSchema Resiliency Enhacementshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/488GONRG-7029: update anthos to baremetal2023-05-19T08:46:16ZYauheni Rykhter (EPAM)GONRG-7029: update anthos to baremetalM18 - Release 0.21Yauheni Rykhter (EPAM)Yauheni Rykhter (EPAM)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/487base image changed2023-05-12T14:33:49ZOrest Khud [EPAM / GCP]base image changedbase image changedbase image changedM18 - Release 0.21https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/486M18 service and CICD updates2023-05-09T18:39:31ZMadalyn MarabellaM18 service and CICD updatesAWS code changes only.
Changes in `testing/schema-test-core` only update `AwsServicePrincipalUtil.java` and `os-core-lib-aws` version in `pom.xml`AWS code changes only.
Changes in `testing/schema-test-core` only update `AwsServicePrincipalUtil.java` and `os-core-lib-aws` version in `pom.xml`M18 - Release 0.21Madalyn MarabellaMadalyn Marabellahttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/484add and upgrade snakeyaml dependency2023-05-09T16:10:29ZAbhay Joshiadd and upgrade snakeyaml dependency(cherry picked from commit 5baad7263a76de68caf8f869f663da244d8ff3a5)(cherry picked from commit 5baad7263a76de68caf8f869f663da244d8ff3a5)M18 - Release 0.21Okoun-Ola Fabien HouetoAbhay JoshiYong ZengOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/483Refactor system schema header check2023-05-17T05:51:48Zharshit aggarwalRefactor system schema header checkMoving the check for data partition header in system apis out of Auth Filter as it is not the right place, in current scheme of things if data partition header is passed a 400 exception is thrown in auth filter which is finally returned ...Moving the check for data partition header in system apis out of Auth Filter as it is not the right place, in current scheme of things if data partition header is passed a 400 exception is thrown in auth filter which is finally returned as 5xx to the client which is not the correct status
Clients should receive a 4xx code only
Testing
Changes were tested by containerizing the code and deploying against ADME instance
[Service IT link](https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_build/results?buildId=88417&view=results)harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/482wrap origin exception into app exception (GONRG-7053)2023-05-04T06:25:48ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comwrap origin exception into app exception (GONRG-7053)# Description:
Original exception wrapped into AppException, which makes it descoverable in logs.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that s...# Description:
Original exception wrapped into AppException, which makes it descoverable in logs.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] Google Cloud
- [ ] Azure
- [ ] AWS
- [ ] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [ ] Existing Tests pass
- [x] Verified functionality locally
- [ ] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Any comments to approvers hereM18 - Release 0.21Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/481Updated mappers.2023-06-05T13:34:19ZRiabokon Stanislav(EPAM)[GCP]Updated mappers.Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/480Revert "Merge branch 'GONGR-6885-reduce_requests_according_to_VPA' into 'mast...2023-08-18T21:55:05ZOleksandr Kosse (EPAM)Revert "Merge branch 'GONGR-6885-reduce_requests_according_to_VPA' into 'master'"This reverts merge request !478This reverts merge request !478M18 - Release 0.21https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/479Set FailOnError to false2023-04-28T19:41:11ZYash DholakiaSet FailOnError to falseSet FailOnError to false so build does not fail when dependency-check-maven fails.Set FailOnError to false so build does not fail when dependency-check-maven fails.Yash DholakiaYash Dholakiahttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/478GONGR-6885 Reduce requests according to VPA2023-08-18T21:55:07ZAndriy Halka [EPAM / GCP]GONGR-6885 Reduce requests according to VPAM18 - Release 0.21https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/477Merge azure/cherrypick-m14-m16 to azure/m16-master2023-04-24T17:04:46ZOsborn ChenMerge azure/cherrypick-m14-m16 to azure/m16-master(cherry picked from commit b726d1d2d85230dd495ad1a858ee06b6ad5c414c)(cherry picked from commit b726d1d2d85230dd495ad1a858ee06b6ad5c414c)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/476Update NOTICE, remove snapshot2023-04-13T20:28:10ZChad LeongUpdate NOTICE, remove snapshotUpdate FOSSA notice, remove SNAPSHOT dependenciesUpdate FOSSA notice, remove SNAPSHOT dependenciesM17 - Release 0.20https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/475Cherry-pick 'OpenAPI 3.0 Documentation using springdoc' into release/0.202023-04-11T09:37:33ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'OpenAPI 3.0 Documentation using springdoc' into release/0.20**Original MR**: !470
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !470
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/schema-service/-/pipelines/new?ref=cherry-pick-for-470)M17 - Release 0.20David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/474Full Upgrade of First Party Library Dependencies for Release 0.202023-05-22T15:50:40ZDavid Diederichd.diederich@opengroup.orgFull Upgrade of First Party Library Dependencies for Release 0.20This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will ...This generated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to try to fully upgrade all dependent libraries to see if the latest code will work.
It is expected that these will often fail, since the upgrades were previously rejected for failing pipelines and have not been directly addressed yet.
This upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
This MR may co-exist with a separate, smaller upgrade MR.
If both pass, this one should be used instead.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: c6bc65a3773a3d009e7e9e2d7e3ac72cc663ad49
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ----------------------------------------------------- | ---------------- | ---------- |
| core-lib-azure | 0.14.0-rc2 | 0.6.1 |
| core-lib-gcp | 0.19.0-rc3 | |
| os-core-lib-aws | 0.21.0-rc1 | 0.13.0 |
| obm | 0.18.0 | |
| oqm | 0.18.0 | |
| os-core-common | 0.19.0-rc6 | 0.19.0-rc6 |
| os-core-lib-ibm | 0.16.0-rc1 | 0.15.2 |
| osm | 0.18.0 | |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.3 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
| (3rd Party) org.springframework.spring-webflux | 5.3.12 | |
| (3rd Party) org.yaml.snakeyaml | 1.30, 1.33, 1.26 | 1.30 |
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
├─ _Root_
│ ├─ org.springdoc.springdoc-openapi-ui == 1.6.14
│ │ └─ org.springdoc.springdoc-openapi-webmvc-core == 1.6.14
│ │ └─ org.springdoc.springdoc-openapi-common == 1.6.14
│ │ └─ org.springframework.boot.spring-boot-autoconfigure == 2.7.7
│ │ └─ io.swagger.core.v3.swagger-core == 2.2.7
│ │ └─ org.yaml.snakeyaml == 1.30
│ ├─ org.opengroup.osdu.os-schema-core == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.7.7
│ │ └─ org.springframework.boot.spring-boot-starter == 2.7.7
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.os-schema-aws == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-test == 2.7.7
│ │ └─ org.springframework.boot.spring-boot-starter == 2.7.7
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.os-schema-gc == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-test == 2.7.7
│ │ └─ org.springframework.boot.spring-boot-starter == 2.7.7
│ │ └─ org.yaml.snakeyaml == 1.33
│ ├─ org.opengroup.osdu.os-schema-ibm == 0.21.0-SNAPSHOT
│ │ └─ org.springframework.boot.spring-boot-starter-web == 2.7.7
│ │ └─ org.springframework.boot.spring-boot-starter == 2.7.7
│ │ └─ org.yaml.snakeyaml == 1.33
│ └─ org.opengroup.osdu.os-schema-azure == 0.21.0-SNAPSHOT
│ └─ org.springframework.boot.spring-boot-starter-test == 2.4.12
│ └─ org.springframework.boot.spring-boot-starter == 2.4.12
│ └─ org.yaml.snakeyaml == 1.26
└─ testing/
└─ org.opengroup.osdu.schema-test-core == 0.21.0-SNAPSHOT
└─ org.opengroup.osdu.os-schema-core == 0.21.0-SNAPSHOT
└─ org.springframework.boot.spring-boot-starter-web == 2.7.7
└─ org.springframework.boot.spring-boot-starter == 2.7.7
└─ org.yaml.snakeyaml == 1.30
```
```
Critical: Found Vulnerable Spring WebFlux dependency (<5.2.20 || >=5.3.0 <5.3.18)
└─ _Root_
└─ org.opengroup.osdu.os-schema-azure == 0.21.0-SNAPSHOT
└─ org.springframework.boot.spring-boot-starter-webflux == 2.6.6
└─ org.springframework.spring-webflux == 5.3.12
```
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade-3
SHA: 79045808c04ca813e2d4e8374cb8514c5358821f
Maven: 0.21.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ------------------------------ | ---------- | ---------- |
| core-lib-azure | 0.20.0 | 0.20.0 |
| core-lib-gc | 0.20.0 | |
| os-core-lib-aws | 0.21.0-rc2 | 0.21.0-rc2 |
| obm | 0.20.0 | |
| oqm | 0.20.0 | |
| os-core-common | 0.20.1 | 0.20.1 |
| os-core-lib-ibm | 0.20.0 | 0.20.0 |
| osm | 0.20.0 | |
| (3rd Party) org.yaml.snakeyaml | 1.30, 2.0 | 2.0 |
```
Critical: Found Vulnerable Snake YAML dependency (<2.0)
└─ _Root_
└─ org.springdoc.springdoc-openapi-ui == 1.6.14
└─ org.springdoc.springdoc-openapi-webmvc-core == 1.6.14
└─ org.springdoc.springdoc-openapi-common == 1.6.14
└─ org.springframework.boot.spring-boot-autoconfigure == 2.7.7
└─ io.swagger.core.v3.swagger-core == 2.2.7
└─ org.yaml.snakeyaml == 1.30
```M18 - Release 0.21Srinivasan NarayananSrinivasan Narayananhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/473modify bootstrap to push image to msosdu registry2023-08-18T21:55:08Zsaketh somarajumodify bootstrap to push image to msosdu registry- Currently schema bootstrapping for demo and preship envs are being done by running container locally
- This MR supports azure-devops pipeline to automate schema bootstrapping for demo and preship envs
- Add script to azure_bootstrap jo...- Currently schema bootstrapping for demo and preship envs are being done by running container locally
- This MR supports azure-devops pipeline to automate schema bootstrapping for demo and preship envs
- Add script to azure_bootstrap job to push image to msosdu registry.M18 - Release 0.21saketh somarajusaketh somarajuhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/472AWS Integration merge2023-04-06T01:25:56ZManish JangidAWS Integration mergeAWS Integration mergeAWS Integration mergeM18 - Release 0.21https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/471Cherry-pick 'Fix Azure integration tests' into release/0.202023-04-07T05:24:12ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'Fix Azure integration tests' into release/0.20**Original MR**: !469
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporati...**Original MR**: !469
### This MR is a Cherry Pick into a Release Branch.
After the release branch is first created, any subsequent changes use this process to update the release (often resulting in a new patch tag) without incorporating all changes in the default branch.
These MRs must be approved by the PMC before they are merged, since they alter the scope of the release.
To see more details about the change itself, look at the Original MR listed above.
#### Skipped Pipeline
Normally, pipelines are not executed on the cherry pick branch/MR prior to merging.
This optimization is accepted because the code was tested when it merged into the default branch, and will be tested again in the release branch prior to tagging.
However, if anybody feels that the MR requires further scrutiny -- whether because it had conflicts in the cherry-picking, it interfaces with some drastically altered logic between the branches, or any other reason -- we can run the pipeline here prior to merging.
#### If There's Reason to Run a Pipeline
If you want to see a pipeline result before this merges, first add a comment explaining why you'd like to see the pipeline results so the PMC and others know your thinking.
Then, mark the MR as a Draft MR (using the vertical ellipsis above, choose 'Mark as Draft').
This prevents the MR from being approved & merged accidentally by a busy release coordinator who didn't see your comment.
Finally, if you are a maintainer on the project, launch a pipeline on this branch.
Since this branch is a protected branch and the MR has ~no-detached-pipeline set, all integration tests will run and there's no need for any `trusted-*` branches.
[Launch a Pipeline for this Branch](https://community.opengroup.org/osdu/platform/system/schema-service/-/pipelines/new?ref=cherry-pick-for-469)M17 - Release 0.20David Diederichd.diederich@opengroup.orgChad LeongSrinivasan NarayananDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/470OpenAPI 3.0 Documentation using springdoc2023-04-11T09:30:58ZThulasi Dass SubramanianOpenAPI 3.0 Documentation using springdoc**Link to ADR(Architecture Decision Record)** : [Swagger using springdoc-openapi](https://community.opengroup.org/osdu/platform/system/home/-/issues/97)
## OpenAPI 3.0 related changes
* upgraded to latest **springdoc openapi** latest v...**Link to ADR(Architecture Decision Record)** : [Swagger using springdoc-openapi](https://community.opengroup.org/osdu/platform/system/home/-/issues/97)
## OpenAPI 3.0 related changes
* upgraded to latest **springdoc openapi** latest version [1.6.14](https://mvnrepository.com/artifact/org.springdoc/springdoc-openapi-ui/1.6.14)
* used **[@OpenAPIDefinition](https://docs.swagger.io/swagger-core/v2.0.9/apidocs/io/swagger/v3/oas/annotations/OpenAPIDefinition.html)** annotation to populate OpenAPI Object fields *\[info, tags, servers, security\]*
* Documented the below API's with OpenAPI 3.0 **Annotations**
- Schema API
- System Schema API
- Info API
* Added the standard HTTP Response(4xx, 5x\*\*\*\*x) for API Responses
* Custom Path for
* **Swagger UI**: https://host/context-path/swagger (will redirect to https://host/context-path/swagger-ui/index.html)
* **api-docs (JSON)** : https://host/context-path/api-docs
* **api-docs (YAML)** : https://host/context-path/api-docs.yaml
* Azure Swagger GLAB(for Reference)
* **Swagger UI**: https://osdu-glab.msft-osdu-test.org/api/schema-service/v1/swagger (will redirect to https://osdu-glab.msft-osdu-test.org/api/schema-service/v1/swagger-ui/index.html)
* **api-docs (JSON)** : https://osdu-glab.msft-osdu-test.org/api/schema-service/v1/api-docs
* **api-docs (YAML)** :https://osdu-glab.msft-osdu-test.org/api/schema-service/v1/api-docs.yaml
## Other Changes
- **Configurable** descriptions managed in [swagger.properties](https://community.opengroup.org/osdu/platform/system/partition/-/blob/az/td-oas/partition-core/src/main/resources/swagger.properties)
- Deleted HomeController
- Marked the 'WhoamiController' as 'Hidden' for documentation
- Updated Readme for swagger related information
## References
- https://springdoc.org/faq.html#_can_i_use_spring_property_with_swagger_annotations
- https://springdoc.org/migrating-from-springfox.htmlM17 - Release 0.20Thulasi Dass SubramanianThulasi Dass Subramanian