Schema merge requestshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests2022-04-05T10:38:44Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/264Upgrade First Party Library Dependencies for Release 0.142022-04-05T10:38:44ZDavid Diederichd.diederich@opengroup.orgUpgrade First Party Library Dependencies for Release 0.14This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any...This automated MR upgrades the first party libraries (other OSDU libraries) to utilize the latest release.
The intent is to keep the OSDU projects utilizing the latest available code to ensure widespread usage and stability.
However, any library that is older than the previous release will be left as-is, since the upgrade is likely to be more complicated.
Furthermore, the upgrade should only be merged in the CI pipeline reports success.
If this MR has failed, we can spend a little time investigating to see if a trivial upgrade could achieve compatiblity to the new library.
But significant upgrade efforts should not occur on this MR, as part of the release tagging process.
Instead, significant work should be scheduled for a subsequent milestone.
### Dependency Information Before the Upgrade
```
Branch: master
SHA: 36b05fad3414ee4a382a0a0000d04b42a1bf8947
Maven: 0.14.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ----------------------------------------------------- | --------------- | -------- |
| core-lib-azure | 0.14.0-rc2 | 0.6.1 |
| core-lib-gcp | 0.14.0-rc2 | |
| os-core-lib-aws | 0.13.0 | 0.3.16 |
| obm | 0.13.1-SNAPSHOT | |
| oqm | 0.13.0-SNAPSHOT | |
| os-core-common | 0.13.0 | 0.13.0 |
| os-core-lib-ibm | 0.13.0 | 0.7.0 |
| osm | 0.13.0-SNAPSHOT | |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.3 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |
### Dependency Information After the Upgrade
```
Branch: dependency-upgrade
SHA: 57f28fec301545aca644bed21b3a001862501233
Maven: 0.14.0-SNAPSHOT
```
| Maven Dependencies | _Root_ | testing/ |
| ----------------------------------------------------- | ------ | -------- |
| core-lib-azure | 0.14.0 | 0.6.1 |
| core-lib-gcp | 0.14.0 | |
| os-core-lib-aws | 0.14.0 | 0.3.16 |
| obm | 0.14.0 | |
| oqm | 0.14.0 | |
| os-core-common | 0.14.0 | 0.14.0 |
| os-core-lib-ibm | 0.14.0 | 0.7.0 |
| osm | 0.14.0 | |
| (3rd Party) net.minidev.json-smart | 2.4.7 | 2.3 |
| (3rd Party) org.apache.logging.log4j.log4j-api | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-core | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-jul | 2.17.1 | 2.13.3 |
| (3rd Party) org.apache.logging.log4j.log4j-slf4j-impl | 2.17.1 | 2.13.3 |M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/267Merge branch 'aws-update-to-system-schema' into 'trusted-users/amaverma/chang...2022-03-31T05:38:29ZAbhishek ChowdhryMerge branch 'aws-update-to-system-schema' into 'trusted-users/amaverma/changesInSchemaCore'Aws update to system schema
See merge request osdu/platform/system/schema-service!258
(cherry picked from commit 5e0784907ce82e2b4f137d8cf7d90e77be18ddb8)
46a1498a GONRG-4378: update rules
682c0d66 Merge branch 'GONRG-4378-update-rule...Aws update to system schema
See merge request osdu/platform/system/schema-service!258
(cherry picked from commit 5e0784907ce82e2b4f137d8cf7d90e77be18ddb8)
46a1498a GONRG-4378: update rules
682c0d66 Merge branch 'GONRG-4378-update-rules' into 'master'
ce395bec sensitive property look-up from env variables(GONRG-4404)
a5b19c1c Merge branch 'gcp-property-lookup' into 'master'
bfa67629 GONRG-4518
a9ac78f1 Merge branch 'gcp/GONRG-4518' into 'master'
a1fadc15 Merge branch 'master' into users/amaverma/changesInSchemaCore
fea99d14 update gcp docs
a3545959 fixing int tests to use Service Principal user
00728fa8 overriding method for iMessageBus AwsImpl
93a9464b cleaning up comments
2f3b067d moving core lib version to top of POM
19d62686 Merge branch 'users/amaverma/changesInSchemaCore' into schemaCore
89f0e132 cleaning up README auth requirements
8ff9742a adding SP auth to bootstrapping
0d31efc1 adding region to boto client and adding pip requirements
5ad94658 printing resource prefix to see if it's available from env variables
28a823c0 fixing bootstrapping empty bearer token
400be5b5 Update NOTICEhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/167Removing env cred2022-03-16T17:09:13ZAbhishek PatilRemoving env credAbhishek PatilAbhishek Patilhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/241Azure Deployment issue fix - Changing version of os-core-lib-azure2022-02-25T08:12:38ZAbhishek ChowdhryAzure Deployment issue fix - Changing version of os-core-lib-azureAbhishek ChowdhryAbhishek Chowdhryhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/163Usage of Publisher Facade2022-02-15T13:21:44ZNikhil Singh[MicroSoft]Usage of Publisher Facade## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [NA] I have ...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [NA] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [NA] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
The MR contains changes to use publisher facade for publishing to message brokers.
The schema service is a producer service for the notification service i.e it publishes notifications to Event grid that were getting pushed to notification service. Now the notification service has been migrated to Service bus and all the producer services are provided by a facade which they can use and start publishing to service bus.
Link to facade MR:
https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/143
Link of related issue:
https://community.opengroup.org/osdu/platform/system/notification/-/issues/25
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
NA
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
NoNikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/234Merge branch 'setting-subscription-az-login' into 'Azure/OSDU-Schema-M8-Master'2022-02-12T05:53:57ZAbhishek PatilMerge branch 'setting-subscription-az-login' into 'Azure/OSDU-Schema-M8-Master'set subscription when using az login
See merge request osdu/platform/system/schema-service!199
(cherry picked from commit 740d52ca73ac10b97291d9e1c1c6b2798cc6ec30)
35734a54 Update bootstrap.shset subscription when using az login
See merge request osdu/platform/system/schema-service!199
(cherry picked from commit 740d52ca73ac10b97291d9e1c1c6b2798cc6ec30)
35734a54 Update bootstrap.shAbhishek PatilAbhishek Patilhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/232CoreLib Version Change2022-02-18T10:44:05ZAbhishek ChowdhryCoreLib Version Changehttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/151[Draft] Test new schemas2022-02-03T15:10:29ZAbhishek Kumar (SLB)[Draft] Test new schemashttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/216GONRG-4214-bootstrap: fix for gcp schema service bootstrap, errors with pip3 ...2022-01-20T10:40:16ZAliaksandr Lubouski (EPAM)GONRG-4214-bootstrap: fix for gcp schema service bootstrap, errors with pip3 dependenciesGONRG-4214-bootstrap: fix for gcp schema service bootstrap, errors with pip3 dependenciesGONRG-4214-bootstrap: fix for gcp schema service bootstrap, errors with pip3 dependencieshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/215GONRG-4214: fix for gcp schema service bootstrap, errors with pip3 dependencies2022-01-20T10:33:34ZAliaksandr Lubouski (EPAM)GONRG-4214: fix for gcp schema service bootstrap, errors with pip3 dependenciesGONRG-4214: fix for gcp schema service bootstrap, errors with pip3 dependenciesGONRG-4214: fix for gcp schema service bootstrap, errors with pip3 dependencieshttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/200Updating core-lib azure and core-common versions for log4j vulnerabilities2022-01-13T04:46:59ZMayank Saggar [Microsoft]Updating core-lib azure and core-common versions for log4j vulnerabilitiesUpdating the core-lib and Core-common versions in provide-azure pom files which have the Log4J Versions without vulnerability,Updating the core-lib and Core-common versions in provide-azure pom files which have the Log4J Versions without vulnerability,Mayank Saggar [Microsoft]Mayank Saggar [Microsoft]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/124Resolve "Correct Internal Versioning" and "Upgrade Core Azure Dependency"2022-01-11T13:23:51ZDavid Diederichd.diederich@opengroup.orgResolve "Correct Internal Versioning" and "Upgrade Core Azure Dependency"Closes #58, #59
See osdu/platform&9 for more details.
## MR's Raison d'être
This MR simultaneously solves the internal versioning of the build, and upgrades the core Azure lib to use the latest release candidate version of the librar...Closes #58, #59
See osdu/platform&9 for more details.
## MR's Raison d'être
This MR simultaneously solves the internal versioning of the build, and upgrades the core Azure lib to use the latest release candidate version of the library. Correcting the internal versioning is an essential first step to all library cleanups -- without it, the testing libraries are permanently locked to the decisions of a very old version of the core libraries. Azure had to be updated at the same time, because it referenced a non-existent version of the library. That would prevent any pipeline from finishing successfully.
*Dependency Table, after the application of this MR's changes*
| Maven Dependencies | _Root POM_ | testing/ |
| ------------------------ | ---------------------- | ----------- |
| os-core-common | 0.9.0, 0.10.0-SNAPSHOT | 0.9.0 |
| core.aws.os-core-lib-aws | 0.9.0 | 0.3.16 |
| core-lib-gcp | 0.9.0 | |
| os-core-lib-ibm | 0.7.0 | 0.7.0 |
| core-lib-azure | 0.10.0-rc10 | 0.10.0-rc10 |
## Housekeeping Updates
### FOSSA Update
This MR also updates the FOSSA attributions, since these need to be kept up to date on every MR. Changes are expected whenever the library dependencies are updated.
### Update Deprecated Pipeline Includes
This MR also updates the `.gitlab-ci.yml` file to use the newer pipeline includes, rather than relying on deprecated includes that mimic old behavior. These housekeeping updates should be done regularly on MRs when they are discovered so the old CI include files can be deleted and no longer supported.
### Testing POM Structure
This MR marks the child modules under `testing/` as child modules. That allows proper detection of the root POMs for this project.M7 - Release 0.10David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/123Draft: Resolve "Upgrade Core Common Dependency"2022-01-11T13:23:50ZDavid Diederichd.diederich@opengroup.orgDraft: Resolve "Upgrade Core Common Dependency"Closes #55
See osdu/platform&9 for more details.
## MR's Raison d'être
This MR updates the core common to use the latest release candidate version of the library. This utilizes newly developed features, and will be upgraded to a full...Closes #55
See osdu/platform&9 for more details.
## MR's Raison d'être
This MR updates the core common to use the latest release candidate version of the library. This utilizes newly developed features, and will be upgraded to a full release during the new release cycle.
*Dependency Table, after the application of this MR's changes*
| Maven Dependencies | _Root POM_ | testing/ |
| ------------------------ | ----------- | ----------- |
| os-core-common | 0.10.0-rc1 | 0.10.0-rc1 |
| core.aws.os-core-lib-aws | 0.9.0 | 0.3.16 |
| core-lib-gcp | 0.9.0 | |
| os-core-lib-ibm | 0.7.0 | 0.7.0 |
| core-lib-azure | 0.10.0-rc10 | 0.10.0-rc10 |
## Housekeeping Updates
### FOSSA Update
This MR also updates the FOSSA attributions, since these need to be kept up to date on every MR. Changes are expected whenever the library dependencies are updated.M7 - Release 0.10David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/201remove offset concept from IBM service2021-12-17T13:28:52ZAshwani Pandeyremove offset concept from IBM serviceremove offset concept which is not require in IBM serviceremove offset concept which is not require in IBM serviceAshwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/73Azure: Fix shared partition value fetch from pipeline2021-11-19T12:00:36ZOrsu AkhilAzure: Fix shared partition value fetch from pipelineAt present in azure implementation , shared partition name is hard-coded . This change enables to fetch the value from environment variable that can be set via pipeline [here](https://community.opengroup.org/osdu/platform/system/schema-s...At present in azure implementation , shared partition name is hard-coded . This change enables to fetch the value from environment variable that can be set via pipeline [here](https://community.opengroup.org/osdu/platform/system/schema-service/-/blob/master/devops/azure/chart/templates/deployment.yaml#L105)https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/82Containerized the Schema Load Process so that it could be run without a pipeline2021-11-19T12:00:35ZDaniel SchollContainerized the Schema Load Process so that it could be run without a pipelineThis Merge Request adds in Containerization Support for Azure Schema Loads. Necessary for non pipeline built environments.This Merge Request adds in Containerization Support for Azure Schema Loads. Necessary for non pipeline built environments.https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/92Adding multi tenancy tests2021-11-19T12:00:30ZAshay BansodAdding multi tenancy testsAdding multi-tenancy test cases to schema service integration tests.
These are not to run in gitlab using tag feature as only single tenant infra is available.
When ever any consumer would have infra to execute this, it will have to run...Adding multi-tenancy test cases to schema service integration tests.
These are not to run in gitlab using tag feature as only single tenant infra is available.
When ever any consumer would have infra to execute this, it will have to run specific tag 'NotToRunInGitlab' along with 'SchemaService' tag.
Removed pre and post integration file (runner, step-def along with pom changes) as they are not configured for schema service anyways. Setup part of schema service is being already handled using background feature of BDD. So, no impact of current integration setup.https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/188Cherry pick 493aa638 (M9 bootstrap schema)2022-08-23T13:29:51ZAbhishek Kumar (SLB)Cherry pick 493aa638 (M9 bootstrap schema)These schemas are supposed to be bootstrapped with M9These schemas are supposed to be bootstrapped with M9M9 - Release 0.12Thomas Gehrmann [slb]Paresh BehedeThomas Gehrmann [slb]https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/100Container Benchmarking2021-09-30T02:59:14ZAbhishek PatilContainer Benchmarking## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it? YES
* [YES] Does the MR contain pipeline/ helm chart related changes? NO
* [NA] ...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it? YES
* [YES] Does the MR contain pipeline/ helm chart related changes? NO
* [NA] I have updated the documentation accordingly. NA
* [NA] I have added tests to cover my changes. NA
* [YES] All new and existing tests passed. YES
* [YES] My code follows the code style of this project. YES
* [NA] I ran lint checks locally prior to submission. NA
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
- Schema service's pod does not have container limits defined which causes it to use all the available cpu and memory.
- [Tomcat's MBean registry is disabled by default](https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.2-Release-Notes#tomcats-mbean-registry) in the spring boot's version used for schema service.
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
- Set the limits for cpu and memory.
- Enabled the Tomcat's MBean registry in order to make tomcat metrics available for monitoring.
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
NA
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
NONE
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->Abhishek PatilAbhishek Patilhttps://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/138fix grpc class not found exception2021-09-03T10:39:50ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comfix grpc class not found exception# Description:
Fix grpc class not found exception.
# How to test:
Does functionality was tested and how?
# Changes include:
- [x] Bugfix (a non-breaking change that solves an issue).
# Changes in:
- [x] GCP# Description:
Fix grpc class not found exception.
# How to test:
Does functionality was tested and how?
# Changes include:
- [x] Bugfix (a non-breaking change that solves an issue).
# Changes in:
- [x] GCPRostislav Dublin (EPAM)Rostislav Dublin (EPAM)