Notification merge requestshttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests2023-08-18T15:27:45Zhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/9Update version of os-core-common and fix broken integration tests2023-08-18T15:27:45ZHong YanUpdate version of os-core-common and fix broken integration tests## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
[[System/Storage] Bulk Update for ACLs on Records](https://community.opengroup.org/osdu/platform/system/storage/...## Type of change
- [x] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
[[System/Storage] Bulk Update for ACLs on Records](https://community.opengroup.org/osdu/platform/system/storage/-/issues/10)
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
The os-core-common lib is outdated
## What is the new/expected behavior?
No behavior changes.
## Have you added/updated Unit Tests and Integration Tests?
os-core-common lib is updated to the newest version. Integration test is updated to include the newly added Access-Control-Allow-Methods "PATCH" in os-core-common.
## Any other useful informationM1 - Release 0.1ethiraj krishnamanaiduNitin-slbNeelesh ThakurAlok Joshiethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/8add swagger2023-08-18T15:27:47ZNeelesh Thakuradd swaggerM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/7fix typo on documentation2023-08-18T15:27:49ZNeelesh Thakurfix typo on documentationM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/6add openapi spec2023-08-18T15:27:51ZNeelesh Thakuradd openapi specM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/5add tutorial2023-08-18T15:27:52ZNeelesh Thakuradd tutorialM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/4remove register service endpoint from gcp deployment on integration tests2023-08-18T15:27:54ZNeelesh Thakurremove register service endpoint from gcp deployment on integration testsaddressing [Issue](https://community.opengroup.org/osdu/platform/system/notification/-/issues/7#note_9655)addressing [Issue](https://community.opengroup.org/osdu/platform/system/notification/-/issues/7#note_9655)M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/3Notification Service Azure Provider Changes2023-08-18T15:27:56ZKomal MakkarNotification Service Azure Provider Changes# Description
The changes for Azure provider implementation for Notification service.
- Functionally, the AAD JWT token generation is facilitated by the changes.
- Caching of jwt token, tenant etc are added.
- All DIs are done for the...# Description
The changes for Azure provider implementation for Notification service.
- Functionally, the AAD JWT token generation is facilitated by the changes.
- Caching of jwt token, tenant etc are added.
- All DIs are done for the interfaces in notification-core.
# Closes issue(s)
- Partially closes (issue)[https://community.opengroup.org/osdu/platform/system/notification/-/issues/3]
The changes that are planned for the same issue/feature, in different MRs.
- The PubSub Authentication will be a different MR and is marked as TODO as of now
- Role-based auth, basic authZ, and AuthZ (securityConfig) is to be added.
# How to test / repro
- Deploy the services locally run integration Tests
# Changes include
- [ ] Bugfix (non-breaking change that solves an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
# Checklist
- [x] Added Unit Tests, wherever applicable.
- [x] Updated the Readme
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments
- Data partitioning in azure cosmos DB: currently data from all the tenants are being stored in a single cosmos DB instance, partitioned by the dataPartitionId of that tenant. This model might change in the future, and the changes will follow in the Notification Service.
- Integration test config requires refactoring in the notification-core. The IT for notification-azure will consumable post that.
*FYI*
@dkodeih @danielschollM1 - Release 0.1Hema Vishnu Pola [Microsoft]Kishore BattulaHema Vishnu Pola [Microsoft]https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/2remove gcp dependency from core module2023-08-18T15:27:57ZNeelesh Thakurremove gcp dependency from core moduleM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/1Remove the image override for maven tasks2023-08-18T15:27:59ZDavid Diederichd.diederich@opengroup.orgRemove the image override for maven tasksThe code was modified to use JDK 8, so the pipeline no longer needs this
override (and, in fact, shouldn't have it at all).The code was modified to use JDK 8, so the pipeline no longer needs this
override (and, in fact, shouldn't have it at all).M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/57Authz addition for Event Grid2023-08-18T15:26:42ZKomal MakkarAuthz addition for Event Grid## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
1. #10 Authorization for pubsub is done differently. Entitlements shouldn't be called for the PubSub requests.
AuthZ for Ops/ITs is done via Entitlements. This change introduces the AuthZ for PubSub.
2. #19 This issue is partly solved. The call to tenant factory is short-circuited to make sure the circular dependency doesn't block the core update.
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
The AuthZ for Event Grid production flow is enabled.
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
#21
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->M3 - Release 0.5Hema Vishnu Pola [Microsoft]Hema Vishnu Pola [Microsoft]https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/56[Core] [Azure] Adding handshake filter2023-08-18T15:26:43ZKomal Makkar[Core] [Azure] Adding handshake filter## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES] I have added tests...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [YES] I have updated the documentation accordingly.
* [YES] I have added tests to cover my changes.
* [YES] All new and existing tests passed.
* [YES] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
#16
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
The changes are done for the following.
1. **Reaction to Storage service publishing to event grid**. The format in which the event is generated changed in the storage service. The same is accommodated here.
SAMPLE EVENT
`[
{
"id": "32197ce0-3ee7-46e3-92a2-d2788d2b94d2",
"subject": "RecordsChanged",
"data": {
"data": [
{
"id": "opendes:doc:6a6f917dbab6479dad548d71515131c5",
"kind": "opendes:at:wellbore:1.0.0",
"op": "create"
},
{
"id": "opendes:doc:6d6246ace0964b419cec6f32c05c21ae",
"kind": "opendes:at:wellbore:1.0.0",
"op": "create"
}
],
"account-id": "opendes",
"correlation-id": "c3718fb5-d60e-45bd-a2d3-48d3c2bdd7e7",
"data-partition-id": "opendes"
},
"eventType": "RecordsChanged",
"dataVersion": "1.0",
"metadataVersion": "1",
"eventTime": "2020-12-28T12:09:42.331Z",
"topic": "fhtfc"
}
]`
2. **Handshake request #16 is addressed.** Filter of order 0 is written.
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- No
DISCLAIMER: We are using a filter for handshake requests. This filter is blindly accepting ALL the calls for handshake requests. There is a **security vulnerability** as we are not validating anything. This is might make it easy for anyone to make a DOS attack.
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
- Block the MR if you feel there is less testing or no details in the MR
- Please cover the following aspects in the MR
-- Coding design: _\<Reviewer1>_
-- Backward Compatibility: _\<Reviewer2>_
-- Feature Logic: _\<Logic design\>_
-- _\<Any other context mention here>_
OR
-- _\<Component 1>_: _\<Reviewer1>_
-- _\<CosmosDB>_: _\<Reviewer2>_
-- _\<ServiceBus>_ _\<Reviewer3>_
-- _\<Mention any other component and owner>_
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->M3 - Release 0.5Nitin-slbNeelesh ThakurHema Vishnu Pola [Microsoft]harshit aggarwalNitin-slbhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/65CORS Fix Update2023-08-18T15:26:32ZSpencer Suttonsuttonsp@amazon.comCORS Fix UpdateM4 - Release 0.7ethiraj krishnamanaiduDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy RudkoMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/61Logging Enhancements for GCP modules (GONRG-1735, GONRG-1780)2023-08-18T15:26:35ZArtem Dobrynin (EPAM)Logging Enhancements for GCP modules (GONRG-1735, GONRG-1780)## Type of change
- [ ] Bug Fix
- [X] Feature
Some enhancements for GCP logging system. Supplies the logs with user id, data-partition-id and correlation id
https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/pre-shipping/home/...## Type of change
- [ ] Bug Fix
- [X] Feature
Some enhancements for GCP logging system. Supplies the logs with user id, data-partition-id and correlation id
https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/pre-shipping/home/-/issues/50 https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/pre-shipping/home/-/issues/49
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
The logs are printed in an unparseable inline format.
## What is the new/expected behavior?
The logs are printed in JSON format and compatible with GCP Logger.
## Have you added/updated Unit Tests and Integration Tests?
No, since it is not required for these changes
## Any other useful information
There are two types of logging: for local Spring profile (`spring.profiles.active=local`) and for any other, incl. default. This was done for better readability while testing locally.
The MR should be merged after https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/merge_requests/10M4 - Release 0.7Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/60CORS Fix, AWS Updates2023-08-18T15:26:37ZSpencer Suttonsuttonsp@amazon.comCORS Fix, AWS UpdatesM4 - Release 0.7ethiraj krishnamanaiduDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy Rudkoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/58adds support azure multi-tenancy2023-08-18T15:26:40ZAliaksei Darafeyeuadds support azure multi-tenancy## Type of change
- [X] Feature
https://community.opengroup.org/osdu/platform/system/notification/-/issues/22
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementat...## Type of change
- [X] Feature
https://community.opengroup.org/osdu/platform/system/notification/-/issues/22
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [X] Azure
## Does this introduce a breaking change?
- [NO]
## Any other useful information
some refactoring in core due to the transition to the latest version os-core-commonM4 - Release 0.7Neelesh ThakurAlok JoshiDuvelis CaraoNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/72Define OSDU_GCP_LOG_LEVEL for the service [GONRG-2132]2023-08-18T15:26:23ZOleksandr Kosse (EPAM)Define OSDU_GCP_LOG_LEVEL for the service [GONRG-2132]- Format gitlab-ci.yml
- Define LOG_LEVEL for the service [GONRG-2132]- Format gitlab-ci.yml
- Define LOG_LEVEL for the service [GONRG-2132]M5 - Release 0.8https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/70Log level gcp (GONRG-2101)2023-08-18T15:26:25ZRiabokon Stanislav(EPAM)[GCP]Log level gcp (GONRG-2101)## Type of change
- [ ] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider...## Type of change
- [ ] Bug Fix
- [X] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [YES/NO]
## What is the current behavior?
Notification Service logged debug events.
## What is the new/expected behavior?
Notification Service can log debug/info events depending on current env.
## Have you added/updated Unit Tests and Integration Tests?
## Any other useful informationM5 - Release 0.8Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/68Updating documentation for Azure2023-08-18T15:26:26ZKomal MakkarUpdating documentation for Azure## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have ...## All Submissions:
-------------------------------------
* [YES] I have added an explanation of what changes in this merge do and why we should include it?
* [NO] Does the MR contain pipeline/ helm chart related changes?
* [YES] I have updated the documentation accordingly.
* [NA] I have added tests to cover my changes.
* [NA] All new and existing tests passed.
* [NA] My code follows the code style of this project.
* [YES] I ran lint checks locally prior to submission.
## What is the issue or story related to the change?
-------------------------------------
<!-- Please describe the current behavior that you are modifying, 'or' link to a relevant issue.
Feel free to add references to any design documents you might have shared with the team or any
related MR that you are building on top of. -->
High level design:
Issue: <!-- Link any __GitLab__ workitem(s) to this pull request. -->
<!-- Please add implementation details of current set of changes and how the code changes are
doing what they are expected to do. Are there any complex loops or designated code blocks that
should be elaborated? Is there some contextual knowledge that the reviewer should be aware of? -->
Change details:
Updated the documentation.
## Test coverage:
------------------
<!-- Mention unit test coverage of changes. -->
## Does this introduce a breaking change?
-------------------------------------
- [NO]
<!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. -->
## Pending items
----------------
<!-- Are there changes that you'll introduce in upcoming MRs and hence did not add in this one? Next steps of your
feature can also be mentioned here. -->
## Reviewer request
-------------------
- **Documentations follow the moto "the more the merrier" in terms of contributors. Please feel free to contribute.**
- Please provide an ETA when you plan to review this MR. Write a comment to decline or provide an ETA.
## Other information
-------------------------------------
<!-- Any other information that is important to this MR such as screenshots of how the component looks before and after the change. -->M5 - Release 0.8Nitin-slbHema Vishnu Pola [Microsoft]ashley kelhamNitin-slbhttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/67Switching the dependencies to release versions2023-08-18T15:26:28ZDavid Diederichd.diederich@opengroup.orgSwitching the dependencies to release versionsThis changes the library dependencies to use released versions of the core libraries.
They were previously depending on SNAPSHOT versions, which are less stable versions.
More importantly, the SNAPSHOT versions are periodically purged fr...This changes the library dependencies to use released versions of the core libraries.
They were previously depending on SNAPSHOT versions, which are less stable versions.
More importantly, the SNAPSHOT versions are periodically purged from the system to save disk space -- this happened recently.
Since these libraries no longer exist on community, building becomes difficult.
This MR moves those dependencies to a release version, which is better going forward and allows FOSSA to do the build and get good dependency information.
I assert that there are no substantial changes between the SNAPSHOT version I moved from and the latest release version that I moved to.
It's difficult to know which commit the SNAPSHOT dependency linked to, since it moved many times, but here are the differences from the last time the SNAPSHOT dependency was listed and the one commit that has the release version (0.7.0).
All of these changes were from me, updating versions and references as part of the release process.
- [GCP Differences from v0.6.1-SNAPSHOT to 0.7.0](https://community.opengroup.org/osdu/platform/system/lib/cloud/gcp/os-core-lib-gcp/-/compare/ff52818d929b7a32e491b75743285026c4c0a9b4...v0.7.0)
- [IBM Differences from v0.3.8-SNAPSHOT to 0.7.0](https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/compare/42d057eeee8f0299e7f90f1158c24f7fc0187dbc...v0.7.0)
For the second IBM change, there were code changes between the latest version of v0.3.6-SNAPSHOT and 0.7.0, but they appear to be additions only.
I don't think this will impact compatibility.
- [IBM Differences from v0.3.6-SNAPSHOT to 0.7.0](https://community.opengroup.org/osdu/platform/system/lib/cloud/ibm/os-core-lib-ibm/-/compare/030d7bdb47271193eebfe85515fc83542daa9a55...v0.7.0)
Separately, since I was working with FOSSA, I updated the configuration file and the corresponding NOTICE changes resulting from the new module.M5 - Release 0.8David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/66entitlements v2 cutover2023-08-18T15:26:30ZMingyang Zhuentitlements v2 cutoverM5 - Release 0.8Mingyang ZhuMingyang Zhu