OS Core Common - Spring 5 merge requestshttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests2023-08-18T13:09:33Zhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/1Updated README2023-08-18T13:09:33Zethiraj krishnamanaiduUpdated READMEUpdated README and testing the Git flow.Updated README and testing the Git flow.Release 2.0ethiraj krishnamanaiduethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/2Remove Pipeline Aggregate Include2023-08-18T13:09:32ZDavid Diederichd.diederich@opengroup.orgRemove Pipeline Aggregate IncludeThe library.gitlab-ci.yml include is a deprecated style, which I'd like to start dropping support for. This replaces it with the newer style.
Also, the FOSSA NOTICE file was slightly different due to some backend FOSSA settings that I m...The library.gitlab-ci.yml include is a deprecated style, which I'd like to start dropping support for. This replaces it with the newer style.
Also, the FOSSA NOTICE file was slightly different due to some backend FOSSA settings that I made. I updated the NOTICE here to get a passing pipeline.M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/3Update response headers2020-06-05T19:37:03ZRostislav Vatolinvatolinrp@gmail.comUpdate response headersRemove on-behalf-of & account-id headers from apis.Remove on-behalf-of & account-id headers from apis.Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/4OpenDES GA code merge2023-08-18T13:09:30Zethiraj krishnamanaiduOpenDES GA code mergeOn Friday, Jun 12th we reviewed the OpenDES GA code merge with cloud providers and agreed to merge on or before Jun 19th. This MR is to support the GA changes in multiple core services.
More Details: https://community.opengroup.org/os...On Friday, Jun 12th we reviewed the OpenDES GA code merge with cloud providers and agreed to merge on or before Jun 19th. This MR is to support the GA changes in multiple core services.
More Details: https://community.opengroup.org/osdu/platform/system/home/-/wikis/OpenDES-GA-code-mergeM1 - Release 0.1Brandt BealDaniel SchollDmitriy RudkoAlan BrazBrandt Bealhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/5Opendes ado changes2020-06-17T20:38:09ZRostislav Vatolinvatolinrp@gmail.comOpendes ado changes1) Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".
2) Joint os-notification's subscription factory lib with core-common's util
3) Create...1) Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".
2) Joint os-notification's subscription factory lib with core-common's util
3) Create a Notification model module in os-core-common's model module.
4) Change default root URL point to opendes envRostislav Vatolinvatolinrp@gmail.comRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/6Opendes ado changes2020-06-19T16:54:12ZRostislav Vatolinvatolinrp@gmail.comOpendes ado changes1. Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".
1. Joint os-notification's subscription factory lib with core-common's util
1. Create a...1. Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".
1. Joint os-notification's subscription factory lib with core-common's util
1. Create a Notification model module in os-core-common's model module.
1. Change default root URL point to opendes env.Rostislav Vatolinvatolinrp@gmail.comashley kelhamRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/7merge notification service and legal service updates from opendes2022-09-15T11:12:59ZNeelesh Thakurmerge notification service and legal service updates from opendesFrom Opendes:
Merged PR 1663: extract subscription(containing model) to os-core-common
1) joint os-notification's subscription factory lib with core-common's util
2) create a Notification module in os-core-common .
3) change default roo...From Opendes:
Merged PR 1663: extract subscription(containing model) to os-core-common
1) joint os-notification's subscription factory lib with core-common's util
2) create a Notification module in os-core-common .
3) change default root URL point to opendes env
Merged PR 1682: Set default expirationDate if empty provided
Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".M1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)JoeBrandt BealNitin-slbDaniel SchollDmitriy RudkoAlan Brazashley kelhamethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/8Add storage bulk update API request models2023-08-18T13:09:28ZHong YanAdd storage bulk update API request models# Add new apis, models and validators for storage bulk update API request
From SLB GA storage to os-core-common (models and validators and interface methods)
- Merged PR 156139: placeholder for the bulk update API
- Merged PR 162923: add...# Add new apis, models and validators for storage bulk update API request
From SLB GA storage to os-core-common (models and validators and interface methods)
- Merged PR 156139: placeholder for the bulk update API
- Merged PR 162923: add optimistic lock for bulk update API
- Merged PR 172654: Change Bulk Update API to use JSONPatch request body
This API allows the update of records metadata in batch.
## 1. Models and validators
Add new storage request body: `RecordBulkUpdateParam` and its fields
An example of this request body
```json
{
"query": {
"ids": [
"common:welldb:123456:version",
"common:welldb:345678:version",
"common:welldb:567890:version"
]
},
"ops": [
{
"op": "replace",
"path": "/acl/viewers",
"value": [
"data.default.viewers@<DataPartition>.<Domain>.com",
"test1.viewers@<DataPartition>.<Domain>.com"
]
},
{
"op": "replace",
"path": "/legal/legaltags",
"value": [
"DataPartition-legaltag-1",
"DataPartition-legaltag-2"
]
},
{
"op": "replace",
"path": "/acl/owners",
"value": [
"data.default.owners@<DataPartition>.<Domain>.com",
"test1.viewers@<DataPartition>.<Domain>.com"
]
}
]
}
```
In this case, the `RecordBulkUpdateParam` contains a `RecordQuery` field `query` and a `List<PatchOperation>` field `ops`. With this response body we can then implement a storage bulk update api for data in Storage service.
## 2. New interface methods
In package org.opengroup.osdu.core.common.storage.IPersistenceService add method
```java
/**
* This interface method do the actual update of the metadata
* of records in the Record Repository and Cloud Storage and
* revert the metadata back in case there is an error.
*
* @param recordMetadata The list of new record metadata
* @param recordsId The list of ids of the records that need to be updated
* @param recordsIdMap A map of each record id to their version
* @return A list of locked records (idWithVersion) in case they are found locked during the updating.
*/
List<String> updateMetadata(List<RecordMetadata> recordMetadata, List<String> recordsId, Map<String, String> recordsIdMap);
```M1 - Release 0.1ethiraj krishnamanaiduJohnny Guojguo26@slb.comNitin-slbethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/9Support old version of date representation2023-08-18T13:09:27ZRostislav Vatolinvatolinrp@gmail.comSupport old version of date representationAt the moment the expiration date is kept as "long" in the database,
to work with the existing legaltags, it is needed to parse the date in the old format.
Issue: #2At the moment the expiration date is kept as "long" in the database,
to work with the existing legaltags, it is needed to parse the date in the old format.
Issue: #2M1 - Release 0.1ethiraj krishnamanaiduNeelesh Thakurethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/10Enforce owner access in Storage APIs2023-08-18T13:09:25ZJohnny Guojguo26@slb.comEnforce owner access in Storage APIsThis PR is to add a method to enforce owner access for Storage APIs(create/update/purge), we add boolean `hasOwnerAccess(DpsHeaders headers, String[] ownerList)` method in `IEntitlementsAndCacheService` interface, this method will be ove...This PR is to add a method to enforce owner access for Storage APIs(create/update/purge), we add boolean `hasOwnerAccess(DpsHeaders headers, String[] ownerList)` method in `IEntitlementsAndCacheService` interface, this method will be overwritten in Storage-Core module.
Following are the original task's requirements.
- I can see only record owners can update records
- I can see only record owners can delete records
- I can see only record owners can patch records
- I can see record viewer cannot do any of the above operations
- I can see record viewer can read the record
Following are more interpretations:
1. Why are we doing this change? what is business/technical reason?
- To enforce that: If a user wants to create/update/purge a record, the user must be in the ACL groups of that record.
2. Which core services are impacted/implement this method?
- We only need to change the Storage service.
3. How does this impact cloud providers? if they have to implement what should they do?
- This only change storage-core module, files including `RecordServiceImpl.java` and `IngestionServiceImpl.java`, cloud providers do not need to change providers' implementations.
- After the change, if a user wants to create/update/purge a record, the user must be in the ACL groups of that record, so there might be users losing their permission to create/update/purge certain records.M1 - Release 0.1JoeNitin-slbNeelesh ThakurDaniel SchollAlan BrazAlok Joshiashley kelhamJoehttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/11Added new methods in JaxRsDpsLogger and ILogger interface to support logger name2023-08-18T13:09:24ZKishore BattulaAdded new methods in JaxRsDpsLogger and ILogger interface to support logger nameADR reference https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/1ADR reference https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/1M1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)JoeBrandt BealDaniel SchollAlan Brazethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/12Add global exception mappers2020-07-16T20:12:32ZSviatoslav NekhaienkoAdd global exception mappersMove common (each service has this logic now) errors handling to the os-core-common library.
Remove duplicated code in service when you start using this version of the common library.Move common (each service has this logic now) errors handling to the os-core-common library.
Remove duplicated code in service when you start using this version of the common library.Neelesh ThakurNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/13Fix/gcp merge changes (GONRG-443)2020-08-25T12:45:29ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comFix/gcp merge changes (GONRG-443)Fix deserialization issue with Delivery - Search
communication.
Added a new field 'schema_version' in headers
to make reference to schemas configurable.
Deleted commented method with secret.
Changed pom file for repository and distributi...Fix deserialization issue with Delivery - Search
communication.
Added a new field 'schema_version' in headers
to make reference to schemas configurable.
Deleted commented method with secret.
Changed pom file for repository and distributionManagement
and community maven settings changed according to resolutionDmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/14GCP merge ingestion, delivery changes2023-08-18T13:09:22ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comGCP merge ingestion, delivery changesChanges:
- latest changes related to Ingestion services from Google GitHub
- fix de-serialization issue with Delivery - Search communication.
- added a new field 'schema_version' in headers to make reference to schemas configurable.
- d...Changes:
- latest changes related to Ingestion services from Google GitHub
- fix de-serialization issue with Delivery - Search communication.
- added a new field 'schema_version' in headers to make reference to schemas configurable.
- deleted commented method with secret.
- changed pom file for repository and distributionManagement and community maven settings changed according to resolutionM1 - Release 0.1ethiraj krishnamanaiduDmitriy Rudkoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/15Add patch method to response header2023-08-18T13:09:21ZHong YanAdd patch method to response headerAdd Patch method to response header's Access-Control-Allow-Methods for Bulk Update API. It's a minor change to the common response header. All services use this header might need to update their unit tests and integration tests accordinglyAdd Patch method to response header's Access-Control-Allow-Methods for Bulk Update API. It's a minor change to the common response header. All services use this header might need to update their unit tests and integration tests accordinglyM1 - Release 0.1ethiraj krishnamanaiduAlok Joshiethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/16AWS Provider CI2020-08-04T20:37:33ZMatt WiseAWS Provider CIhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/17Switch elastic to 6.8.8 because of whitesource recommendation2020-08-26T03:08:55ZPavel BachylaSwitch elastic to 6.8.8 because of whitesource recommendationThe PR is intended to fix high and medium whitesource vulnerabilities.
Most of them are related to libraries version upgrades.
Also, there is a feature that prevents creation of bean with security vulnerability introduced [there](https:/...The PR is intended to fix high and medium whitesource vulnerabilities.
Most of them are related to libraries version upgrades.
Also, there is a feature that prevents creation of bean with security vulnerability introduced [there](https://github.com/spring-projects/spring-framework/commit/76964e16efbcee7c19608e826aa26c032493baa0)ethiraj krishnamanaiduethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/18Moving IDToken in Util as it goes beyond the scope of Search2021-02-18T21:12:26ZKomal MakkarMoving IDToken in Util as it goes beyond the scope of Search# Description
Moving the IdToken to util, as it will be used by services beyond search (Notification Service will be using it).
# Changes include
- [x] Refactoring
- [ ] Bugfix (non-breaking change that solves an issue)
- [ ] New featur...# Description
Moving the IdToken to util, as it will be used by services beyond search (Notification Service will be using it).
# Changes include
- [x] Refactoring
- [ ] Bugfix (non-breaking change that solves an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
# Checklist
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme
- [x] Existing Tests pass: The test shouldSuccessfullySerializeLegalTag is breaking in the existing code.
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments
*FYI*
@dkodeih @danielschollethiraj krishnamanaiduHema Vishnu Pola [Microsoft]Kishore Battulaethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/19WIP: Added a signedUrl for FileLocationResponse (GONRG-623)2020-10-04T22:14:32ZRiabokon Stanislav(EPAM)[GCP]WIP: Added a signedUrl for FileLocationResponse (GONRG-623)# Description:
`File Service -> getLocation` should return SignedURL to get access from DAG to file content by FileID
# How to test:
Tested on GCP Dev manually
# Changes include:
- [ ] Refactor (a non-breaking change that improves ...# Description:
`File Service -> getLocation` should return SignedURL to get access from DAG to file content by FileID
# How to test:
Tested on GCP Dev manually
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [x] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [x] Azure
- [x] AWS
- [x] IBM
# Dev Checklist:
- [ ] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.
# Other comments:
Any comments to approvers hereDmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/20Partition client implementation2023-08-18T13:09:19ZDuvelis CaraoPartition client implementationThis is the common client implementation for the new service Partition service.
Any client who needs to call Partition service should use this implementation.
Partition service is a new service to support multi-tenant in OSDU.
Partitio...This is the common client implementation for the new service Partition service.
Any client who needs to call Partition service should use this implementation.
Partition service is a new service to support multi-tenant in OSDU.
Partition service is in the process to be released to gitlab.M1 - Release 0.1ethiraj krishnamanaiduFerris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoJoeDaniel SchollDmitriy RudkoAlan BrazMatt Wiseethiraj krishnamanaidu