OS Core Common - Spring 5 merge requestshttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests2023-08-18T13:09:21Zhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/15Add patch method to response header2023-08-18T13:09:21ZHong YanAdd patch method to response headerAdd Patch method to response header's Access-Control-Allow-Methods for Bulk Update API. It's a minor change to the common response header. All services use this header might need to update their unit tests and integration tests accordinglyAdd Patch method to response header's Access-Control-Allow-Methods for Bulk Update API. It's a minor change to the common response header. All services use this header might need to update their unit tests and integration tests accordinglyM1 - Release 0.1ethiraj krishnamanaiduAlok Joshiethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/14GCP merge ingestion, delivery changes2023-08-18T13:09:22ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comGCP merge ingestion, delivery changesChanges:
- latest changes related to Ingestion services from Google GitHub
- fix de-serialization issue with Delivery - Search communication.
- added a new field 'schema_version' in headers to make reference to schemas configurable.
- d...Changes:
- latest changes related to Ingestion services from Google GitHub
- fix de-serialization issue with Delivery - Search communication.
- added a new field 'schema_version' in headers to make reference to schemas configurable.
- deleted commented method with secret.
- changed pom file for repository and distributionManagement and community maven settings changed according to resolutionM1 - Release 0.1ethiraj krishnamanaiduDmitriy Rudkoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/11Added new methods in JaxRsDpsLogger and ILogger interface to support logger name2023-08-18T13:09:24ZKishore BattulaAdded new methods in JaxRsDpsLogger and ILogger interface to support logger nameADR reference https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/1ADR reference https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/1M1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)JoeBrandt BealDaniel SchollAlan Brazethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/10Enforce owner access in Storage APIs2023-08-18T13:09:25ZJohnny Guojguo26@slb.comEnforce owner access in Storage APIsThis PR is to add a method to enforce owner access for Storage APIs(create/update/purge), we add boolean `hasOwnerAccess(DpsHeaders headers, String[] ownerList)` method in `IEntitlementsAndCacheService` interface, this method will be ove...This PR is to add a method to enforce owner access for Storage APIs(create/update/purge), we add boolean `hasOwnerAccess(DpsHeaders headers, String[] ownerList)` method in `IEntitlementsAndCacheService` interface, this method will be overwritten in Storage-Core module.
Following are the original task's requirements.
- I can see only record owners can update records
- I can see only record owners can delete records
- I can see only record owners can patch records
- I can see record viewer cannot do any of the above operations
- I can see record viewer can read the record
Following are more interpretations:
1. Why are we doing this change? what is business/technical reason?
- To enforce that: If a user wants to create/update/purge a record, the user must be in the ACL groups of that record.
2. Which core services are impacted/implement this method?
- We only need to change the Storage service.
3. How does this impact cloud providers? if they have to implement what should they do?
- This only change storage-core module, files including `RecordServiceImpl.java` and `IngestionServiceImpl.java`, cloud providers do not need to change providers' implementations.
- After the change, if a user wants to create/update/purge a record, the user must be in the ACL groups of that record, so there might be users losing their permission to create/update/purge certain records.M1 - Release 0.1JoeNitin-slbNeelesh ThakurDaniel SchollAlan BrazAlok Joshiashley kelhamJoehttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/9Support old version of date representation2023-08-18T13:09:27ZRostislav Vatolinvatolinrp@gmail.comSupport old version of date representationAt the moment the expiration date is kept as "long" in the database,
to work with the existing legaltags, it is needed to parse the date in the old format.
Issue: #2At the moment the expiration date is kept as "long" in the database,
to work with the existing legaltags, it is needed to parse the date in the old format.
Issue: #2M1 - Release 0.1ethiraj krishnamanaiduNeelesh Thakurethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/8Add storage bulk update API request models2023-08-18T13:09:28ZHong YanAdd storage bulk update API request models# Add new apis, models and validators for storage bulk update API request
From SLB GA storage to os-core-common (models and validators and interface methods)
- Merged PR 156139: placeholder for the bulk update API
- Merged PR 162923: add...# Add new apis, models and validators for storage bulk update API request
From SLB GA storage to os-core-common (models and validators and interface methods)
- Merged PR 156139: placeholder for the bulk update API
- Merged PR 162923: add optimistic lock for bulk update API
- Merged PR 172654: Change Bulk Update API to use JSONPatch request body
This API allows the update of records metadata in batch.
## 1. Models and validators
Add new storage request body: `RecordBulkUpdateParam` and its fields
An example of this request body
```json
{
"query": {
"ids": [
"common:welldb:123456:version",
"common:welldb:345678:version",
"common:welldb:567890:version"
]
},
"ops": [
{
"op": "replace",
"path": "/acl/viewers",
"value": [
"data.default.viewers@<DataPartition>.<Domain>.com",
"test1.viewers@<DataPartition>.<Domain>.com"
]
},
{
"op": "replace",
"path": "/legal/legaltags",
"value": [
"DataPartition-legaltag-1",
"DataPartition-legaltag-2"
]
},
{
"op": "replace",
"path": "/acl/owners",
"value": [
"data.default.owners@<DataPartition>.<Domain>.com",
"test1.viewers@<DataPartition>.<Domain>.com"
]
}
]
}
```
In this case, the `RecordBulkUpdateParam` contains a `RecordQuery` field `query` and a `List<PatchOperation>` field `ops`. With this response body we can then implement a storage bulk update api for data in Storage service.
## 2. New interface methods
In package org.opengroup.osdu.core.common.storage.IPersistenceService add method
```java
/**
* This interface method do the actual update of the metadata
* of records in the Record Repository and Cloud Storage and
* revert the metadata back in case there is an error.
*
* @param recordMetadata The list of new record metadata
* @param recordsId The list of ids of the records that need to be updated
* @param recordsIdMap A map of each record id to their version
* @return A list of locked records (idWithVersion) in case they are found locked during the updating.
*/
List<String> updateMetadata(List<RecordMetadata> recordMetadata, List<String> recordsId, Map<String, String> recordsIdMap);
```M1 - Release 0.1ethiraj krishnamanaiduJohnny Guojguo26@slb.comNitin-slbethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/7merge notification service and legal service updates from opendes2022-09-15T11:12:59ZNeelesh Thakurmerge notification service and legal service updates from opendesFrom Opendes:
Merged PR 1663: extract subscription(containing model) to os-core-common
1) joint os-notification's subscription factory lib with core-common's util
2) create a Notification module in os-core-common .
3) change default roo...From Opendes:
Merged PR 1663: extract subscription(containing model) to os-core-common
1) joint os-notification's subscription factory lib with core-common's util
2) create a Notification module in os-core-common .
3) change default root URL point to opendes env
Merged PR 1682: Set default expirationDate if empty provided
Set default expirationDate if empty provided. When legalTag is created expiration date may be passed or not, need to handle a case, when it is "".M1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)JoeBrandt BealNitin-slbDaniel SchollDmitriy RudkoAlan Brazashley kelhamethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/4OpenDES GA code merge2023-08-18T13:09:30Zethiraj krishnamanaiduOpenDES GA code mergeOn Friday, Jun 12th we reviewed the OpenDES GA code merge with cloud providers and agreed to merge on or before Jun 19th. This MR is to support the GA changes in multiple core services.
More Details: https://community.opengroup.org/os...On Friday, Jun 12th we reviewed the OpenDES GA code merge with cloud providers and agreed to merge on or before Jun 19th. This MR is to support the GA changes in multiple core services.
More Details: https://community.opengroup.org/osdu/platform/system/home/-/wikis/OpenDES-GA-code-mergeM1 - Release 0.1Brandt BealDaniel SchollDmitriy RudkoAlan BrazBrandt Bealhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/2Remove Pipeline Aggregate Include2023-08-18T13:09:32ZDavid Diederichd.diederich@opengroup.orgRemove Pipeline Aggregate IncludeThe library.gitlab-ci.yml include is a deprecated style, which I'd like to start dropping support for. This replaces it with the newer style.
Also, the FOSSA NOTICE file was slightly different due to some backend FOSSA settings that I m...The library.gitlab-ci.yml include is a deprecated style, which I'd like to start dropping support for. This replaces it with the newer style.
Also, the FOSSA NOTICE file was slightly different due to some backend FOSSA settings that I made. I updated the NOTICE here to get a passing pipeline.M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/52Fix multi record info2023-08-18T13:08:38ZMatt WiseFix multi record infoNo existing core service uses MultiRecordInfo that I have seen. Therefore, should be a non-breaking fix
The IStorageService implementation in core was returning Strings rather than Records for the query records route. Other Record quer...No existing core service uses MultiRecordInfo that I have seen. Therefore, should be a non-breaking fix
The IStorageService implementation in core was returning Strings rather than Records for the query records route. Other Record query interfaces use the Record object.
The implementation was also throwing two different Exceptions that are resolved by this MR.
It appears the function was broken during the change to use the new BodyMapper, but went unnoticed until now.M3 - Release 0.5ethiraj krishnamanaiduJoeethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/51Cors2023-08-18T13:08:40ZSpencer Suttonsuttonsp@amazon.comCorsM3 - Release 0.5Spencer Suttonsuttonsp@amazon.comSpencer Suttonsuttonsp@amazon.comhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/50add view operation for policy service2023-08-18T13:08:42ZAlok Joshiadd view operation for policy serviceAdd "view" as an OperationType; required for Policy service integrationAdd "view" as an OperationType; required for Policy service integrationM3 - Release 0.5Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/49R3 record validation2023-08-18T13:08:43ZMatt WiseR3 record validationM3 - Release 0.5ethiraj krishnamanaiduDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy RudkoMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/59fix record api deserialization crash2023-08-18T13:08:28ZMatt Wisefix record api deserialization crashM4 - Release 0.7ethiraj krishnamanaiduDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy RudkoMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/58Regression in record id validation2023-08-18T13:08:30ZSanjeev-SLBRegression in record id validationDetails :https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/32Details :https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/32M4 - Release 0.7Sanjeev-SLBSanjeev-SLBhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/57add policy service model object and client lib for evaluating policy2023-08-18T13:08:31ZNeelesh Thakuradd policy service model object and client lib for evaluating policyM4 - Release 0.7Neelesh ThakurNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/56Support HTTP PATCH method2023-08-18T13:08:33ZSherman YangSupport HTTP PATCH methodThe HTTPUrlConnection class does not support the HTTP PATCH method. This MR enhances OSDU HttpClient to support HTTP PATCH method.
Update the UpdateGroupOperation to conform with entitlements v2 expectation.The HTTPUrlConnection class does not support the HTTP PATCH method. This MR enhances OSDU HttpClient to support HTTP PATCH method.
Update the UpdateGroupOperation to conform with entitlements v2 expectation.M4 - Release 0.7Dania Kodeih (Microsoft)Matt WiseDania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/55add support for tags in query2023-08-18T13:08:35ZYauheni Lesnikauadd support for tags in queryAdd tags search supportAdd tags search supportM4 - Release 0.7ethiraj krishnamanaiduNitin-slbNeelesh ThakurSherman YangAlok JoshiYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/53Port Entitlement v2 client changes to OSDU2023-08-18T13:08:37ZSherman YangPort Entitlement v2 client changes to OSDUPort Entitlement v2 clients to OSDU.
Fix a bug in AbstractHttpClient.Port Entitlement v2 clients to OSDU.
Fix a bug in AbstractHttpClient.M4 - Release 0.7https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/48update RecordBulkUpdate entities for tags support;2023-08-18T13:08:45ZYauheni Lesnikauupdate RecordBulkUpdate entities for tags support;Decided to add tag api support through the extension of patch api with ne path - "/tags". It should support 3 types of operations:
- add
- replace
- removeDecided to add tag api support through the extension of patch api with ne path - "/tags". It should support 3 types of operations:
- add
- replace
- removeM4 - Release 0.7ethiraj krishnamanaiduNitin-slbNeelesh ThakurAlok JoshiDuvelis CaraoYauheni Lesnikauethiraj krishnamanaidu