OS Core Common - Spring 5 merge requestshttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests2023-08-18T13:09:17Zhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/21upgrade packages to resolve security related issues in os-core-common2023-08-18T13:09:17ZSherman Yangupgrade packages to resolve security related issues in os-core-commonUpgrade packages to resolve security related issues:
Spring boot version from 2.1.7.RELEASE to 2.1.16.RELEASE
Elastic search version from 6.6.2 to 6.8.1
Jackson data bind version from 2.9.9.3 to 2.9.10.6
Incremented package version...Upgrade packages to resolve security related issues:
Spring boot version from 2.1.7.RELEASE to 2.1.16.RELEASE
Elastic search version from 6.6.2 to 6.8.1
Jackson data bind version from 2.9.9.3 to 2.9.10.6
Incremented package version to 0.3.8M1 - Release 0.1https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/22added azure blob storage (abs)2021-02-25T21:39:47ZChristian Leckneradded azure blob storage (abs)Added ABS for Azure Blob Storage to driver type.Added ABS for Azure Blob Storage to driver type.Christian LecknerChristian Lecknerhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/23Fix issue with persistable reference handling in converters;2023-08-18T13:09:16ZYauheni LesnikauFix issue with persistable reference handling in converters;The issue was found in the **Storage GA** and concerned feature of the **Gson** lib, namely `getAsString()` method of the `com.google.gson.JsonElement` class with the subclasses (**JsonPrimitive**, **JsonObject**, etc). The thing is that...The issue was found in the **Storage GA** and concerned feature of the **Gson** lib, namely `getAsString()` method of the `com.google.gson.JsonElement` class with the subclasses (**JsonPrimitive**, **JsonObject**, etc). The thing is that the method overriden safely only for **JsomPrimitive** subclass, and if the field mapped as **JsonObject** one (persistableReference in the 'meta' part, in our case) this call throws **UnsupportedOperationException**.
My enhancement prevents undesirable throwing the exception. Also tests were extendedM1 - Release 0.1ethiraj krishnamanaiduFerris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoJoeYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/24Updated NOTICE file with new dependency information2023-08-18T13:09:14ZDavid Diederichd.diederich@opengroup.orgUpdated NOTICE file with new dependency informationLooks like MRs !20 and !21 brought in some dependency changes without updating the NOTICE. This includes the new dependencies.
The SnakeYAML and Elasticsearch removals are coming from GPL / MIT sections. I believe FOSSA database had the...Looks like MRs !20 and !21 brought in some dependency changes without updating the NOTICE. This includes the new dependencies.
The SnakeYAML and Elasticsearch removals are coming from GPL / MIT sections. I believe FOSSA database had these older versions
miscategorized (common among multi-licensed libraries, since it is hard for FOSSA to know which license you would choose). Newer
versions seem to have different choices.M1 - Release 0.1ethiraj krishnamanaiduethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/25address contract update for partition API and make partition API exception co...2023-08-18T13:09:13ZNeelesh Thakuraddress contract update for partition API and make partition API exception consistent- [x] address contract update from [ADR](https://community.opengroup.org/osdu/platform/system/partition/-/issues/4)
- [x] make Partition exception consistent with rest of service exception. It should extend from `DpsException` so client ...- [x] address contract update from [ADR](https://community.opengroup.org/osdu/platform/system/partition/-/issues/4)
- [x] make Partition exception consistent with rest of service exception. It should extend from `DpsException` so client lib consumers can get the response status code.
- [x] fixed typos in PartitionAPIConfig
- [x] minor refactoring to remove unused variables in PartitionServiceM1 - Release 0.1Ferris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoJoeDmitriy RudkoFerris Argylehttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/26fix issue with frame of reference conversion invalid unitKey and some refacto...2023-08-18T13:09:11ZYauheni Lesnikaufix issue with frame of reference conversion invalid unitKey and some refactoring;This enhancement fixes the issue in the **Storage** with measurement unit conversion.
When "/api/storage/v2/records:batch" calls for some record the conversion performs, but the measurement unit used to be the same.
This fix eliminates t...This enhancement fixes the issue in the **Storage** with measurement unit conversion.
When "/api/storage/v2/records:batch" calls for some record the conversion performs, but the measurement unit used to be the same.
This fix eliminates the issue.M1 - Release 0.1ethiraj krishnamanaiduFerris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoJoeNitin-slbNeelesh ThakurKishore BattulaMatt WiseYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/27Add client factory and service lib for search and storage2023-08-18T13:09:09ZHong YanAdd client factory and service lib for search and storageAdd client library for search service and storage service. It will not break any of our current pipeline.
Basic usage example
```java
import org.opengroup.osdu.core.common.model.http.DpsHeaders;
import org.opengroup.osdu.core.common.m...Add client library for search service and storage service. It will not break any of our current pipeline.
Basic usage example
```java
import org.opengroup.osdu.core.common.model.http.DpsHeaders;
import org.opengroup.osdu.core.common.model.search.CursorQueryRequest;
import org.opengroup.osdu.core.common.model.storage.Schema;
import org.opengroup.osdu.core.common.search.ISearchService;
import org.opengroup.osdu.core.common.search.SearchAPIConfig;
import org.opengroup.osdu.core.common.search.SearchFactory;
import org.opengroup.osdu.core.common.storage.IStorageService;
import org.opengroup.osdu.core.common.storage.StorageAPIConfig;
import org.opengroup.osdu.core.common.storage.StorageFactory;
...
DpsHeaders headers = new DpsHeaders();
headers.put(DpsHeaders.DATA_PARTITION_ID, "partition");
headers.put(DpsHeaders.AUTHORIZATION, "token");
headers.addCorrelationIdIfMissing();
// Search Service
SearchFactory searchFactory = new SearchFactory(SearchAPIConfig.builder().rootUrl("search url").apiKey("search apikey").build());
ISearchService searchService = searchFactory.create(headers);
searchService.searchCursor(new CursorQueryRequest());
// Storage Service
StorageFactory storageFactory = new StorageFactory(StorageAPIConfig.builder().rootUrl("storage url").apiKey("storage apikey").build());
IStorageService storageService = storageFactory.create(headers);
Schema schema = storageService.getSchema("kind");
```M1 - Release 0.1ethiraj krishnamanaiduNitin-slbNeelesh ThakurSherman YangAlok Joshiashley kelhamDuvelis Caraoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/28add jackson core depedency that is required in the new version for jackson-da...2023-08-18T13:09:08ZMatt Wiseadd jackson core depedency that is required in the new version for jackson-databind to work correctly with depedenciesAWS SDK doesnt work with 0.3.8-0.3.13 of Core Common. This dependency fix resolves the dependency issue caused by the upgrade of the jackson databind lib.
This should resolve the issue where os-core-common's version couldnt be updated ...AWS SDK doesnt work with 0.3.8-0.3.13 of Core Common. This dependency fix resolves the dependency issue caused by the upgrade of the jackson databind lib.
This should resolve the issue where os-core-common's version couldnt be updated in the Indexer Service.
https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/6M1 - Release 0.1ethiraj krishnamanaiduJoeSrihari PrabaharanRucha DeshpandeDmitriy RudkoSpencer Suttonsuttonsp@amazon.comethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/29Refactor http body parsing2023-08-18T13:09:06ZPavel BachylaRefactor http body parsingResolves [the issue: [Bug] expirationDate failed to be parsed by Gson](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/11)
1. Move the parsing logic of the HttpResponse#body field to a separate clas...Resolves [the issue: [Bug] expirationDate failed to be parsed by Gson](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/11)
1. Move the parsing logic of the HttpResponse#body field to a separate class.
2. Replace gson->jackson
3. Encapsulate the internal implementation of the parsing so that clients will not be affected if any low-level changes happen in the parsing logic.
# Changes include
* [x] Refactoring
* [x] Bugfix (non-breaking change that solves an issue)
* [ ] New feature (non-breaking change that adds functionality)
* [x] Breaking change (change that is not backwards-compatible and/or changes current functionality)
# Checklist
* [x] Added Unit Tests, wherever applicable.
* [ ] Updated the Readme
* [x] Existing Tests pass: The test shouldSuccessfullySerializeLegalTag is breaking in the existing code.
* [x] Verified functionality locally
* [x] Self Reviewed my code for formatting and complex business logic.M1 - Release 0.1Neelesh ThakurSherman Yangashley kelhamSviatoslav NekhaienkoNeelesh Thakurhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/30New RedisCache constructor to support SSL and Password2023-08-18T13:09:04ZAlok JoshiNew RedisCache constructor to support SSL and PasswordCreate a separate RedisCache constructor which supports password and SSL enabled connection to Redis clusters. It will be consumed by services that wish to use Redis cache instead of instance based memcache to implement caching. Current ...Create a separate RedisCache constructor which supports password and SSL enabled connection to Redis clusters. It will be consumed by services that wish to use Redis cache instead of instance based memcache to implement caching. Current cluster settings are non-SSL which do not use password.
This change is in response to the bug filed https://community.opengroup.org/osdu/platform/system/storage/-/issues/34. The bug may be filed specifically for Azure bug this is a generic connection change
Will follow up with changes in services after this is mergedM1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)Neelesh Thakurethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/31Added new Partition API (List) to client library2023-08-18T13:09:02ZDuvelis CaraoAdded new Partition API (List) to client libraryPartition service has a new API to list all partitions Id
Client is updated in order to support the new API added to partition service.
This MR depends on the Partition Service MR that includes the new API:
https://community.opengroup.o...Partition service has a new API to list all partitions Id
Client is updated in order to support the new API added to partition service.
This MR depends on the Partition Service MR that includes the new API:
https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/15M1 - Release 0.1ethiraj krishnamanaiduNitin-slbNeelesh ThakurSherman YangAlok JoshiDuvelis Caraoethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/32accept database number as parameter to RedisCache2020-10-28T17:40:22ZAlok Joshiaccept database number as parameter to RedisCacheTo avoid cache conflicts by allowing Redis to connect to numbered databases instead of always connecting to default database '0'. Database number will be passed down from calling services.To avoid cache conflicts by allowing Redis to connect to numbered databases instead of always connecting to default database '0'. Database number will be passed down from calling services.ethiraj krishnamanaiduNeelesh Thakurashley kelhamethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/33Replace Gson by Jackson on legal tag creation in LegalService2023-08-18T13:09:00ZRostyslav Matushkin (SLB)Replace Gson by Jackson on legal tag creation in LegalServiceResolves [the issue: [Bug] 'expirationDate' field is serialized incorrectly due to the use of Gson](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/16)
1. Add `ObjectMapper` to `LegalService`.
2. Ch...Resolves [the issue: [Bug] 'expirationDate' field is serialized incorrectly due to the use of Gson](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/16)
1. Add `ObjectMapper` to `LegalService`.
2. Change `HttpRequest.post(<Object>)` by `HttpRequest.post().body(<String>)` in `LegalService` using `ObjectMapper`.
3. Add `LegalServiceTest` to check the field `expirationDate` serializes correctly.
# Changes include
* [ ] Refactoring
* [x] Bugfix (non-breaking change that solves an issue)
* [ ] New feature (non-breaking change that adds functionality)
* [ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
# Checklist
* [x] Added Unit Tests, wherever applicable.
* [ ] Updated the Readme
* [x] Existing Tests passed
* [x] Verified functionality locally
* [x] Self Reviewed my code for formatting and complex business logic.M1 - Release 0.1ethiraj krishnamanaiduRostyslav Matushkin (SLB)Mingyang Zhuethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/34Move Search Config out of os-core-common (GONRG-744)2023-08-18T13:08:58ZArtem Dobrynin (EPAM)Move Search Config out of os-core-common (GONRG-744)# Description:
ADR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/5
* Removed search-related config from os-core-common
* Removed unit tests for the config
# How to test:
Unit tests were modifie...# Description:
ADR: https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/5
* Removed search-related config from os-core-common
* Removed unit tests for the config
# How to test:
Unit tests were modified in accordance with changes in config
# Changes include:
- [ ] Refactor (a non-breaking change that improves code maintainability).
- [ ] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [X] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [X] GCP
- [X] Azure
- [X] AWS
- [X] IBM
# Dev Checklist:
- [X] Added Unit Tests, wherever applicable.
- [X] Updated the Readme, if applicable.
- [X] Existing Tests pass
- [X] Verified functionality locally
- [X] Self Reviewed my code for formatting and complex business logic.
# Other comments:M1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/35Storage tags2023-08-18T13:08:56ZYauheni LesnikauStorage tagsThis MR introduces the support of the Tags for the Stoarage Record and RecordMetadata entitiesThis MR introduces the support of the Tags for the Stoarage Record and RecordMetadata entitiesM1 - Release 0.1ethiraj krishnamanaiduPavel BachylaAlok JoshiYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/36Partition patch api2023-08-18T13:08:55ZAlok JoshiPartition patch apiAdd client library support for Partition patch apiAdd client library support for Partition patch apiM1 - Release 0.1ethiraj krishnamanaiduNeelesh ThakurAliaksei DarafeyeuYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/37Add record tag DTO to the record model2021-01-25T18:11:11ZYauheni LesnikauAdd record tag DTO to the record modelAdd record tag DTO for supporting update tags for multiple recordsAdd record tag DTO for supporting update tags for multiple recordsethiraj krishnamanaiduNeelesh ThakurAlok JoshiYauheni Lesnikauethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/38Url normalization (GONRG-1211)2023-08-18T13:08:53ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comUrl normalization (GONRG-1211)# Description:
Service endpoints receiving with simple string joining
`return StringUtils.join(this.rootUrl, pathAndQuery);`
makes service configuration more complex,
it leads to issues related to root url value.
Service endpoint rece...# Description:
Service endpoints receiving with simple string joining
`return StringUtils.join(this.rootUrl, pathAndQuery);`
makes service configuration more complex,
it leads to issues related to root url value.
Service endpoint receiving was changed from simple string concatenation
to URL normalization.
# Changes include:
- [x] Refactor (a non-breaking change that improves code maintainability).
- [x] Bugfix (a non-breaking change that solves an issue).
- [ ] New feature (a non-breaking change that adds functionality).
- [ ] Breaking change (a change that is not backward-compatible and/or changes current functionality).
# Changes in:
- [x] GCP
- [x] Azure
- [x] AWS
- [x] IBM
# Dev Checklist:
- [x] Added Unit Tests, wherever applicable.
- [ ] Updated the Readme, if applicable.
- [x] Existing Tests pass
- [x] Verified functionality locally
- [x] Self Reviewed my code for formatting and complex business logic.M1 - Release 0.1Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/39WIP: Refactoring ingest2021-03-02T06:26:36ZRiabokon Stanislav(EPAM)[GCP]WIP: Refactoring ingestDmitriy RudkoDmitriy Rudkohttps://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/40Update fossa notice2023-08-18T13:08:51ZDavid Diederichd.diederich@opengroup.orgUpdate fossa noticeM1 - Release 0.1