Storage merge requestshttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests2023-08-18T22:23:08Zhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/254Provide default value false to schema endpoint config2023-08-18T22:23:08ZKrishna Nikhil VedurumudiProvide default value false to schema endpoint config## Type of change
- [ ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud p...## Type of change
- [ ] Bug Fix
- [ ] Feature
**Please provide link to gitlab issue or ADR(Architecture Decision Record)**
## Does this introduce a change in the core logic?
- [YES/NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [YES/NO]
## What is the current behavior?
## What is the new/expected behavior?
Provide default value to schema endpoint disablement feature flag.
## Have you added/updated Unit Tests and Integration Tests?
## Any other useful informationM8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/253handle erroneous AsIngestedCoordinates payload2023-08-18T22:23:09ZNeelesh Thakurhandle erroneous AsIngestedCoordinates payloadHandle erroneous AsIngestedCoordinates payload, Storage batch request fails with 500 error code for following payload:
```json
"data": {
"SpatialLocation": {
"AsIngestedCoordinates": {
"Type": "AnyCrsFeatureCollection"...Handle erroneous AsIngestedCoordinates payload, Storage batch request fails with 500 error code for following payload:
```json
"data": {
"SpatialLocation": {
"AsIngestedCoordinates": {
"Type": "AnyCrsFeatureCollection",
"CoordinateReferenceSystemID": "NAD27",
"VerticalCoordinateReferenceSystemID": null,
"PersistableReferenceCrs": null,
"PersistableReferenceVerticalCrs": null,
"PersistableReferenceUnitZ": null,
"Features": [
{
"Type": "AnyCrsFeature",
"Properties": null,
"Geometry": {
"Coordinates": [
30.8793048,
-87.7801454
],
"Type": "AnyCrsPoint",
"Bbox": null
}
}
],
"Bbox": null
},
"Wgs84Coordinates": null
}
}
```M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/252FoR(Frame-Of-Reference) fixes - ibm2024-01-27T06:33:11ZRitika KaushalFoR(Frame-Of-Reference) fixes - ibmissue #55
This fix is for FoR (Frame-of-Reference) ,basically a header value needs to be passed with storage /query/records:batch endpoint to enable conversion of data. The mandatory values in the system to initiate this are :
1. Head...issue #55
This fix is for FoR (Frame-of-Reference) ,basically a header value needs to be passed with storage /query/records:batch endpoint to enable conversion of data. The mandatory values in the system to initiate this are :
1. Header in request : frame-of-reference=units=SI;crs=wgs84;elevation=msl;azimuth=true north;dates=utc;
1. CRS_API value in storage service at environment level which should point to CRS-CONVESRION-SERVICE Example: `CRS_API=https://<<csp-api-gateway>>/api/crs/converter/v2`
1. A variable `createCrsJWTToken=false` ha been added in CrsConversionService.java in storage core to stop regeneration of JWTToken and to pass the original header as is. This is kept for making it non breaking for CSP. In case CSP wants to pass same authorization header, the value can be false else default value of true results in new token generated as per CSP specific implementation of generating id token.
1. All test cases which were ignored earlier in PostFetchRecordsIntegrationTests.java have been enabled. Configuration discussed in 2 and 3 need to considered for passing the test cases.M8 - Release 0.11Ritika KaushalRitika Kaushalhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/251Storage Schema endpoints feature flag2022-09-16T08:39:23ZLarissa PereiraStorage Schema endpoints feature flagRelated ADR: [62](https://community.opengroup.org/osdu/platform/system/storage/-/issues/62)
**Background** Instead of complete irreversible removal of the Storage Schema endpoints, a fail-safe option to restore application functionalit...Related ADR: [62](https://community.opengroup.org/osdu/platform/system/storage/-/issues/62)
**Background** Instead of complete irreversible removal of the Storage Schema endpoints, a fail-safe option to restore application functionality is needed in cases where missed dependencies on Storage Schemas APIs have gone undetected. The remediation for application failures could be heavy-duty if schemas need to be migrated, thus increasing the need for at least a read-only ability on Storage Schemas to be available for the duration of the remediation.
This MR provides:
- The described fail-safe capability in case of application failure
- Controlled shutoffs of the Storage Schema (hopefully proactively) in order to identify non-conforming or failing client apps.M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/249update core-lib-azure2023-08-18T22:23:11ZAlok Joshiupdate core-lib-azureIncrement core-lib-azure library to incorporate some caching [improvements](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/131) on KV accessIncrement core-lib-azure library to incorporate some caching [improvements](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/131) on KV accessM8 - Release 0.11Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/248Add node selector2022-12-22T23:32:05ZDzmitry_Paulouski (slb)Add node selector# Merge request template# Merge request templateM8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/245Update resources limits [GONRG-2974]2023-08-18T22:23:12ZMikhail Piatliou (EPAM)Update resources limits [GONRG-2974]# Merge request template# Merge request templateM8 - Release 0.11Oleksandr Kosse (EPAM)Oleksandr Kosse (EPAM)https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/244Connected outer services for version info endpoint (GONRG-2681)2022-09-16T08:47:03ZDmitrii Novikov (EPAM)Connected outer services for version info endpoint (GONRG-2681)## Type of change
- [ ] Bug Fix
- [x] Feature
osdu/platform/system/lib/core/os-core-common#47
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so whic...## Type of change
- [ ] Bug Fix
- [x] Feature
osdu/platform/system/lib/core/os-core-common#47
## Does this introduce a change in the core logic?
- [YES]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [x] AWS
- [x] Azure
- [x] GCP
- [x] IBM
## Does this introduce a breaking change?
- [YES]
## What is the current behavior?
Provides connected outer services infoM8 - Release 0.11Rostislav Dublin (EPAM)Rostislav Dublin (EPAM)https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/243response content-type as application/json not text/plain. use media-types2023-08-18T22:23:14ZNeelesh Thakurresponse content-type as application/json not text/plain. use media-typesaddresses #85addresses #85M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/241Resolve: Gonrg 2833 storage on gke2023-08-18T22:23:16ZMykola Zamkovyi (EPAM)Resolve: Gonrg 2833 storage on gke# Merge request template# Merge request templateM8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/239Added GeoJson conversion implementation for Storage batch API2022-09-16T08:52:00ZSanjeev-SLBAdded GeoJson conversion implementation for Storage batch APIhttps://community.opengroup.org/osdu/platform/system/storage/-/issues/82https://community.opengroup.org/osdu/platform/system/storage/-/issues/82M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/238Changes identified in Storage performance testing for legal tag update2023-08-18T22:23:17ZAkansha Rajput[Microsoft]Changes identified in Storage performance testing for legal tag update## Type of change
- Enhancement with some code cleanup.
## Does this introduce a change in the core logic?
- NO
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] Azure
## Does this intr...## Type of change
- Enhancement with some code cleanup.
## Does this introduce a change in the core logic?
- NO
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] Azure
## Does this introduce a breaking change?
- NO
## What is the current behavior?
If the feature flag for legal tag update is enabled and storage service is on >1 pods, all api requests start giving 415 Unsupported Media Type error after a few successful requests.
This is happening because thread context(request headers) is not getting cleared properly and new requests are using older values for Thread Local variables.
## What is the new/expected behavior?
The context will be cleared for any new api requests where we need to set new headers explicitly, this will ensure that even when an old thread is picked from the thread pool for a new request, it's context is cleared before hand and we dont face 415 error code because of it.M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/236Bug: Null array for meta block2023-08-18T22:23:19ZMina OtgonboldBug: Null array for meta blockAdd a fix for bug when the meta is defined as:
"meta": [null]
Issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/84Add a fix for bug when the meta is defined as:
"meta": [null]
Issue: https://community.opengroup.org/osdu/platform/system/storage/-/issues/84M8 - Release 0.11Mina OtgonboldMina Otgonboldhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/202Remove custom acl validation for Azure2023-08-18T22:23:48ZNeelesh ThakurRemove custom acl validation for AzureAddresses issue [73 ](https://community.opengroup.org/osdu/platform/system/storage/-/issues/73)
Added integration test with entitlements v2 for validation.Addresses issue [73 ](https://community.opengroup.org/osdu/platform/system/storage/-/issues/73)
Added integration test with entitlements v2 for validation.M8 - Release 0.11https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/288Upgrade OSDU dependencies2021-10-27T17:17:15ZDavid Diederichd.diederich@opengroup.orgUpgrade OSDU dependencies# Merge request template# Merge request templateM9 - Release 0.12David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/286upgrade IBM test core lib version to 0.12.0-SNAPSHOT2023-08-18T22:22:37ZAshwani Pandeyupgrade IBM test core lib version to 0.12.0-SNAPSHOTupgrade IBM test core lib version to 0.12.0-SNAPSHOTupgrade IBM test core lib version to 0.12.0-SNAPSHOTM9 - Release 0.12Ashwani PandeyAshwani Pandeyhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/285Update to use latest core common2022-12-22T16:49:14ZAlok JoshiUpdate to use latest core commonPlease refer to this issue https://community.opengroup.org/osdu/platform/system/home/-/issues/90Please refer to this issue https://community.opengroup.org/osdu/platform/system/home/-/issues/90M9 - Release 0.12Alok JoshiAlok Joshihttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/282consume blobstorage fixes - throw explicit 503 when server is busy to process...2023-08-18T22:22:38ZNeelesh Thakurconsume blobstorage fixes - throw explicit 503 when server is busy to process BlobStorage request so client can retryRelated Azure Lib MR: [155](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/155)Related Azure Lib MR: [155](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/155)M9 - Release 0.12https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/281Health check2023-08-18T22:22:40ZSmitha ManjunathHealth checkusing common.propertiesusing common.propertiesM9 - Release 0.12Smitha ManjunathSmitha Manjunathhttps://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/279Added logback into GCP IT Module (GONRG-3445)2021-10-10T16:05:41ZRiabokon Stanislav(EPAM)[GCP]Added logback into GCP IT Module (GONRG-3445)## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM...## Type of change
- [X] Bug Fix
- [ ] Feature
## Does this introduce a change in the core logic?
- [NO]
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [X] GCP
- [ ] IBM
## Does this introduce a breaking change?
- [NO]
## What is the current behavior?
The id_token is exposed in int-tests logs
## What is the new/expected behavior?
The id_token logs are disabledM9 - Release 0.12Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]