... | ... | @@ -12,6 +12,7 @@ In keeping with our policy of releasing new capabilities early and often for fee |
|
|
|
|
|
- [`Policy service (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/security-and-compliance/policy) - `AWS`, `Azure`, `IBM`, `GCP`
|
|
|
- `GCP` Performance issue is noted during preship testing, `Policy` service has been turned off by default.
|
|
|
- `AWS` Policy service is on by default.
|
|
|
- [`Reservoir DDMS (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir) - `Azure`, `IBM`
|
|
|
- [`External Data Services (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources) - `Azure`, `GCP`
|
|
|
- [`Geospatial Consumption Zone (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/consumption/geospatial) - `IBM`, `AWS (Developer/Local only)`
|
... | ... | @@ -60,30 +61,31 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
* The content of TubularAssemblyType was by accident replicated in TubularAssenblyStatusType. The content has been corrected for [TubularAssemblyStatusType:1.0.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.18.1/E-R/reference-data/TubularAssemblyStatusType.1.0.0.md).
|
|
|
|
|
|
### Core and Helper Services
|
|
|
|
|
|
- `Indexer`
|
|
|
- Info api implementation for adding outerservice version in response. See MR [!424](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/424)
|
|
|
- Info api implementation for adding outerservice version in response. See MR [!424](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/424 "info api implementation for adding outerservice version in response")
|
|
|
- `Storage`
|
|
|
- Include collaboration context when pushing to new record changed topic. See MR [!553](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/553)
|
|
|
- Update Storage to be Collaboration Context Aware. See MR [!546](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/546)
|
|
|
- Changed CRC32C to MD5 to calculate a checksum (GONRG-5884). See MR [!542](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/542)
|
|
|
- Added ability to config redis password and ssl (GONRG-5301). See MR [!540](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/540)
|
|
|
- ADR 92: Populate recordBlocks in record modification events. See MR [!539](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/539)
|
|
|
- Include collaboration context when pushing to new record changed topic. See MR [!553](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/553 "Include collaboration context when pushing to new record changed topic")
|
|
|
- Update Storage to be Collaboration Context Aware. See MR [!546](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/546 "Update Storage to be Collaboration Context Aware")
|
|
|
- Changed CRC32C to MD5 to calculate a checksum (GONRG-5884). See MR [!542](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/542 "Changed CRC32C to MD5 to calculate a checksum (GONRG-5884)")
|
|
|
- Added ability to config redis password and ssl (GONRG-5301). See MR [!540](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/540 "added ability to config redis password and ssl (GONRG-5301)")
|
|
|
- ADR 92: Populate recordBlocks in record modification events. See MR [!539](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/539 "ADR 92: Populate recordBlocks in record modification events")
|
|
|
- `File Service`
|
|
|
- `AWS` - Now the file checksum is added to the metadata. See [ADR](https://community.opengroup.org/osdu/platform/system/file/-/issues/64 "ADR: Calculate Checksum before saving metadata") and MR [!383](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/383)
|
|
|
- `AWS` - Now the file checksum is added to the metadata. See [ADR](https://community.opengroup.org/osdu/platform/system/file/-/issues/64 "ADR: Calculate Checksum before saving metadata") and MR [!383](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/383 "Implement the file checksum in AWS")
|
|
|
- `Schema`
|
|
|
- OSDU-DD-Delivery-M15 (v0.18.0). See MR [!407](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/407)
|
|
|
- OSDU-DD-Delivery-M15 (v0.18.0). See MR [!407](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/407 "OSDU-DD-Delivery-M15 (v0.18.0)")
|
|
|
- `CRS Conversion v3`
|
|
|
- "Add convertBinGrid endpoint". See MR [!235](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/235)
|
|
|
- "Add convertBinGrid endpoint". See MR [!235](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/235 'Resolve "Add convertBinGrid endpoint"')
|
|
|
- `Policy`
|
|
|
- Translate, Logging and Correlation ID update. See MR [!256](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/256)
|
|
|
- Translate, Logging and Correlation ID update. See MR [!256](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/256 "Translate, Logging and Correlation ID update")
|
|
|
- `Lib`
|
|
|
- Expose Collaboration Context. See MR [!183](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/183)
|
|
|
- Add validator to record ancestry to make sure parent record id is valid. See MR [!174](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/174)
|
|
|
- Expose Collaboration Context. See MR [!183](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/183 "Expose Collaboration Context")
|
|
|
- Add validator to record ancestry to make sure parent record id is valid. See MR [!174](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/174 "add validator to record ancestry to make sure parent record id is valid")
|
|
|
|
|
|
### Ingestion services
|
|
|
|
|
|
- Manifest-based Ingestion - Major performance improvement.
|
|
|
- osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30)
|
|
|
- osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30 "GONRG-5877: Parallelize processing manifest")
|
|
|
- osdu-airflow-lib - Added new Airflow variable core__ingestion__thread_save_number to tune a number of concurrent writings into Storage service for ProcessManifestOperatorR3 operator. This variable is optional (default: 1). See [MR](https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/39)
|
|
|
- Manifest-based Ingestion - Manifest by reference.
|
|
|
- `AWS` now support ingestion of manifest by reference.
|
... | ... | @@ -190,54 +192,53 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
## Bug Fixes
|
|
|
|
|
|
### Core and Helper Services
|
|
|
|
|
|
- `Storage`
|
|
|
- Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off. See MR [!582](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/582)
|
|
|
- Added spring-boot-maven plugin version. See MR [!560](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/560)
|
|
|
- Removing the azure message bus message id. See MR [!550](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/550)
|
|
|
- Storage Performance (GONRG-4754). See MR [!535](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/535)
|
|
|
- Delete all versions of the blob when purging a record. See MR [!534](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/534)
|
|
|
- Consume latest version of core common library and update the way of getting parent records. See MR [!532](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/532)
|
|
|
- Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off. See MR [!582](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/582 "Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off")
|
|
|
- Added spring-boot-maven plugin version. See MR [!560](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/560 "added spring-boot-maven plugin version")
|
|
|
- Removing the azure message bus message id. See MR [!550](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/550 "Removing the azure message bus message id")
|
|
|
- Storage Performance (GONRG-4754). See MR [!535](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/535 "Storage Performance (GONRG-4754)")
|
|
|
- Delete all versions of the blob when purging a record. See MR [!534](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/534 "Delete all versions of the blob when purging a record")
|
|
|
- Consume latest version of core common library and update the way of getting parent records. See MR [!532](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/532 "consume latest version of core common library and update the way of getting parent records")
|
|
|
- `CRS Catalog`
|
|
|
- Jar type vulnerability fix for catalog service. See MR [!220](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/220)
|
|
|
- Jar type vulnerability fix for catalog service. See MR [!220](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/220 "jar type vulnerability fix for catalog service")
|
|
|
- `Unit`
|
|
|
- Unit Service vulnerability fix. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/222)
|
|
|
- Squashed commit of the following. See MR [!216](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/216)
|
|
|
- Unit Service vulnerability fix. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/222 "Unit Service vulnerability fix")
|
|
|
- Squashed commit of the following. See MR [!216](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/216 "Squashed commit of the following")
|
|
|
- `Dataset`
|
|
|
- Aws integration merge. See MR [!276](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/276)
|
|
|
- Aws integration merge. See MR [!276](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/276 "Aws integration merge")
|
|
|
- `Register`
|
|
|
- White source updates. See MR [!262](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/262)
|
|
|
- White source updates. See MR [!262](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/262 "white source updates")
|
|
|
- `Schema`
|
|
|
- Fix spring-boot-maven plugin version. See MR [!412](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/412)
|
|
|
- Fix spring-boot-maven plugin version. See MR [!412](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/412 "Fix spring-boot-maven plugin version")
|
|
|
- `Indexer`
|
|
|
- Fix spring-boot-maven-plugin. See MR [!421](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/421)
|
|
|
- Whitesource-update. See MR [!412](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/412)
|
|
|
- Removed retry attempts and call to storage service. See MR [!384](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/384)
|
|
|
- Fix spring-boot-maven-plugin. See MR [!421](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/421 "Fix spring-boot-maven-plugin")
|
|
|
- Whitesource-update. See MR [!412](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/412 "whitesource-update")
|
|
|
- Removed retry attempts and call to storage service. See MR [!384](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/384 "Removed retry attempts and call to storage service")
|
|
|
- `Search`
|
|
|
- Added spring-boot-maven plugin version. See MR [!375](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/375)
|
|
|
- Parse-query changes. See MR [!372](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/372)
|
|
|
- Added spring-boot-maven plugin version. See MR [!375](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/375 "added spring-boot-maven plugin version")
|
|
|
- Parse-query changes. See MR [!372](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/372 "parse-query changes")
|
|
|
- `Partition`
|
|
|
- Update vulnerable dependencies. See MR [!285](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/285)
|
|
|
- Update vulnerable dependencies. See MR [!285](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/285 "update vulnerable dependencies")
|
|
|
- `File`
|
|
|
- Bypass checksum generation for file size more than 5 gbs. See MR [!368](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/368)
|
|
|
- Bypass checksum generation for file size more than 5 gbs. See MR [!368](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/368 "Bypass checksum generation for file size more than 5 gbs")
|
|
|
- `Entitlements`
|
|
|
- Added spring-boot-maven plugin version. See MR [!382](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/382)
|
|
|
- Added spring-boot-maven plugin version. See MR [!382](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/382 "added spring-boot-maven plugin version")
|
|
|
- `Legal`
|
|
|
- Added spring-boot-maven plugin version. See MR [!324](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/324)
|
|
|
- Update vulnerable dependencies. See MR [!315](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/315)
|
|
|
- Added spring-boot-maven plugin version. See MR [!324](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/324 "added spring-boot-maven plugin version")
|
|
|
- Update vulnerable dependencies. See MR [!315](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/315 "update vulnerable dependencies")
|
|
|
- `Policy`
|
|
|
- Test ibm bucket. See MR [!253](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/253)
|
|
|
|
|
|
- Test ibm bucket. See MR [!253](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/253 "Test ibm bucket")
|
|
|
|
|
|
### Ingestion services
|
|
|
|
|
|
- `Lib`
|
|
|
- Upgrade lombok version to fix ingestion request deserialization. See MR [!329](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/329)
|
|
|
- Trusted development. See MR [!147](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/147)
|
|
|
- Trusted development. See MR [!143](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/143)
|
|
|
- Fix Vulnerabilities. See MR [!12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/eds-dms/-/merge_requests/12)
|
|
|
- Upgrade lombok version to fix ingestion request deserialization. See MR [!329](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/329 "upgrade lombok version to fix ingestion request deserialization")
|
|
|
- Trusted development. See MR [!147](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/147 "Trusted development")
|
|
|
- Trusted development. See MR [!143](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/143 "Trusted development")
|
|
|
- Fix Vulnerabilities. See MR [!12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/eds-dms/-/merge_requests/12 "Fix Vulnerabilities")
|
|
|
- `Segy to Zgy Conversion`
|
|
|
- Fix: segy_configure_remote_access to set http headers. See MR [!163](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/merge_requests/163)
|
|
|
|
|
|
- Fix: segy_configure_remote_access to set http headers. See MR [!163](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/merge_requests/163 "fix: segy_configure_remote_access to set http headers")
|
|
|
|
|
|
### DDMS
|
|
|
|
... | ... | @@ -276,7 +277,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
* Ignite Cache Name
|
|
|
* We now require a separate identifying string for the name of a cache, as configured in either `application-local.yml`, `application-remote.yml`, or `application-k8s-remote.yml`
|
|
|
* Each kind previously needed to contain the `name` and `chronExpr` properties, but now must contain a `cacheName` property
|
|
|
* The reason for this change is due to our new support of Persistence-Enabled Ignite Clusters, which strictly impose that cache names do not contain special characters such as `:` due to conflicts with file storage. Please note that taking the kind name and simply replacing all `:` characters with an underscore `\\\\\\\_` should work fine, but you are welcome to name your caches however you like, avoiding special characters.
|
|
|
* The reason for this change is due to our new support of Persistence-Enabled Ignite Clusters, which strictly impose that cache names do not contain special characters such as `:` due to conflicts with file storage. Please note that taking the kind name and simply replacing all `:` characters with an underscore `\\\\\\\\\\\\\\\_` should work fine, but you are welcome to name your caches however you like, avoiding special characters.
|
|
|
* Additionally, you will need to configure `koop-config.json` to include the `cacheName` property or else the Provider will be unable to find the cache by `kind` entity alone.
|
|
|
|
|
|
## Known Issues/Gaps
|
... | ... | @@ -289,8 +290,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
|
|
|
EDS DMS - In the current version EDS DMS can manage bulk data files from one Data Provider only. It will not support a combination of dataset IDs from different data providers.Also, testing can be done using eds-DMS API and not from data set API.
|
|
|
|
|
|
https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/436
|
|
|
Note: This will work fine when one WPC is fetched from the Data Provider.
|
|
|
https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/436 Note: This will work fine when one WPC is fetched from the Data Provider.
|
|
|
|
|
|
### Known Issues/Gaps
|
|
|
|
... | ... | |