Changes

Page history
Update M15 Release Notes authored by Okoun-Ola Fabien Houeto's avatar Okoun-Ola Fabien Houeto
Show whitespace changes
Inline Side-by-side
M15-Release-Notes.md
View page @ 1e25a294
...@@ -12,6 +12,7 @@ In keeping with our policy of releasing new capabilities early and often for fee ...@@ -12,6 +12,7 @@ In keeping with our policy of releasing new capabilities early and often for fee
- [`Policy service (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/security-and-compliance/policy) - `AWS`, `Azure`, `IBM`, `GCP` - [`Policy service (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/security-and-compliance/policy) - `AWS`, `Azure`, `IBM`, `GCP`
- `GCP` Performance issue is noted during preship testing, `Policy` service has been turned off by default. - `GCP` Performance issue is noted during preship testing, `Policy` service has been turned off by default.
- `AWS` Policy service is on by default.
- [`Reservoir DDMS (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir) - `Azure`, `IBM` - [`Reservoir DDMS (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir) - `Azure`, `IBM`
- [`External Data Services (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources) - `Azure`, `GCP` - [`External Data Services (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources) - `Azure`, `GCP`
- [`Geospatial Consumption Zone (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/consumption/geospatial) - `IBM`, `AWS (Developer/Local only)` - [`Geospatial Consumption Zone (Experimental/Early Release)`](https://community.opengroup.org/osdu/platform/consumption/geospatial) - `IBM`, `AWS (Developer/Local only)`
...@@ -60,30 +61,31 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha ...@@ -60,30 +61,31 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha
* The content of TubularAssemblyType was by accident replicated in TubularAssenblyStatusType. The content has been corrected for [TubularAssemblyStatusType:1.0.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.18.1/E-R/reference-data/TubularAssemblyStatusType.1.0.0.md). * The content of TubularAssemblyType was by accident replicated in TubularAssenblyStatusType. The content has been corrected for [TubularAssemblyStatusType:1.0.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.18.1/E-R/reference-data/TubularAssemblyStatusType.1.0.0.md).
### Core and Helper Services ### Core and Helper Services
- `Indexer` - `Indexer`
- Info api implementation for adding outerservice version in response. See MR [!424](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/424) - Info api implementation for adding outerservice version in response. See MR [!424](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/424 "info api implementation for adding outerservice version in response")
- `Storage` - `Storage`
- Include collaboration context when pushing to new record changed topic. See MR [!553](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/553) - Include collaboration context when pushing to new record changed topic. See MR [!553](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/553 "Include collaboration context when pushing to new record changed topic")
- Update Storage to be Collaboration Context Aware. See MR [!546](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/546) - Update Storage to be Collaboration Context Aware. See MR [!546](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/546 "Update Storage to be Collaboration Context Aware")
- Changed CRC32C to MD5 to calculate a checksum (GONRG-5884). See MR [!542](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/542) - Changed CRC32C to MD5 to calculate a checksum (GONRG-5884). See MR [!542](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/542 "Changed CRC32C to MD5 to calculate a checksum (GONRG-5884)")
- Added ability to config redis password and ssl (GONRG-5301). See MR [!540](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/540) - Added ability to config redis password and ssl (GONRG-5301). See MR [!540](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/540 "added ability to config redis password and ssl (GONRG-5301)")
- ADR 92: Populate recordBlocks in record modification events. See MR [!539](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/539) - ADR 92: Populate recordBlocks in record modification events. See MR [!539](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/539 "ADR 92: Populate recordBlocks in record modification events")
- `File Service` - `File Service`
- `AWS` - Now the file checksum is added to the metadata. See [ADR](https://community.opengroup.org/osdu/platform/system/file/-/issues/64 "ADR: Calculate Checksum before saving metadata") and MR [!383](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/383) - `AWS` - Now the file checksum is added to the metadata. See [ADR](https://community.opengroup.org/osdu/platform/system/file/-/issues/64 "ADR: Calculate Checksum before saving metadata") and MR [!383](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/383 "Implement the file checksum in AWS")
- `Schema` - `Schema`
- OSDU-DD-Delivery-M15 (v0.18.0). See MR [!407](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/407) - OSDU-DD-Delivery-M15 (v0.18.0). See MR [!407](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/407 "OSDU-DD-Delivery-M15 (v0.18.0)")
- `CRS Conversion v3` - `CRS Conversion v3`
- "Add convertBinGrid endpoint". See MR [!235](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/235) - "Add convertBinGrid endpoint". See MR [!235](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/235 'Resolve "Add convertBinGrid endpoint"')
- `Policy` - `Policy`
- Translate, Logging and Correlation ID update. See MR [!256](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/256) - Translate, Logging and Correlation ID update. See MR [!256](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/256 "Translate, Logging and Correlation ID update")
- `Lib` - `Lib`
- Expose Collaboration Context. See MR [!183](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/183) - Expose Collaboration Context. See MR [!183](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/183 "Expose Collaboration Context")
- Add validator to record ancestry to make sure parent record id is valid. See MR [!174](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/174) - Add validator to record ancestry to make sure parent record id is valid. See MR [!174](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/174 "add validator to record ancestry to make sure parent record id is valid")
### Ingestion services ### Ingestion services
- Manifest-based Ingestion - Major performance improvement. - Manifest-based Ingestion - Major performance improvement.
- osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30) - osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30 "GONRG-5877: Parallelize processing manifest")
- osdu-airflow-lib - Added new Airflow variable core__ingestion__thread_save_number to tune a number of concurrent writings into Storage service for ProcessManifestOperatorR3 operator. This variable is optional (default: 1). See [MR](https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/39) - osdu-airflow-lib - Added new Airflow variable core__ingestion__thread_save_number to tune a number of concurrent writings into Storage service for ProcessManifestOperatorR3 operator. This variable is optional (default: 1). See [MR](https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/39)
- Manifest-based Ingestion - Manifest by reference. - Manifest-based Ingestion - Manifest by reference.
- `AWS` now support ingestion of manifest by reference. - `AWS` now support ingestion of manifest by reference.
...@@ -190,54 +192,53 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha ...@@ -190,54 +192,53 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha
## Bug Fixes ## Bug Fixes
### Core and Helper Services ### Core and Helper Services
- `Storage` - `Storage`
- Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off. See MR [!582](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/582) - Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off. See MR [!582](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/582 "Bug Fix for Azure: update the message to not be sent to recordstopic v2 if feature flag is off")
- Added spring-boot-maven plugin version. See MR [!560](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/560) - Added spring-boot-maven plugin version. See MR [!560](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/560 "added spring-boot-maven plugin version")
- Removing the azure message bus message id. See MR [!550](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/550) - Removing the azure message bus message id. See MR [!550](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/550 "Removing the azure message bus message id")
- Storage Performance (GONRG-4754). See MR [!535](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/535) - Storage Performance (GONRG-4754). See MR [!535](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/535 "Storage Performance (GONRG-4754)")
- Delete all versions of the blob when purging a record. See MR [!534](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/534) - Delete all versions of the blob when purging a record. See MR [!534](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/534 "Delete all versions of the blob when purging a record")
- Consume latest version of core common library and update the way of getting parent records. See MR [!532](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/532) - Consume latest version of core common library and update the way of getting parent records. See MR [!532](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/532 "consume latest version of core common library and update the way of getting parent records")
- `CRS Catalog` - `CRS Catalog`
- Jar type vulnerability fix for catalog service. See MR [!220](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/220) - Jar type vulnerability fix for catalog service. See MR [!220](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/220 "jar type vulnerability fix for catalog service")
- `Unit` - `Unit`
- Unit Service vulnerability fix. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/222) - Unit Service vulnerability fix. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/222 "Unit Service vulnerability fix")
- Squashed commit of the following. See MR [!216](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/216) - Squashed commit of the following. See MR [!216](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/216 "Squashed commit of the following")
- `Dataset` - `Dataset`
- Aws integration merge. See MR [!276](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/276) - Aws integration merge. See MR [!276](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/276 "Aws integration merge")
- `Register` - `Register`
- White source updates. See MR [!262](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/262) - White source updates. See MR [!262](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/262 "white source updates")
- `Schema` - `Schema`
- Fix spring-boot-maven plugin version. See MR [!412](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/412) - Fix spring-boot-maven plugin version. See MR [!412](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/412 "Fix spring-boot-maven plugin version")
- `Indexer` - `Indexer`
- Fix spring-boot-maven-plugin. See MR [!421](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/421) - Fix spring-boot-maven-plugin. See MR [!421](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/421 "Fix spring-boot-maven-plugin")
- Whitesource-update. See MR [!412](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/412) - Whitesource-update. See MR [!412](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/412 "whitesource-update")
- Removed retry attempts and call to storage service. See MR [!384](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/384) - Removed retry attempts and call to storage service. See MR [!384](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/384 "Removed retry attempts and call to storage service")
- `Search` - `Search`
- Added spring-boot-maven plugin version. See MR [!375](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/375) - Added spring-boot-maven plugin version. See MR [!375](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/375 "added spring-boot-maven plugin version")
- Parse-query changes. See MR [!372](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/372) - Parse-query changes. See MR [!372](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/372 "parse-query changes")
- `Partition` - `Partition`
- Update vulnerable dependencies. See MR [!285](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/285) - Update vulnerable dependencies. See MR [!285](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/285 "update vulnerable dependencies")
- `File` - `File`
- Bypass checksum generation for file size more than 5 gbs. See MR [!368](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/368) - Bypass checksum generation for file size more than 5 gbs. See MR [!368](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/368 "Bypass checksum generation for file size more than 5 gbs")
- `Entitlements` - `Entitlements`
- Added spring-boot-maven plugin version. See MR [!382](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/382) - Added spring-boot-maven plugin version. See MR [!382](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/merge_requests/382 "added spring-boot-maven plugin version")
- `Legal` - `Legal`
- Added spring-boot-maven plugin version. See MR [!324](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/324) - Added spring-boot-maven plugin version. See MR [!324](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/324 "added spring-boot-maven plugin version")
- Update vulnerable dependencies. See MR [!315](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/315) - Update vulnerable dependencies. See MR [!315](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/315 "update vulnerable dependencies")
- `Policy` - `Policy`
- Test ibm bucket. See MR [!253](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/253) - Test ibm bucket. See MR [!253](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/merge_requests/253 "Test ibm bucket")
### Ingestion services ### Ingestion services
- `Lib` - `Lib`
- Upgrade lombok version to fix ingestion request deserialization. See MR [!329](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/329) - Upgrade lombok version to fix ingestion request deserialization. See MR [!329](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/329 "upgrade lombok version to fix ingestion request deserialization")
- Trusted development. See MR [!147](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/147) - Trusted development. See MR [!147](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/147 "Trusted development")
- Trusted development. See MR [!143](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/143) - Trusted development. See MR [!143](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/143 "Trusted development")
- Fix Vulnerabilities. See MR [!12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/eds-dms/-/merge_requests/12) - Fix Vulnerabilities. See MR [!12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/eds-dms/-/merge_requests/12 "Fix Vulnerabilities")
- `Segy to Zgy Conversion` - `Segy to Zgy Conversion`
- Fix: segy_configure_remote_access to set http headers. See MR [!163](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/merge_requests/163) - Fix: segy_configure_remote_access to set http headers. See MR [!163](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/merge_requests/163 "fix: segy_configure_remote_access to set http headers")
### DDMS ### DDMS
...@@ -276,7 +277,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha ...@@ -276,7 +277,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha
* Ignite Cache Name * Ignite Cache Name
* We now require a separate identifying string for the name of a cache, as configured in either `application-local.yml`, `application-remote.yml`, or `application-k8s-remote.yml` * We now require a separate identifying string for the name of a cache, as configured in either `application-local.yml`, `application-remote.yml`, or `application-k8s-remote.yml`
* Each kind previously needed to contain the `name` and `chronExpr` properties, but now must contain a `cacheName` property * Each kind previously needed to contain the `name` and `chronExpr` properties, but now must contain a `cacheName` property
* The reason for this change is due to our new support of Persistence-Enabled Ignite Clusters, which strictly impose that cache names do not contain special characters such as `:` due to conflicts with file storage. Please note that taking the kind name and simply replacing all `:` characters with an underscore `\\\\\\\_` should work fine, but you are welcome to name your caches however you like, avoiding special characters. * The reason for this change is due to our new support of Persistence-Enabled Ignite Clusters, which strictly impose that cache names do not contain special characters such as `:` due to conflicts with file storage. Please note that taking the kind name and simply replacing all `:` characters with an underscore `\\\\\\\\\\\\\\\_` should work fine, but you are welcome to name your caches however you like, avoiding special characters.
* Additionally, you will need to configure `koop-config.json` to include the `cacheName` property or else the Provider will be unable to find the cache by `kind` entity alone. * Additionally, you will need to configure `koop-config.json` to include the `cacheName` property or else the Provider will be unable to find the cache by `kind` entity alone.
## Known Issues/Gaps ## Known Issues/Gaps
...@@ -289,8 +290,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha ...@@ -289,8 +290,7 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha
EDS DMS - In the current version EDS DMS can manage bulk data files from one Data Provider only. It will not support a combination of dataset IDs from different data providers.Also, testing can be done using eds-DMS API and not from data set API. EDS DMS - In the current version EDS DMS can manage bulk data files from one Data Provider only. It will not support a combination of dataset IDs from different data providers.Also, testing can be done using eds-DMS API and not from data set API.
https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/436 https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/436 Note: This will work fine when one WPC is fetched from the Data Provider.
Note: This will work fine when one WPC is fetched from the Data Provider.
### Known Issues/Gaps ### Known Issues/Gaps
... ...
......