Skip to content

Changes

Page history
Update M14 Release Notes authored by Joel Romero's avatar Joel Romero
Hide whitespace changes
Inline Side-by-side
M14-Release-Notes.md
View page @ 4800797a
...@@ -8,11 +8,11 @@ For detailed services that are included in this release, please refer to [M14 ta ...@@ -8,11 +8,11 @@ For detailed services that are included in this release, please refer to [M14 ta
In keeping with our policy of releasing new capabilities early and often for feedback, we are pleased to introduce several new **experimental** platform capabilities in this milestone release. These are currently only available in specific cloud service providers. See details below. In keeping with our policy of releasing new capabilities early and often for feedback, we are pleased to introduce several new **experimental** platform capabilities in this milestone release. These are currently only available in specific cloud service providers. See details below.
- [`Policy service (Experimental)`](https://community.opengroup.org/osdu/platform/security-and-compliance/policy) - [`Policy service (Experimental)`](https://community.opengroup.org/osdu/platform/security-and-compliance/policy)
- [`Reservoir DDMS (Experimental)`](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir) - [`Reservoir DDMS (Experimental)`](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir)
- [`External Data Services (Experimental)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow) - [`External Data Services (Experimental)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow)
- [`Geospatial Consumption Zone (Experimental)`](https://community.opengroup.org/osdu/platform/consumption/geospatial) - [`Geospatial Consumption Zone (Experimental)`](https://community.opengroup.org/osdu/platform/consumption/geospatial)
- [`Manifest ingestion by reference (Experimental)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/17) - [`Manifest ingestion by reference (Experimental)`](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/17 "Manifest by reference, first code version")
# Table of Content # Table of Content
...@@ -20,10 +20,11 @@ In keeping with our policy of releasing new capabilities early and often for fee ...@@ -20,10 +20,11 @@ In keeping with our policy of releasing new capabilities early and often for fee
## New Features ## New Features
### Data Definition ### Data Definition
No in-place schema changes are required in M14 (no M13 schema resources were changes). Refer here for [full changes "OSDU schemas"](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/ChangeReport.md#snapshot-2022-07-22-towards-m14) and [full changes "OSDU reference-values"](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/ReferenceValues/ChangeLogs/M14/README.md) No in-place schema changes are required in M14 (no M13 schema resources were changes). Refer here for [full changes "OSDU schemas"](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/ChangeReport.md#snapshot-2022-07-22-towards-m14) and [full changes "OSDU reference-values"](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/ReferenceValues/ChangeLogs/M14/README.md)
* **New Entity Types**
* **New Entity Types**
* [WellboreOpening](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/WellboreOpening.1.0.0.md) and [IsolatedInterval](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/IsolatedInterval.1.0.0.md) are added to the M13 [PerforationInterval](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/PerforationInterval.1.0.0.md) and [PerforationJob](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/PerforationJob.1.0.0.md). WellboreOpening can relate to experimental Reservoir and ReservoirSegment. A [worked example with a narrative](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Examples/WorkedExamples/WellCompletions/README.md) has been provided in the context of "Well Completions" * [WellboreOpening](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/WellboreOpening.1.0.0.md) and [IsolatedInterval](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/IsolatedInterval.1.0.0.md) are added to the M13 [PerforationInterval](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/PerforationInterval.1.0.0.md) and [PerforationJob](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/PerforationJob.1.0.0.md). WellboreOpening can relate to experimental Reservoir and ReservoirSegment. A [worked example with a narrative](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Examples/WorkedExamples/WellCompletions/README.md) has been provided in the context of "Well Completions"
* [Reservoir](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Reservoir.0.0.0.md) and [ReservoirSegment](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/ReservoirSegment.0.0.0.md) have been added as experimental placeholders with version 0.0.0. These entities are currently under creation. No mass-ingestion should be performed because the schema will update with breaking changes. * [Reservoir](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Reservoir.0.0.0.md) and [ReservoirSegment](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/ReservoirSegment.0.0.0.md) have been added as experimental placeholders with version 0.0.0. These entities are currently under creation. No mass-ingestion should be performed because the schema will update with breaking changes.
* Well Execution added [WellActivity](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/WellActivity.1.0.0.md). It integrates with updated ActivityPlan and OperationsReport. * Well Execution added [WellActivity](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/WellActivity.1.0.0.md). It integrates with updated ActivityPlan and OperationsReport.
...@@ -44,160 +45,158 @@ No in-place schema changes are required in M14 (no M13 schema resources were cha ...@@ -44,160 +45,158 @@ No in-place schema changes are required in M14 (no M13 schema resources were cha
* **Seismic Domain** * **Seismic Domain**
* [SeismicTraceData:1.3.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/SeismicTraceData.1.3.0.md), now with reference data relationships for SeismicPhase and SeismicPolarity; [migration information](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Guides/MigrationGuides/M14/SeismicTraceData.1.2.0.md). * [SeismicTraceData:1.3.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/SeismicTraceData.1.3.0.md), now with reference data relationships for SeismicPhase and SeismicPolarity; [migration information](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Guides/MigrationGuides/M14/SeismicTraceData.1.2.0.md).
* **Patched Entity Types** * **Patched Entity Types**
* PPDM synchronizations for * PPDM synchronizations for
* [SeismicProcessingStageType:1.0.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/reference-data/SeismicProcessingStageType.1.0.1.md), [migration information for content upgrade](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Guides/MigrationGuides/M14/SeismicProcessingStageType.1.0.0.md). * [SeismicProcessingStageType:1.0.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/reference-data/SeismicProcessingStageType.1.0.1.md), [migration information for content upgrade](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/Guides/MigrationGuides/M14/SeismicProcessingStageType.1.0.0.md).
* [VerticalMeasurementType:1.0.2](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/reference-data/VerticalMeasurementType.1.0.2.md) (no content change). * [VerticalMeasurementType:1.0.2](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/reference-data/VerticalMeasurementType.1.0.2.md) (no content change).
* [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/WellboreMarkerSet.1.2.1.md) got a missing description added. No migration required. **Warning:** By accident, the 1.2.1 version refers to AbstractWorkProductComponent:1.1.0, which is correctly identified by the Schema service bootstrapping as breaking change. To fix this, please use this patched version of [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/d0f3e96bc4e7510364411cb14c3d1e3295fa1324/SchemaRegistrationResources/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json#L115). * [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/WellboreMarkerSet.1.2.1.md) got a missing description added. No migration required. **Warning:** By accident, the 1.2.1 version refers to AbstractWorkProductComponent:1.1.0, which is correctly identified by the Schema service bootstrapping as breaking change. To fix this, please use this patched version of [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/d0f3e96bc4e7510364411cb14c3d1e3295fa1324/SchemaRegistrationResources/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json#L115).
* [Wellbore:1.1.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Wellbore.1.1.1.md) had a duplicate property added in version 1.1.0 (by mistake), which is now deprecated. * [Wellbore:1.1.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Wellbore.1.1.1.md) had a duplicate property added in version 1.1.0 (by mistake), which is now deprecated.
### Core and Helper Services ### Core and Helper Services
- `Entitlement and Obligations`
- `AWS` `Azure`- Entitlement and Obligation with Policy Service and Open Policy Agent (OPA) implemented. Policies are deployed by default and can be updated via the Policy Service. Storage and Search services now use Policy Service and OPA by default - see issues [49](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/49), [45](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/45), [46](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/36).
- `AdminCLI` - Entitlement and Obligation introduced and AdminCLI that makes managing and testing policies dramatically easier. It also provides support for other services.
- `Entitlement and Obligations`
- `AWS` `Azure`- Entitlement and Obligation with Policy Service and Open Policy Agent (OPA) implemented. Policies are deployed by default and can be updated via the Policy Service. Storage and Search services now use Policy Service and OPA by default - see issues [49](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/49 "Bundle file read/write SPI for CSP - AWS"), [45](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/45 "Bundle bootstrapping for CSP - AWS"), [46](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/36 "Backup and restore of bundles for CSP - AWS").
- `AdminCLI` - Entitlement and Obligation introduced and AdminCLI that makes managing and testing policies dramatically easier. It also provides support for other services.
- `Indexer` - `Indexer`
- Geoshape decimation. See MR [!387](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/387) - Geoshape decimation. See MR [!387](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/387 "Geoshape decimation")
- Indexing virtual properties. See MR [!357](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/357) - Indexing virtual properties. See MR [!357](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/357 "Indexing virtual properties")
- `Lib` - `Lib`
- Updated type of dataHash to save hash of multiple fields. See MR [!173](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/173) - Updated type of dataHash to save hash of multiple fields. See MR [!173](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/173 "Updated type of dataHash to save hash of multiple fields")
- Added extensionProperties to legal tag properties. See MR [!172](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/172) - Added extensionProperties to legal tag properties. See MR [!172](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/172 "Added extensionProperties to legal tag properties")
- Add rc-versions if it is a merge into master. See MR [!71](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/71) - Add rc-versions if it is a merge into master. See MR [!71](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/71 "Add rc-versions if it is a merge into master")
- `Search` - `Search`
- Added OPA Translation integration. See MR [!290](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/290) - Added OPA Translation integration. See MR [!290](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/290 "Added OPA Translation integration")
### DDMS ### DDMS
- `Seismic DDMS`
- Added list endpoint as post, see [!436](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/436). - `Seismic DDMS`
- Performance improvement on caching solution, see [!453](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/453). - Added list endpoint as post, see [!436](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/436 "feat: added list endpoint as post").
- Update impersonation token to track original user, see [!525](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/525). - Performance improvement on caching solution, see [!453](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/453 "feat: enabled local cache to reduce the usage of the shared cache").
- Update impersonation token to track original user, see [!525](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/525 "feat: update impersonation token to track original user").
- `Wellbore DDMS` - `Wellbore DDMS`
- [Some documentation and tutorials](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/tree/master/docs/tutorial) reviewed and added. - [Some documentation and tutorials](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/tree/master/docs/tutorial) reviewed and added.
- Performance enhancement on reading WellLog record, see [!599](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/599). - Performance enhancement on reading WellLog record, see [!599](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/599 "Read data partial record").
- Performance enhancement on describe bulk data on big arrays, see [!605](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/605). - Performance enhancement on describe bulk data on big arrays, see [!605](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/605 "Performance improvement on data/describe without filter").
- Preview of read bulk data improvement (disabled by default). [!621](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/621). - Preview of read bulk data improvement (disabled by default). [!621](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/621 "Feature: read fast track").
- Support of [Wellbore:1.1.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Wellbore.1.1.1.md) and [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/WellboreMarkerSet.1.2.1.md), see issue [#52](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/52) - Support of [Wellbore:1.1.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/master-data/Wellbore.1.1.1.md) and [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.17.0/E-R/work-product-component/WellboreMarkerSet.1.2.1.md), see issue [#52](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/52 "Add support for wellbore1.1.1 and wellboremarkerset1.2.1")
- `Reservoir DDMS`
- `Reservoir DDMS` - Provide docker image for SSH-compatible ETP client for external RESQML file ingestion in Reservoir DDMS and RESQML file generation from Reservoir DDMS. See [Container Registry](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/container_registry/13762)
- Provide docker image for SSH-compatible ETP client for external RESQML file ingestion in Reservoir DDMS and RESQML file generation from Reservoir DDMS. See [Container Registry](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/container_registry/13762)
### External Data Services (EDS) ### External Data Services (EDS)
- M14 release summary
- [Email notification service will send out a summary of all the jobs based on Failed records on the dates configured in ConnectedSourceRegistryEntry](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/238)
- [To Fetch Failed records of the previous airflow runs](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/223)
- [Reference value mapping functionality will take care of any reference value mismatch when ingesting data into the Data Operator system from the Provider](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/132)
- [Added ingested records summary and count, failed records summary and count in airflow XCOM summary](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/242)
- M14 release summary
- [Email notification service will send out a summary of all the jobs based on Failed records on the dates configured in ConnectedSourceRegistryEntry](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/238 "Email Automation")
- [To Fetch Failed records of the previous airflow runs](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/223 "Error handling on interruption for fetch and ingest")
- [Reference value mapping functionality will take care of any reference value mismatch when ingesting data into the Data Operator system from the Provider](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/132 "Determine reference data strategy")
- [Added ingested records summary and count, failed records summary and count in airflow XCOM summary](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/242 "EDS and managing partially failed fetch-and-ingest runs")
- M14 Schema changes - M14 Schema changes
- Modified "ClientID" into "ClientIDKeyName"
- Modified "ClientID" into "ClientIDKeyName" - Added "EmailTriggerFreqency" - This will hold email trigger frequency
- Added "EmailTriggerFreqency" - This will hold email trigger frequency - "ReferenceValueMappings" as an object - This will hold the respective provider and operator reference schema mapping
- "ReferenceValueMappings" as an object - This will hold the respective provider and operator reference schema mapping - "FailedRecords" - This will hold the previously failed records, and failed records will be processed during the latest run
- "FailedRecords" - This will hold the previously failed records, and failed records will be processed during the latest run
- List of issues that can again be tested in M14 for closure - List of issues that can again be tested in M14 for closure
- [EDS job erroneously changes the kind name when transferring record](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/368 "EDS job erroneously changes the kind name when transferring record")
- [EDS job erroneously changes the kind name when transferring record](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/368) - [EDS Katalyst - error while fetching data from client](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/249 "EDS Katalyst - error while fetching data from client")
- [EDS Katalyst - error while fetching data from client](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/external-data-framework/-/issues/249) - [Unable to fetch & ingest WPC from external data source](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/367 "Unable to fetch & ingest WPC from external data source")
- [Unable to fetch & ingest WPC from external data source](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/367)
### GCZ (Geospatial Consumption Zone) ### GCZ (Geospatial Consumption Zone)
- - Transformer Enhancements:
o Implemented batch loading options `loadInBatch` and `batchSize` for Transformer to significantly reduce memory overhead when ingesting large datasets. The batch size is configurable and can be customized higher or lower to scale for different environments. - Transformer Enhancements:
o Implemented Schema Service fetching in Transformer to determine field types of record attributes with greater authority * Implemented batch loading options `loadInBatch` and `batchSize` for Transformer to significantly reduce memory overhead when ingesting large datasets. The batch size is configurable and can be customized higher or lower to scale for different environments.
o Config changes * Implemented Schema Service fetching in Transformer to determine field types of record attributes with greater authority
`authorizeUrl` removed from configurable options * Config changes
`dataPartitionId` added as configurable option * `authorizeUrl` removed from configurable options
o New log file `geoJsonSummaryLog` generated to track records that failed to ingest due to invalid geometries * `dataPartitionId` added as configurable option
- Provider Bug Fixes: * New log file `geoJsonSummaryLog` generated to track records that failed to ingest due to invalid geometries
o Prevent crash when nonexistent feature layer is requested from Provider - Provider Bug Fixes:
o Prevent error when `returnCountOnly` and `returnExtentOnly` request parameters are simultaneously true in a request sent to a FeatureLayer generated by the Provider * Prevent crash when nonexistent feature layer is requested from Provider
o Prevent `returnExtentOnly` requests from erroneously returning null geometry * Prevent error when `returnCountOnly` and `returnExtentOnly` request parameters are simultaneously true in a request sent to a FeatureLayer generated by the Provider
o General improvements to error handling * Prevent `returnExtentOnly` requests from erroneously returning null geometry
- Wiki updates * General improvements to error handling
- Wiki updates
## Bug Fixes ## Bug Fixes
### Core and Helper Services ### Core and Helper Services
- `Lib` - `Lib`
- correction in case of empty dataset service payload for manifest by reference. See MR [!79](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/79) - correction in case of empty dataset service payload for manifest by reference. See MR [!79](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/79 "correction in case of empty dataset service payload for manifest by reference")
- Upgrade Gson. See MR [!205](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/205) - Upgrade Gson. See MR [!205](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/205 "Upgrade Gson")
- [CRS normalization] Ingested manifest with 2D CRS Persistable reference cannot be found by Search (GONRG-5581). See MR [!175](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/175) - \[CRS normalization\] Ingested manifest with 2D CRS Persistable reference cannot be found by Search (GONRG-5581). See MR [!175](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/175 "[CRS normalization] Ingested manifest with 2D CRS Persistable reference cannot be found by Search (GONRG-5581)")
- Add no-authorization header request capability. See MR [!75](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/75) - Add no-authorization header request capability. See MR [!75](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/75 "Add no-authorization header request capability")
- Upgrade dependencies to fix vulnerability and migrating from springfox to open-api. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/222) - Upgrade dependencies to fix vulnerability and migrating from springfox to open-api. See MR [!222](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/222 "Upgrade dependencies to fix vulnerability and migrating from springfox to open-api")
- Upgrade dependencies to fix vulnerability and migrating from springfox to open-api. See MR [!190](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/190) - Upgrade dependencies to fix vulnerability and migrating from springfox to open-api. See MR [!190](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/190 "Upgrade dependencies to fix vulnerability and migrating from springfox to open-api")
- Dataset dms client update : add a missing function for DMS client (missing new enpoint). See MR [!74](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/74) - Dataset dms client update : add a missing function for DMS client (missing new enpoint). See MR [!74](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/74 "Dataset dms client update : add a missing function for DMS client (missing new enpoint)")
- Fix typo in setup.py. See MR [!73](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/73) - Fix typo in setup.py. See MR [!73](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/73 "Fix typo in setup.py")
- GONRG-3815: Remove urls from base client. See MR [!67](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/67) - GONRG-3815: Remove urls from base client. See MR [!67](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/67 "GONRG-3815: Remove urls from base client")
- `File` - `File`
- added changes for rookceph. See MR [!356](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/356) - added changes for rookceph. See MR [!356](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/356 "added changes for rookceph")
- Update swagger documentation for end point "downloadURL" and end point - File uploadURL. See MR [!347](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/347) - Update swagger documentation for end point "downloadURL" and end point - File uploadURL. See MR [!347](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/347 'Update swagger documentation for end point "downloadURL" and end point - File uploadURL')
- Fix WhiteSource vulnerability issues. See MR [!327](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/327) - Fix WhiteSource vulnerability issues. See MR [!327](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/327 "Fix WhiteSource vulnerability issues")
- `Partition` - `Partition`
- Upgrade Gson. See MR [!274](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/274) - Upgrade Gson. See MR [!274](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/274 "Upgrade Gson")
- Fixed security issues (GONRG-5608). See MR [!260](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/260) - Fixed security issues (GONRG-5608). See MR [!260](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/260 "Fixed security issues (GONRG-5608)")
- `Notification` - `Notification`
- Upgrade Gson. See MR [!267](https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/267) - Upgrade Gson. See MR [!267](https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/267 "Upgrade Gson")
- `Dataset` - `Dataset`
- Upgrade Gson. See MR [!255](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/255) - Upgrade Gson. See MR [!255](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/255 "Upgrade Gson")
- Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi. See MR [!250](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/250) - Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi. See MR [!250](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/250 "Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi")
- Update DatasetFilter. See MR [!247](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/247) - Update DatasetFilter. See MR [!247](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/247 "Update DatasetFilter")
- `Storage` - `Storage`
- [CRS normalization] Coordinates in the original CRS system are not preserved in the index (GONRG-5581). See MR [!514](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/514) - \[CRS normalization\] Coordinates in the original CRS system are not preserved in the index (GONRG-5581). See MR [!514](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/514 "[CRS normalization] Coordinates in the original CRS system are not preserved in the index (GONRG-5581)")
- fix CrsConversionService potential class cast issues. See MR [!513](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/513) - fix CrsConversionService potential class cast issues. See MR [!513](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/513 "fix CrsConversionService potential class cast issues")
- batch legaltag to size of 25 during ingestion validation, which is the current limit of Legal API. See MR [!511](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/511) - batch legaltag to size of 25 during ingestion validation, which is the current limit of Legal API. See MR [!511](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/511 "batch legaltag to size of 25 during ingestion validation, which is the current limit of Legal API")
- Enabling OPA for AWS and refactor test to wait until OPA cache invalidates. See MR [!499](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/499) - Enabling OPA for AWS and refactor test to wait until OPA cache invalidates. See MR [!499](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/499 "Enabling OPA for AWS and refactor test to wait until OPA cache invalidates")
- Fix permissions for soft delete. See MR [!483](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/483) - Fix permissions for soft delete. See MR [!483](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/483 "Fix permissions for soft delete")
- Fix data inconsistency issue between metadata storage and blob storage. See MR [!451](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/451) - Fix data inconsistency issue between metadata storage and blob storage. See MR [!451](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/451 "Fix data inconsistency issue between metadata storage and blob storage")
- `Register` - `Register`
- Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi. See MR [!243](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/243) - Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi. See MR [!243](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/243 "Upgrade dependencies to fix vulnerabilities and migrate from springfox to openapi")
- remove auth header from hmac challenge request. See MR [!242](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/242) - remove auth header from hmac challenge request. See MR [!242](https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/242 "remove auth header from hmac challenge request")
- `Search` - `Search`
- Search Service API sending out 500 error code for IllegalArgumentException type. See MR [!327](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/327) - Search Service API sending out 500 error code for IllegalArgumentException type. See MR [!327](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/327 "Search Service API sending out 500 error code for IllegalArgumentException type")
- upgrade jackson-databind. See MR [!322](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/322) - upgrade jackson-databind. See MR [!322](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/322 "upgrade jackson-databind")
- `Schema` - `Schema`
- Fix white source vulnerabilities. See MR [!372](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/372) - Fix white source vulnerabilities. See MR [!372](https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/372 "Fix white source vulnerabilities")
### Ingestion services ### Ingestion services
- `Manifest ingestion by reference` - `Manifest ingestion by reference`
- Fix error while DAG run. See issue [#108](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/108) - Fix error while DAG run. See issue [#108](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/108 "Manifest by reference : error while DAG run")
### Data loading utilities ### Data loading utilities
- `Wellbore DDMS Data loader` - `Wellbore DDMS Data loader`
- Fix `NumberOfColumns` requirement for consistency check. See MR [!44](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/44) - Fix `NumberOfColumns` requirement for consistency check. See MR [!44](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/44 "Feature/numberofcolumnsfix")
- Fix module import error. See MR [!48](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/48) - Fix module import error. See MR [!48](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/48 (Resolve "No module named 'wbdutil.wrapper' when calling wbdutil"))
- Fix OsduClient commands. See MR [!49](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/49) - Fix OsduClient commands. See MR [!49](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/merge_requests/49 "fix Osdu clients in commands")
### DDMS ### DDMS
- `Seismic DDMS` - `Seismic DDMS`
- General bug fixes & stability improvement. Refer to [service](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Releases#release-016), [sdapi](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/wikis/Releases#release-016), and [sdutil](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/wikis/Releases#release-016) for detailed notes - General bug fixes & stability improvement. Refer to [service](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Releases#release-016), [sdapi](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/wikis/Releases#release-016), and [sdutil](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/wikis/Releases#release-016) for detailed notes
- `Wellbore DDMS` - `Wellbore DDMS`
- Error response exhaustive list declared on openapi spec. see [!509](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/590). - Error response exhaustive list declared on openapi spec. see [!509](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/590 "add 401 403 404 422 500 response where needed").
- `Reservoir DDMS` - `Reservoir DDMS`
- Fixes for deployment issues - Fixes for deployment issues
- Fixes for multi-partition support, see MR [!44](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/merge_requests/44) - Fixes for multi-partition support, see MR [!44](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/merge_requests/44 "Fix issue #20 data-partition-id header should not come from environment variable when the server calls Entilement service api")
## :exclamation: Breaking Changes ## :exclamation: Breaking Changes
- `WellboreMarkerSet:1.2.1` - `WellboreMarkerSet:1.2.1`
- The original M14 v0.17.0 version contains by accident a breaking change, which is fixed in this patch for [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/d0f3e96bc4e7510364411cb14c3d1e3295fa1324/SchemaRegistrationResources/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json#L115). - The original M14 v0.17.0 version contains by accident a breaking change, which is fixed in this patch for [WellboreMarkerSet:1.2.1](https://community.opengroup.org/osdu/data/data-definitions/-/blob/d0f3e96bc4e7510364411cb14c3d1e3295fa1324/SchemaRegistrationResources/shared-schemas/osdu/work-product-component/WellboreMarkerSet.1.2.1.json#L115).
- `Wellbore DDMS` - `Wellbore DDMS`
- Due to performance degradation, auto computation of the bulk statistic on session commit and post bulk has been disabled. Issue [#50](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/50). Bulk statistic feature is still there but computation will be on-demand only. - Due to performance degradation, auto computation of the bulk statistic on session commit and post bulk has been disabled. Issue [#50](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/50 "Disable bulk statistic auto computation"). Bulk statistic feature is still there but computation will be on-demand only.
- v2 search & V3 fast search APIs fully removed. Deprecated respectively in M9 & M12. issue [#48](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/48). - v2 search & V3 fast search APIs fully removed. Deprecated respectively in M9 & M12. issue [#48](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/48 "Removal of legacy (v2) search & V3 fast search APIs").
### Deprecation Notice ### Deprecation Notice
- DDMS - DDMS
- `Seismic DDMS` - `Seismic DDMS`
- Azure database model changes. Refer [here](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Seismic-DMS-on-Azure-database-model-deprecation) for more info. - Azure database model changes. Refer [here](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Seismic-DMS-on-Azure-database-model-deprecation) for more info.
### Known Issues/Gaps ### Known Issues/Gaps
- Refer [here](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/?label_name%5B%5D=M14) for a list of known pre-shipping testing issues. - Refer [here](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/?label_name%5B%5D=M14) for a list of known pre-shipping testing issues.
## Supporting Information ## Supporting Information
- Preship Testing team's test result [summary](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M14/Planned_Scope_for_M14.xlsx) - Preship Testing team's test result [summary](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M14/Planned_Scope_for_M14.xlsx)
......