Skip to content

Changes

Page history
Update M13 Release Notes authored by Chad Leong's avatar Chad Leong
Hide whitespace changes
Inline Side-by-side
M13-Release-Notes.md
View page @ 4a59b62f
# Milestone 13 # Milestone 13
Released on 9-Sep-2022 Released on 9-September-2022.
For detailed services that are included in this release, please refer to [M13 tagging notes](https://community.opengroup.org/osdu/governance/project-management-committee/-/wikis/Release-0.16-Tagging-Notes) For detailed services that are included in this release, please refer to [M13 tagging notes](https://community.opengroup.org/osdu/governance/project-management-committee/-/wikis/Release-0.16-Tagging-Notes).
[[_TOC_]]
In keeping with our policy of releasing new capabilities early and often for feedback, we are pleased to introduce several new **experimental** platform capabilities in this milestone release. These are currently only available in specific cloud service providers. See details below.
- `Reservoir DDMS`
- `External Data Services`
- `Manifest ingestion by reference`
## New Features ## New Features
...@@ -28,48 +36,57 @@ For detailed services that are included in this release, please refer to [M13 ta ...@@ -28,48 +36,57 @@ For detailed services that are included in this release, please refer to [M13 ta
- _Updated_ [_Glossary_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/Chapters/91-Glossary.md) - _Updated_ [_Glossary_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/Chapters/91-Glossary.md)
- _Extended_ [_Schema Usage Guide_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/README.md)_, specifically a new section about_ [_OSDU Schemas_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/Chapters/93-OSDU-Schemas.md) - _Extended_ [_Schema Usage Guide_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/README.md)_, specifically a new section about_ [_OSDU Schemas_](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.16.0/Guides/Chapters/93-OSDU-Schemas.md)
- `Core and Helper Services` - `Core and Helper Services`
* `Storage` - `OS core common`
* `File` - Added data hash and recordBlocks fields as per [ADR 92](https://community.opengroup.org/osdu/platform/system/home/-/issues/92)
* `Search` - Added OPA translation integration to allow new endpoints. See MR [!161](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/161)
* `Indexer Queue` - `OS core lib Azure`
* `E&O` - Added undelete operation to enable restoring soft-deleted blobs. See MR [!216](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/216)
* `Apache SIS` - `File`
* Updated the Apache SIS library to version 1.2. This release includes quite a lot of bug fixes as well as a number of improvements. The most notable features are - `IBM` Now supports file checksum implementation. See MR [!314](https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/314)
* Read World Files and ESRI ASCII grid and BIL/BIP/BSQ rasters - `Dataset`
* Support for Lambert Azimuthal Equal Area projections - Added support for External Data Services. See MR [!218](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/218)
* Support ESRI WKT element GEOGTRAN - `Search`
* More features and descriptions can be found at the [library release notes](https://sis.apache.org/release-notes/1.2.html) - Added support for multi-polygon intersection query. See MR [!301](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/301)
* `CRS Conversion` - Optimized queries for data root user. See MR [!298](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/298)
* The Convert service can now handle Operator defined Transformations between geodetic datums, i.e., a PersistableReference using WKT using GEOGTRAN. Previously the EPSG code was extracted from the PR and used. If that code did not exist in the Apache SIS library then no transformation would be executed. Now the WKT is parsed and used if the given code cannot be found in the Apache SIS library. - `Indexer`
- Provided indexer resiliency updates if aliased index co-exist. See MR [!345](https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/345)
- Added error exception message to allow troubleshooting. See issue [#67](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/67)
- `Apache SIS`
- Updated the Apache SIS library to version 1.2. This release includes quite a lot of bug fixes as well as a number of improvements. The most notable features are
- Read World Files and ESRI ASCII grid and BIL/BIP/BSQ rasters
- Support for Lambert Azimuthal Equal Area projections
- Support ESRI WKT element GEOGTRAN
- More features and descriptions can be found at the [library release notes](https://sis.apache.org/release-notes/1.2.html)
- `CRS Conversion v3`
- The Convert service can now handle Operator defined Transformations between geodetic datums, i.e., a PersistableReference using WKT using GEOGTRAN. Previously the EPSG code was extracted from the PR and used. If that code did not exist in the Apache SIS library then no transformation would be executed. Now the WKT is parsed and used if the given code cannot be found in the Apache SIS library.
See [Issue #46 Converter Service](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/46). See [Issue #46 Converter Service](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/46).
* The EpsgManifestGenerator script has been updated to dynamically generate WKT (PersistableReferences) for CRS and CT entities not in the Esri lookup files. - The EpsgManifestGenerator script has been updated to dynamically generate WKT (PersistableReferences) for CRS and CT entities not in the Esri lookup files.
- `CRS Catalog v3`
* `CRS Catalog` - Improve v3 Search endpoints. See MR [!169](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/169)
* **POST CRS and POST CT** - **POST CRS and POST CT**
* Change default search limit to a large value (10000) - Change default search limit to a large value (10000)
* Add includeDeprecated (exclude InactiveIndicator: true) - Add includeDeprecated (exclude InactiveIndicator: true)
* Add field “returnAllFields” - Add field “returnAllFields”
* Change the default return fields - Change the default return fields
* Make schema authority configurable with an env var “SCHEMA_AUTHORITY” - Make schema authority configurable with an env var “SCHEMA_AUTHORITY”
* **POST CRS** - **POST CRS**
* Correct the handling of returnBoundProjectedAndProjectedBasedOnWgs84 and returnBoundGeographic2DAndWgs84 - Correct the handling of returnBoundProjectedAndProjectedBasedOnWgs84 and returnBoundGeographic2DAndWgs84
* Change extent.name to extent.description (the reason it was not working before was because extent.description was not in initial version of schema) - Change extent.name to extent.description (the reason it was not working before was because extent.description was not in initial version of schema)
* **POST CT** - **POST CT**
* Make source and target crs interchangable - Make source and target crs interchangable
* Add id and code - Add id and code
* Remove "includeHorizontal" and "includeVertical" and add kind instead. Add "All" as a magical keyword. Use “excludeVertical” as default. - Remove "includeHorizontal" and "includeVertical" and add kind instead. Add "All" as a magical keyword. Use “excludeVertical” as default.
- Updated Swagger docs. See MR [!154](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/154/)
- `Ingestion Services` - `Ingestion Services`
* `Manifest Ingestion` - `Manifest Ingestion by Reference`
* `Manifest Ingestion by Reference` - Available in GCP only. (other CSPs to follow later)
* Available in GCP only. (other CSPs to follow later) - _Ingestion By Reference_ feature stores only the `Record_Id` of a manifest into the `XCOM` context and access it from the different DAGS/operators. Ingestion can now be triggered by a `record_id` (of a manifest uploaded on the DatasetService) instead of a complete manifest. This can eventually enhance the performance. Read more in [MR](https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.opengroup.org%2Fosdu%2Fplatform%2Fdata-flow%2Fingestion%2Fosdu-airflow-lib%2F-%2Fmerge_requests%2F17&data=05%7C01%7Cmohammad.malekmakan%40petronas.com%7C53829753901f4dae1bb208da78778108%7C3b2e894179484131978ab2dfc7295091%7C0%7C0%7C637954751953159322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yXNuEIh43iHhHPq5PCga7SkQp7WNV0hC3GBjOtqRXRo%3D&reserved=0)or see the [Postman collection](https://community.opengroup.org/osdu/platform/testing/-/tree/master/Dev/Manifest_By_Reference).
* _Ingestion By Reference_ feature <span dir="">stores only the `Record_Id` of a manifest into the `XCOM` context and access it from the different DAGS/operators. Ingestion can now be triggered by a `record_id` (of a manifest uploaded on the DatasetService) instead of a complete manifest. This can eventually enhance the performance</span>. Read more in [MR ](https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.opengroup.org%2Fosdu%2Fplatform%2Fdata-flow%2Fingestion%2Fosdu-airflow-lib%2F-%2Fmerge_requests%2F17&data=05%7C01%7Cmohammad.malekmakan%40petronas.com%7C53829753901f4dae1bb208da78778108%7C3b2e894179484131978ab2dfc7295091%7C0%7C0%7C637954751953159322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yXNuEIh43iHhHPq5PCga7SkQp7WNV0hC3GBjOtqRXRo%3D&reserved=0)or see the [Postman collection](https://community.opengroup.org/osdu/platform/testing/-/tree/master/Dev/Manifest_By_Reference). - `SegY to OpenVDS conversion DAG`
* `CSV Ingestion` - Byte location is now supported from Dataset (FileCollection.SegY) provided. See issue [#12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/issues/12)
* `WITSML Parser` - `Azure` Now supports SDSTORE similar to SegY to OpenZgy conversion process. See details in this issue [#13](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/issues/13)
* `SegY to OpenVDS conversion`
* Now working inside SD STORE similar to SegY to OpenZgy conversion process. See details in this [issue](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-vds-conversion/-/issues/13).
- `DDMS` - `DDMS`
- `Seismic DDMS` - `Seismic DDMS`
- [Implementation](/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/420 "feat: SDDMS-V4 implementation") for [\[ADR\] Domain API](/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/32 "[ADR] Domain API") - [Implementation](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/420) for [[ADR] Domain API](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/32)
- `Reservoir DDMS` - `Reservoir DDMS`
- [Link to repository](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir) - [Link to repository](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir)
- [Full release notes](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/home/-/blob/main/ReleaseNotes.md) - [Full release notes](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/home/-/blob/main/ReleaseNotes.md)
...@@ -80,111 +97,117 @@ See [Issue #46 Converter Service](https://community.opengroup.org/osdu/platform/ ...@@ -80,111 +97,117 @@ See [Issue #46 Converter Service](https://community.opengroup.org/osdu/platform/
- Implemented in Azure only (other CSPs to follow later) - Implemented in Azure only (other CSPs to follow later)
- `Wellbore DDMS` - `Wellbore DDMS`
- Issue [#43](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/43), adding support of: - Issue [#43](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/43), adding support of:
- [well schema 1.1.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/master-data/Well.1.1.0.md) - [Well 1.1.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/master-data/Well.1.1.0.md)
- [Wellbore 1.1.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/master-data/Wellbore.1.1.0.md) - [Wellbore 1.1.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/master-data/Wellbore.1.1.0.md)
- [WellboreMarkerSet 1.2.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/work-product-component/WellboreMarkerSet.1.2.0.md) - [WellboreMarkerSet 1.2.0](https://community.opengroup.org/osdu/data/data-definitions/-/blob/v0.15.0/E-R/work-product-component/WellboreMarkerSet.1.2.0.md)
- Marker documentation added, see [!571](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/571) - Marker documentation added, see [!571](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/571)
- `Audit and Metric` - `External Data Services (EDS)`
- `EDS`(External Data Services) - The very first release of EDS. Currently available in Azure environment only. (other CSPs to follow later)
* The very first release of EDS. - EDS allows connection to the data providers (OSDU external source environment), fetch the data and ingest it into the operator OSDU environment.
* Available in Azure environment only. (other CSPs to follow later) - This release offers two capabilities.
* EDS allows connection to the data providers (OSDU external source environment), fetch the data and ingest it into the operator OSDU environment. - **Registration Process** - This functionality enables the registration (persistence) of configuration information needed for automated services to orchestrate Fetch and Ingestion workflows between a consuming OSDU Data Platform and OSDU compliant data sources.
* This release offers two capabilities.
- **Registration Process** - This functionality enables the registration (persistence) of configuration information needed for automated services to orchestrate Fetch and Ingestion workflows between a consuming OSDU Data Platform and OSDU compliant data sources.
- **Fetch and Ingest** (External Data Workflow Services) - Automated services that consume the Registration Process (also called Source Registry configuration) to handle the Fetch and Ingest orchestration. - **Fetch and Ingest** (External Data Workflow Services) - Automated services that consume the Registration Process (also called Source Registry configuration) to handle the Fetch and Ingest orchestration.
* EDS DMS feature (for obtaining bulk data files "on demand") is deferred till next release. - EDS DMS feature (for obtaining bulk data files "on demand") is deferred till next release.
* Associated DAGS deployed - eds_ingest and eds_scheduler - Associated DAGS deployed - eds_ingest and eds_scheduler
* New schema entities to support EDS - New schema entities to support EDS
- Master - ConnectedSourceRegistryEntry:1.0.0, ConnectedSourceDataJob:1.0.0 - Master - ConnectedSourceRegistryEntry:1.0.0, ConnectedSourceDataJob:1.0.0
- Reference - SecuritySchemeType:1.0.0, OAuth2FlowType:1.0.0 - Reference - SecuritySchemeType:1.0.0, OAuth2FlowType:1.0.0
- `GCZ (Geospatial Consumption Zone)`
- `GCZ`(Geospatial Consumption Zone) - The very first release of GCZ. Only available from code base. Not yet implemented by any CSPs.
* The very first release of GCZ. Only available from code base. Not yet implemented by any CSPs. - Implements transformer, consumer, map integration for common mappable objects.
* Implements transformer, consumer, map integration for common mappable objects. - Please refer to [GCZ overview](https://community.opengroup.org/osdu/platform/consumption/geospatial/-/blob/master/osdu_gcz_wiki/README.md) for details.
* Please refer to [GCZ overview](https://community.opengroup.org/osdu/platform/consumption/geospatial/-/blob/master/osdu_gcz_wiki/README.md) for details.
## Bug Fixes ## Bug Fixes
- `Core and Helper Services` - `Core and Helper Services`
* `Storage` - `Storage`
* `File` - Fixed error message using response from OPA. See issue [#133](https://community.opengroup.org/osdu/platform/system/storage/-/issues/133)
* `Search` - Fixed OPA endpoint config. See MR [!426](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/426)
* `Indexer Queue` - `File`
* `CRS Conversion V3` - Fixed issue on the `PreloadFilePath` attribute. See issue [#82](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/issues/82)
- `Ingestion services` - `Azure` Fixed slow requests to POST metadata. See issue [#69](https://community.opengroup.org/osdu/platform/system/file/-/issues/69)
- `Notification`
- `Azure` Fixed issue on notification error messages. See issue [#44](https://community.opengroup.org/osdu/platform/system/notification/-/issues/44)
- `Dataset`
- Fixed issue on the `PreloadFilePath` attribute. See issue [#82](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/issues/82)
- Fixed internal server error messaging response. See MR [!214](https://community.opengroup.org/osdu/platform/system/dataset/-/merge_requests/214)
- `Partition`
- Fixed partition property id. See MR [!212](https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/212)
- `Search`
- Removed non-functional and deprecated CCS endpoint from Search service. See MR [!302](https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/302)
- `Indexer`
- Fixed mapping for text array attribute inside nested array object. See issue [#72](https://community.opengroup.org/osdu/platform/system/indexer-service/-/issues/72)
- `CRS Conversion V3`
- Fixed parsing of WKTS with file parameters. See issue [#60](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/60)
- `DDMS` - `DDMS`
* _Seismic DDMS_ - _Seismic DDMS_
* Fixes related to list API in Azure - Fixes related to list API in Azure
* General bug fixes & stability improvement. Refer to [service](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Releases#release-016), [sdapi](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/wikis/Releases#release-016), and [sdutil](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/wikis/Releases#release-016) for detailed notes - General bug fixes & stability improvement. Refer to [service](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Releases#release-016), [sdapi](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/wikis/Releases#release-016), and [sdutil](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-sdutil/-/wikis/Releases#release-016) for detailed notes
* _Wellbore DDMS_ - _Wellbore DDMS_
* Security fix, upgrade pyJWT library. See issue [#43](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/43) - Security fix, upgrade pyJWT library. See issue [#43](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/43)
* Documentation fixes and clean up. - Documentation fixes and clean up.
* Log recognition APIs mounted as other APIs. - Log recognition APIs mounted as other APIs.
* no longer retry on calls to storage in case of failure or timeout. see [!585](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/585). - no longer retry on calls to storage in case of failure or timeout. see [!585](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/585).
* Azure pipeline: Migrate python image to use MCR, see [!587](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/587). - Azure pipeline: Migrate python image to use MCR, see [!587](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/587).
* Fixes in [well log consistency validation](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/wikis/WellLog-consistency-rules#rules-1): [#46](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/46), [#47](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/47) - Fixes in [well log consistency validation](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/wikis/WellLog-consistency-rules#rules-1): [#46](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/46), [#47](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/47)
- `Open Test Data` - `Open Test Data`
* Fixes in different modules to align the code to the data schema. This fixes the GetretrievalUrl method in the Dataset module for the TNO/Volve dataset - see [#82](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/issues/82) - Fixes in different modules to align the code to the data schema. This fixes the GetretrievalUrl method in the Dataset module for the TNO/Volve dataset - see [#82](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/issues/82)
- Various security fixes
## Supporting Information ## Supporting Information
### Test results from Preship team
- Overall summary can be found [here](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M13/Planned_Scope_for_M13.xlsx)
- Folder with documented steps and test results - [here](https://community.opengroup.org/osdu/platform/pre-shipping/-/tree/main/R3-M13/Test_Plan_Results_M13)
### Postman Collections ### Postman Collections
- [Ingestion by Reference ](https://community.opengroup.org/osdu/platform/testing/-/tree/master/Dev/Manifest_By_Reference)
- [Ingestion by Reference](https://community.opengroup.org/osdu/platform/testing/-/tree/master/Dev/Manifest_By_Reference)
- [CRS Catalog V3](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/blob/master/docs/CRS%20Catalog%20Service%20API%20V3.postman_collection.json) - [CRS Catalog V3](https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/blob/master/docs/CRS%20Catalog%20Service%20API%20V3.postman_collection.json)
- [CRS Conversion V3](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/blob/master/docs/v3/CRS%20Conversion%20Service%20API%20v3.postman_collection.json) - [CRS Conversion V3](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/blob/master/docs/v3/CRS%20Conversion%20Service%20API%20v3.postman_collection.json)
### Load Manifests (JSON) ### Load Manifests (JSON)
- [Added Seismic 4D to worked examples](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/Examples/WorkedExamples/Seismic4D) - [Added Seismic 4D to worked examples](https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/tree/master/Examples/WorkedExamples/Seismic4D)
- [Updated Wellbore Trajectories for TNO](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/TNO/work-products/trajectories_1_1_0) - [Updated Wellbore Trajectories for TNO](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/TNO/work-products/trajectories_1_1_0)
- [Updated Wellbore Trajectories for Volve ](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/Volve/work-products/trajectories_1_1_0) - [Updated Wellbore Trajectories for Volve](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/Volve/work-products/trajectories_1_1_0)
- [Vertical Seismic Profile added for NOPIMS](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/NOPIMS) - [Vertical Seismic Profile added for NOPIMS](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/NOPIMS)
- [Seismic 2D traces and navigation added for Volve](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/Volve/work-products/seismics_1_2_0) - [Seismic 2D traces and navigation added for Volve](https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/tree/M13/rc--3.0.0/4-instances/Volve/work-products/seismics_1_2_0)
## Breaking Changes ## :heavy_exclamation_mark: Breaking Changes
### Deprecation Notice ### Deprecation Notice
- DDMS - DDMS
- `Seismic DDMS` - `Seismic DDMS`
- [Azure database model](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Seismic-DMS-on-Azure-database-model-deprecation) - Azure database model changes. Refer [here](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/wikis/Seismic-DMS-on-Azure-database-model-deprecation) for more info.
### Known Issues/Gaps
## Known Issues/Gaps
- `Core and Helper Services` - `Core and Helper Services`
* `Storage` - `Search`
* `File` - A search for UOM by record id using query “osdu:reference-data--UnitOfMeasure:ft:” returns ft[US] and ft[GC] units. A query for "osdu:reference-data--UnitOfMeasure:ft[US]" does not work and neither does "unitZ": "osdu:reference-data--UnitOfMeasure:ft%5BUS%5D"
* `Search` - `CRS Conversion V3`
* Known issue: a search for UOM by record id using query “osdu:reference-data--UnitOfMeasure:ft:” returns ft[US] and ft[GC] units. A query for "osdu:reference-data--UnitOfMeasure:ft[US]" does not work and neither does "unitZ": "osdu:reference-data--UnitOfMeasure:ft%5BUS%5D" - `ConvertTrajectory`
* `Indexer Queue` - Incorrectly adds meters to feet and feet to meters if the unit of the projected CRS is in the one and the Measured Depths are in the other. See issue [#52](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/52).
* `CRS Conversion V3` - Gap: there is no option to compute a wellpath without applying a correction for map projection point scale factor. See issue [#29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29)
* `ConvertTrajectory` - All CoordinateTransformations stored in OSDU must be “to WGS 84”. It is not guaranteed that the generated reference data satisfies this because it is not checked. Geodetic reference data loaded to OSDU are OK, and in general transformations are 99.99% defined as such in the EPSG Database, but EpsgManifestGenerator does not check for this. See issue [#44](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/44)
* Known issue: incorrectly adds meters to feet and feet to meters if the unit of the projected CRS is in the one and the Measured Depths are in the other. - Convert does not return continuous Eastings when converting latitude, longitude to a local projected CRS that spans the antimeridian (the 180d meridian), e.g., for a seismic line from 179E to 179W. This has no impact unless such local projection would be used (which is uncommon). See issue [#49](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/49).
See [issue 52](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/52). - Convert custom WKT for a grid file Transformation expects parameter name expected by SIS is "Latitude and longitude difference file" for NTv2 method (and similar for NADCON and vertical models). EpsgManifestGenerator and the underlying database do not typically contain the correct file names. Hence, any custom WKT for transformations that are not implicitly defined by the EPSG code and stored in the Apache SIS library will not work. This is a low priority/low value, because in general Operators will not create NTv2 custom files (there is a small risk a new transform is defined by EPSG that is intended to be used, but before SIS is updated). See [issue 39](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/39)
* Gap: there is no option to compute a wellpath without applying a correction for map projection point scale factor. See [issue 29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29) - `convertTrajectory` does not allow interpolation on a specific MD (which is needed to get the TVD for a log at a given MD that is not in the survey). See [issue 29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29)
* Known issue: All CoordinateTransformations stored in OSDU must be “to WGS 84”. It is not guaranteed that the generated reference data satisfies this because it is not checked. Geodetic reference data loaded to OSDU are OK, and in general transformations are 99.99% defined as such in the EPSG Database, but EpsgManifestGenerator does not check for this. See [issue 44](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/44) - `CRS Convert` has no endpoint for Bin Grid squaring, deriving p6 parameters, and conversion. Such checks (if the grid is square, and derive the parameters from the corners) should be part of the seismic ingest workflow which then would need such endpoint to be developed. See [issue 27](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/27)
* Known issue: Convert does not return continuous Eastings when converting latitude, longitude to a local projected CRS that spans the antimeridian (the 180d meridian), e.g., for a seismic line from 179E to 179W. This has no impact unless such local projection would be used (which is uncommon). - `CRS Convert` has no endpoint that directly returns the point scale factor and grid convergence at a given location. `ConvertTrajectory` can be called with a dummy call to obtain these. See [issue 29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29)
See [Issue #49 Converter Service](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/49). - `CRS Convert` cannot execute a direct transformation between CRS A and CRS B, it requires a pivot (through WGS 84).
* Known issue: Convert custom WKT for a grid file Transformation expects parameter name expected by SIS is "Latitude and longitude difference file" for NTv2 method (and similar for NADCON and vertical models). EpsgManifestGenerator and the underlying database do not typically contain the correct file names. Hence, any custom WKT for transformations that are not implicitly defined by the EPSG code and stored in the Apache SIS library will not work. This is a low priority/low value, because in general Operators will not create NTv2 custom files (there is a small risk a new transform is defined by EPSG that is intended to be used, but before SIS is updated). See [issue 39](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/39)
* Gap: `convertTrajectory` does not allow interpolation on a specific MD (which is needed to get the TVD for a log at a given MD that is not in the survey). See [issue 29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29)
* Gap: `CRS Convert` has no endpoint for Bin Grid squaring, deriving p6 parameters, and conversion. Such checks (if the grid is square, and derive the parameters from the corners) should be part of the seismic ingest workflow which then would need such endpoint to be developed. See [issue 27](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/27)
* Gap: `CRS Convert` has no endpoint that directly returns the point scale factor and grid convergence at a given location. `ConvertTrajectory` can be called with a dummy call to obtain these. See [issue 29](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/29)
* Gap: `CRS Convert` cannot execute a direct transformation between CRS A and CRS B, it requires a pivot (through WGS 84).
See [Issue #15 Converter Service](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/15). Also see [issue 34](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/34) See [Issue #15 Converter Service](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/15). Also see [issue 34](https://gitlab.opengroup.org/osdu/subcommittees/ea/projects/geomatics/home/-/issues/34)
- `Data Flow Services` - `Data Flow Services`
* `Manifest based Ingestion` - `Manifest based Ingestion`
* Known issue: AWS - 503 response when ingesting 50k records - see [issue 356](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/356) - `AWS` - 503 response when ingesting 50k records. See issue [#356](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/356)
* `Manifest Ingestion by Reference` - `Azure` - 413 response when ingesting 50k records. See issue [#362](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/362)
* `CSV Ingestion` - `IBM` - 504 response when ingesting 50k records. See issue [#357](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/357)
* `WITSML Parser` - `GCP` - 502 response when ingesting 50k records. See issue [#365](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/365)
* [Failure when schema-authority is not the same as data partition ID](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/347) - `WITSML Parser`
* [Reference data SchemaFormatType - need to add manually EnergisticsWITSML](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/62) - Failure when schema-authority is not the same as data partition ID. See issue [#347](https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/347)
* [Tubular issue - need to pre-create parent record manually](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/58 - Reference data SchemaFormatType - need to add manually EnergisticsWITSML. See issue [#62](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/62)
- Tubular issue - need to pre-create parent record manually. See issue [#58](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/58
) )
* [Problem with Trajectory data type - recent change for schema 1.1.0](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/61 - Problem with Trajectory data type - recent change for schema 1.1.0. See issue [#61](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics/witsml-parser/-/issues/61
) )
* `LAS loader for Wellbore DDMS` - `LAS loader for Wellbore DDMS`
* Known issue: Broken due to change in schema structure. [See issue.](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/issues/53) - Broken due to change in schema structure. See issue [#53](https://community.opengroup.org/osdu/platform/data-flow/data-loading/wellbore-ddms-data-loader/-/issues/53)
\ No newline at end of file