|
|
|
M8 Release Notes |
|
|
\ No newline at end of file |
|
|
|
# Milestone 8 - September 30, 2021
|
|
|
|
|
|
|
|
For detailed services that are included in this release, please refer to M8 tagging notes:
|
|
|
|
|
|
|
|
* [R3 Milestone 8 deliverables - 0.11.0](https://community.opengroup.org/osdu/governance/project-management-committee/-/wikis/Release-0.11-Tagging-Notes)
|
|
|
|
|
|
|
|
## New Features
|
|
|
|
|
|
|
|
* `Data Definition`
|
|
|
|
* None major for this milestone. See some information in "Deprecated" section below about "Collection" entity.
|
|
|
|
* `Core Services`
|
|
|
|
* `Storage`
|
|
|
|
* `SpatialLocation` supports `WGS84` now. [More information](https://community.opengroup.org/osdu/platform/system/storage/-/issues/82)
|
|
|
|
* `Search`
|
|
|
|
* `Audit Trail` attributes (ex: createUser, createTime, modifyUser, modifyTime) are now available from Search by enabling these for Index. [More information](https://community.opengroup.org/osdu/platform/system/search-service/-/issues/59)
|
|
|
|
* Version info. endpoint is now available for all the core services. [More information](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/47)
|
|
|
|
* `Python SDK`
|
|
|
|
* Performance improvement to send requests to Search services by batches
|
|
|
|
* Base Client is refactored based on this [ADR](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/issues/5) which has the implementation [here](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/merge_requests/30)
|
|
|
|
* SDK Configuration can be done now via `python-dotenv` without using `open_api.ini`. [More information](https://community.opengroup.org/osdu/platform/system/sdks/common-python-sdk/-/issues/3)
|
|
|
|
* `Workflow Services`
|
|
|
|
* Airflow 2.0 backward compatibility introduces possibility to use the same Manifest Based Ingestion code base both for Airflow >=1.10.10 and Airflow 2.0. [More information](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/issues/4)
|
|
|
|
* Started movement of OSDU airflow common code to [https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib) project.
|
|
|
|
* `Data Ingestion`
|
|
|
|
* `Manifest Based Ingestion`
|
|
|
|
* An implementation for splitting Large Manifest into smaller chunks has taken in place which allows parallel ingestion. [More information](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/63)
|
|
|
|
* External Data Framework had a refactoring in `validate_referential_integrity.py` for accepting a whitelist of non-validated properties which allows a configuration-driven approach for excluding validation of those properties that match OSDU ID patterns. This is tested on `GCP` Community Env.
|
|
|
|
* `CSV Parser Enhancements`
|
|
|
|
* CSV ingestor is able to generate OSDU compliance records including Arrays and nested Arrays.
|
|
|
|
* The support of OSDU Governance and Schema constructs plus Airflow 2.0 support is finalized
|
|
|
|
* `WITSML Parser`
|
|
|
|
* No new changes
|
|
|
|
* `Core` Major improvement in Integration Testing coverage has been done to ensure the consistency of behavior across all the CSPs. More details can be found [here](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/124) and [here](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/131). Tests are all pass for `Azure`,`GCP` and `IBM`.
|
|
|
|
* `DDMS`
|
|
|
|
* `Wellbore`
|
|
|
|
* Data Access API’s
|
|
|
|
* Support OSDU entities: Well 1.0.0, Wellbore 1.0.0 , WellLog 1.1.0, WellboreTrajectory 1.1.0, WellboreMarkerSet 1.0.0
|
|
|
|
* API’s to access bulk data (some CSPs only)
|
|
|
|
* Log recognition service
|
|
|
|
* `Seismic`
|
|
|
|
* [OpenVDS](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds) and [OpenZGY](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy) format conversion features are now available on all CSPs.
|
|
|
|
* `DAGs` There is an update for SEGY -> OpenVDS Parser to support Airflow 2+. This is a core logic update which impacted on CSPs implementations [Read more](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/78)
|
|
|
|
* `DAGs` There is an update for SEGY -> OpenZgy Converter to support Airflow 2+. Added • support for AWS (fixed region env var)
|
|
|
|
* CSP agnostic impersonation token
|
|
|
|
* updated dataset list endpoint to return paginated results if limit and cursor query parameters are present
|
|
|
|
* set the subproject and dataset creator using the subid/sub value from the payload jwt
|
|
|
|
|
|
|
|
## Bug Fixes
|
|
|
|
|
|
|
|
* `E&O`
|
|
|
|
* Wrong CURL command in Swagger of DELETE API is fixed. [see the issue here](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/issues/75)
|
|
|
|
* `Legal Service`
|
|
|
|
* Problem with Partition Service is fixed and you may see more details [here](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/issues/13)
|
|
|
|
* `Reference & Helper Service`
|
|
|
|
* `CRS Conversions` 5XX error is fixed upon receiving a bad request by adding a [new logger](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/61). Read more [here](https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/issues/22)
|
|
|
|
* `Unit` Logger is fixed which was causing 500 error in `unit-service`. See more in [Unit-Service-Availability Fast Burn Alert](https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/issues/23) issue.
|
|
|
|
* `Seismic DDMS`
|
|
|
|
* misc. security vulnerabilities fixes
|
|
|
|
* misc. performance / scalability improvements
|
|
|
|
* misc. observability improvements
|
|
|
|
* `Manifest Based Ingestion`
|
|
|
|
* The “frame of reference” handling issue - (Unit of Measure) by Manifest based Ingestion is [fixed](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/77).
|
|
|
|
|
|
|
|
## Breaking Changes
|
|
|
|
|
|
|
|
### Deprecation Notice
|
|
|
|
|
|
|
|
* `Storage Service` Referring to [Deprecation Notice in M7](https://community.opengroup.org/osdu/governance/project-management-committee/-/wikis/Release-Notes#deprecation-notice) `Schema Management` has been deprecated in this release.
|
|
|
|
* `master-data--Collection` will be deprecated (removed from the deployment, but no longer registered); the next snapshot will come with `work-product-component--PersistedCollection` instead. The next snapshot change report [can be previewed here](https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.opengroup.org%2Fosdu%2Fdata%2Fdata-definitions%2F-%2Fblob%2F21-sync-with-dd%2FE-R%2FChangeReport.md&data=04%7C01%7Cmohammad.malekmakan%40petronas.com.my%7C07fb9194feba4d20643308d97692f2b7%7C3b2e894179484131978ab2dfc7295091%7C0%7C0%7C637671195790082522%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=68ochpWLwgrqReeUuYVaLbQZ9AEjYGqYZCTQarwiWdA%3D&reserved=0).
|
|
|
|
|
|
|
|
## Known Issues
|
|
|
|
|
|
|
|
* `E&O` is an experimental feature, meant to be deployed in a dedicated test partition for evaluation and community feedback. It is not recommended to be enabled in production environments as there is a known impact in manifest-based Ingestion (integrity check)
|
|
|
|
* CSV Ingestion is not available in `AWS`.
|
|
|
|
* WITSML Parser is not available in `Azure`.
|
|
|
|
* WITSML Parser - works for Well, Wellbore Master data types. But has issues for work-product component types Log, Trajectory, Marker, Tubular.
|
|
|
|
* Wellbore DDMS Bulk Access API is not available in `AWS`. |
|
|
\ No newline at end of file |
