|
|
M7 Release Notes |
|
|
\ No newline at end of file |
|
|
# Milestone 7 - August 20, 2021 (Released)
|
|
|
|
|
|
For detailed services that are included in this release, please refer to M7 tagging notes:
|
|
|
|
|
|
* [R3 Milestone 7 deliverables - 0.10.0](https://community.opengroup.org/osdu/governance/project-management-committee/-/wikis/Release-0.10-Tagging-Notes)
|
|
|
|
|
|
## New Features
|
|
|
|
|
|
* `Data Definitions` To accommodate Wellbore DDMS requirements, WellLog and WellboreTrajectory models have been enhanced. A data model for Activity was added to support Seismic domain's requirement. Take a glance at the exhaustive list of modifications on its [snapshot](https://community.opengroup.org/osdu/data/data-definitions/-/blob/master/E-R/ChangeReport.md#snapshot-2021-04-29).
|
|
|
* `Core Services`
|
|
|
* [Version info.](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/107) endpoint been added partially; only in Storage Service.
|
|
|
* `Entitlement V2` Updated [Entitlement service](https://community.opengroup.org/osdu/platform/security-and-compliance) is now available on all cloud platforms
|
|
|
* `Ingestion`
|
|
|
* `CSV Parser Enhancements`:
|
|
|
* Search Client escapes special characters. [More Details](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/20)
|
|
|
* [Nested Schema](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/23) Supporting data ingestion into nested attributes and utilizing the metadata file's delimiter characters.
|
|
|
* [Spatial Data Handler](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/22)
|
|
|
* [Tokens for long running jobs](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/21) An interface `AuthJwtToken` is added for generating tokens. Regarding implementations are `AwsServiceAccountAuthToken`, `ServiceAccountAuthToken`, and `IBMServicePrincipalAuthToken` classes which needs actual implementation.
|
|
|
* [Multi-thread Optimization](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/26) Every record being read and added as a task to an executor service, where it is enriched and stored with other records Parallelly.
|
|
|
* [ID Generation Standard](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/25) ID is being generated based on OSDU standard now ( `<authority/data-partition-id>:<source>:<entity-type>:<base64-of-xosdu-natural-keys>`).
|
|
|
* [Relationship Enhancement](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/issues/24) In the file metadata record, the `EhxtensionProperties` block now can be used to offer extra details for ingestion and to provide relationship(Schema/Data) information.
|
|
|
* `Manifest-based Ingestion` Performance improvements by adding cache for Search ID handler. [Find out more](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/62)
|
|
|
* `WITSML Ingestion` is supported on `AWS` `IBM`
|
|
|
* `Python SDK`
|
|
|
* Supporting Entitlement v2
|
|
|
* Entitlement CLI is added
|
|
|
* Some refactoring [See more](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/issues/73)
|
|
|
* `Seismic Ingestion` [OpenVDS](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds) and [OpenZGY](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-zgy) format conversion features are now available
|
|
|
* OpenVDS supported on `AWS` `GCP` `IBM`
|
|
|
* OpenZGY supported on `GCP` `IBM` `Azure`
|
|
|
* `E&O` Improvement on the experimental Entitlements & Obligations feature: Default policies bootstrapped for ensuring [Policy Service](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Policy-service) operational readiness to guarantee consistent current behavior. [Find out more](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/10)
|
|
|
|
|
|
## Bug Fixes
|
|
|
|
|
|
* `Ingestion`
|
|
|
* When calling `Workflow API`, User receives 401/403 as a valid HTTP code if s/he is not a valid User or does not have any Entitlements Groups. [See More Here](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/95)
|
|
|
* New version of `WITSML` parser does not pass date-time validation. Now, date-time fields must follow RFC3339 standard. more information [here ](https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-dags/-/merge_requests/50#updates-description)and [here](https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics-osdu-integration/-/issues/39).
|
|
|
* `Security` Known Vulnerabilities in 3rd Party Dependent Libraries resolved. [See More Here](https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/44)
|
|
|
|
|
|
## Breaking Changes
|
|
|
|
|
|
### Deprecation Notice
|
|
|
|
|
|
`Storage` \
|
|
|
Given that Storage Service Schema Management does not support R3 schemas, we should avoid employing two services to facilitate schema management since the majority of users already use the Schema Service. As a result, we propose phasing out Storage Service Schema Management as soon as possible. \
|
|
|
\
|
|
|
As a _NOTICE_, [some Storage Schema APIs](https://community.opengroup.org/osdu/platform/system/storage/-/issues/62) have been marked as _deprecated_ and will be _eliminated_ from the M8 release.
|
|
|
|
|
|
## Known Issues
|
|
|
|
|
|
* `CSV Parser` is not available for AWS as for now due to the [issue of loading reference Data](https://community.opengroup.org/osdu/platform/testing/-/issues/23).
|
|
|
* New `Policy Service` is an experimental feature, meant to be deployed in a dedicated test partition for evaluation and community feedback. It is not recommended to be enabled in production environments as there is a known impact in manifest-based Ingestion (integrity check)
|
|
|
|
|
|
# |
|
|
\ No newline at end of file |