... | ... | @@ -79,6 +79,9 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
|
|
|
### Ingestion services
|
|
|
|
|
|
- Manifest-based Ingestion - Major performance improvement.
|
|
|
- osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30)
|
|
|
- osdu-airflow-lib - Added new Airflow variable core__ingestion__thread_save_number to tune a number of concurrent writings into Storage service for ProcessManifestOperatorR3 operator. This variable is optional (default: 1). See [MR](https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/39)
|
|
|
- Manifest-based Ingestion - Manifest by reference.
|
|
|
- `AWS` now support ingestion of manifest by reference.
|
|
|
|
... | ... | @@ -216,9 +219,14 @@ No in-place schema changes are required in M15 (no M14 schema resources were cha |
|
|
|
|
|
### Ingestion services
|
|
|
|
|
|
- Manifest-based Ingestion - Major performance improvement.
|
|
|
- osdu-ingestion-lib - writing records in parallel/concurrent manner into Storage service. See [MR](https://urldefense.com/v3/\__https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-ingestion-lib/-/merge_requests/30_\_;!!GF_29dbcQIUBPA!3PiTTmCUIwwQ6UToa6PfFOgrrAkdvsHADXCXk_QEpa6sG5oPUV6xrdRcoDu2jKjk-\_ar4V9ssumqtnn9kSfqrMapqwDrFqIJlts%24)
|
|
|
- osdu-airflow-lib - Added new Airflow variable core__ingestion__thread_save_number to tune a number of concurrent writings into Storage service for ProcessManifestOperatorR3 operator. This variable is optional (default: 1). See [MR](https://urldefense.com/v3/\__https:/community.opengroup.org/osdu/platform/data-flow/ingestion/osdu-airflow-lib/-/merge_requests/39_\_;!!GF_29dbcQIUBPA!3PiTTmCUIwwQ6UToa6PfFOgrrAkdvsHADXCXk_QEpa6sG5oPUV6xrdRcoDu2jKjk-\_ar4V9ssumqtnn9kSfqrMapqwDr98gQipI%24)
|
|
|
- `Lib`
|
|
|
- Upgrade lombok version to fix ingestion request deserialization. See MR [!329](https://community.opengroup.org/osdu/platform/data-flow/ingestion/csv-parser/csv-parser/-/merge_requests/329)
|
|
|
- Trusted development. See MR [!147](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/147)
|
|
|
- Trusted development. See MR [!143](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/core-external-data-workflow/-/merge_requests/143)
|
|
|
- Fix Vulnerabilities. See MR [!12](https://community.opengroup.org/osdu/platform/data-flow/ingestion/external-data-sources/eds-dms/-/merge_requests/12)
|
|
|
- `Segy to Zgy Conversion`
|
|
|
- Fix: segy_configure_remote_access to set http headers. See MR [!163](https://community.opengroup.org/osdu/platform/data-flow/ingestion/segy-to-zgy-conversion/-/merge_requests/163)
|
|
|
|
|
|
|
|
|
### DDMS
|
|
|
|
... | ... | |