EDS M26: Release Notes and Test Summary

EDS M26 Patches Releases

EDS M26 Known Issues

M26 - Functionality list

Feature/Issue Name Testing Check List Overview To Check Documentation Link
1

EDS- Enhance pipeline to ensure unit test cases pass (#155) · Issue
  • Primary
  • Secondary
  • Preship KT
 To ensure pipeline passes for unit test cases Pipeline will fail/pass according to the unit tests cases

Pipeline behaviour
2

EDS Naturalization in AWS - Upload FileCollection.* not working (#151) · Issue
  • Primary
  • Secondary
  • Preship KT
 EDS Naturalization, To ensure FileCollection can be uploaded to AWS storage Able to upload File Collection to AWS Storage
3

EDS: Fix to resolve Partition URL inconsistencies in different CSP (#113) · Issue
  • Primary
  • Secondary
  • Preship KT
 To standardize the partition URL from env variables for each environment Able to consume the partition url from env variables

EDS Configuration Guide - EDS Services Dependency
4

EDS Scheduler - validation error for Activity in eds_ingest (#168) · Issue

 During a scheduled job, the activity template could be created according to the scheduled job naming convention Able to schedule and create activity template

EDS Configuration Guide - Creation of Activity Logs for monitoring ingestion workflow
5

[EDS Scheduler] - Not creating new token token when current token expires (#171) · Issue

 In an environment that has less users and the airflow is mostly inactive, the token will not be refreshed timely by others tasks EDS Scheduler should be able to renew the token before its execution
6

EDS Naturalization: Transition to Incubator state (#139) · Issue

 No testing required EDS Naturalization is matured to be in incubator state following PMC guideline None
7

EDS Monitoring - Ingestion: (Spike) Identify parameters to be logged in Activity Template from OSDU ingest/ingest by reference & naturalization in EDS Ingest (#158) · Issue

 No testing required To identify which field to be exposed in the activity template None
8

EDS Monitoring - Ingestion: Enhance Activity Template from OSDU ingest in EDS Ingest (#160) · Issue

EDS Monitoring - Ingestion: Enhance Activity Template from OSDU ingest by reference in EDS Ingest (#144) · Issue
  • To expose the EDS runtime logs from EDS during OSDU ingest by reference flow
  • To expose the EDS runtime logs from EDS during OSDU ingest flow
 Able to populate Activity template and XCOM from EDS Ingest

EDS Configuration Guide - Creation of Activity Log for monitoring ingestion workflow
9

EDS Monitoring - Package: Enhance Airflow Logs & Activity Template to include package release version (#147) · Issue
  • Better understanding of functionality available for a particular release
  • Better understanding of known issue of the particular release
  • Ease of troubleshooting of EDS release
  • Able to articulate the release version in the Activity template
 Able to populate python versions in Airflow XCOM and Activity template

EDS Configuration Guide - Creation of Activity Log for monitoring ingestion workflow
10

EDS Naturalization: Transition to Graduated state (#159) · Issue

 No testing needed Get Naturalization to Graduate
11

Successfully ingested records listed as failed (#198) · Issue

 Bug fix where all occurrences of source data-partition-id was replaced with target value causing issues Only the data partition ID slot is replaced
12

EDS Ingest - Fallback to Manifest Ingestion if Manifest By Reference is not available (#200) · Issue

 Fix issue with chunking when MIR DAG is unavailable Manifest Ingest chunking is triggered in ADME
13

EDS Onboarding Plan (#193) · Issue

 No testing needed Come up with a reusable onboarding template

OSDU EDS Documentation - EDS Contributor Onboarding
14

Storage search limit of 100 when FullOSDUImplementationIndicator Flag set to true. (#203) · Issue

 Bug fix for issue with Storage search limit when FullOSDUImplementation flag is true Record batches passed to Storage are kept to 100 max
15

EDS Fetch and Ingest: Cursor Looping Error (#205) · Issue

 Bug fix for Cursor looping logic during EDS fetch Records are fetched properly with cursor iteration
16

Manifest size calculation is incorrect (#202) · Issue

 To ensure manifest payload size is calculated using UTF-8 encoding and returns the actual size. Able to ingest manifests of varying sizes into Azure instance

Edited by Azfar Sabri