Schema entity to hold summary information from completed runs
As a Data Manager,
I want to persist airflow xcom summary information when a scheduled fetch-and-ingest job completes execution,
so that I can review the details of each job, including the details of the run, and any errors encountered.
Include the following details in the summary:
- Data provider
- Time started
- Time completed
- Total number of records fetched from the provider
- Total number of records ingested into the consumer's OSDU platform
- Details of any errors encountered during the job
Acceptance Criteria:
-
Code changes -
Code changes to be deployed on Dev. Environment -
Primary Testing in Dev. Environment[Primary Developer] -
Secondary Testing in Dev. Environment[Secondary Developer] -
Unit Test Cases -
MR Raised to master branch -
Internal Code Review -
Code approval -
MR approved[Merged to master] -
Integration Testing[Dev. Env] -
Pre-ship Testing[All CSP's]
Edited by Jeyakumar Devarajulu