Schema entity to hold summary information from completed runs

As a Data Manager,

I want to persist airflow xcom summary information when a scheduled fetch-and-ingest job completes execution,

so that I can review the details of each job, including the details of the run, and any errors encountered.

Include the following details in the summary:

  • Data provider
  • Time started
  • Time completed
  • Total number of records fetched from the provider
  • Total number of records ingested into the consumer's OSDU platform
  • Details of any errors encountered during the job

Acceptance Criteria:

  • Code changes
  • Code changes to be deployed on Dev. Environment
  • Primary Testing in Dev. Environment[Primary Developer]
  • Secondary Testing in Dev. Environment[Secondary Developer]
  • Unit Test Cases
  • MR Raised to master branch
  • Internal Code Review
  • Code approval
  • MR approved[Merged to master]
  • Integration Testing[Dev. Env]
  • Pre-ship Testing[All CSP's]
Edited by Jeyakumar Devarajulu