[Data Flow/Ingestion] Job management capability for large ingestion runs
Do you see a way to get the list of currently running ingestion jobs and be able to a. query status/logs and b. cancel jobs?
Doesn’t have to be in R2, but want to make sure the design can support this in due course.
I see getStatus as workflow level today, but no logs or cancel ability – should we know the implementation is Airflow and go to Airflow for the logs? Is there value in abstraction for OSDU apps