Skip to content

Enable Support for Packaged DAGs

harshit aggarwal requested to merge haaggarw/packaged-dags into master

Type of change

  • Bug Fix
  • Feature

Does this introduce a change in the core logic?

  • [Yes]

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Updates description?

The MR here is making changes to support packaged DAGs for manifest, the ADR for this change has been approved

New folder structure

├── osdu_manifest
│   ├── __init__.py
│   ├── libs
│   │   ├── __init__.py
│   │   └── utils.py
│   └── operators
│   |    ├── __init__.py
│   |    └── customOperator1.py
|   |___ hooks
|   |    |__  __init__.py
|   | 
|   |___ configs
|        |__ __init__.py
|
|___ osdu-ingest-r3.py

The changes include in the MR include

  • Restructuring the folders
  • Fixing any import statements
  • Minor changes to run existing tests

Related Issue - #86 (closed)

ADR - https://community.opengroup.org/osdu/platform/data-flow/home/-/issues/47

Edited by harshit aggarwal

Merge request reports