Enable Support for Packaged DAGs
Type of change
-
Bug Fix -
Feature
Does this introduce a change in the core logic?
- [Yes]
Does this introduce a change in the cloud provider implementation, if so which cloud?
-
AWS -
Azure -
GCP -
IBM
Updates description?
The MR here is making changes to support packaged DAGs for manifest, the ADR for this change has been approved
New folder structure
├── osdu_manifest
│ ├── __init__.py
│ ├── libs
│ │ ├── __init__.py
│ │ └── utils.py
│ └── operators
│ | ├── __init__.py
│ | └── customOperator1.py
| |___ hooks
| | |__ __init__.py
| |
| |___ configs
| |__ __init__.py
|
|___ osdu-ingest-r3.py
The changes include in the MR include
- Restructuring the folders
- Fixing any import statements
- Minor changes to run existing tests
Related Issue - #86 (closed)
ADR - https://community.opengroup.org/osdu/platform/data-flow/home/-/issues/47