Move ingestion DAGs and operators under a folder named osdu
Currently the DAGs and operators are in top level folder src
. Clients deploying these DAGs and operators will copy the DAGs into DAGs folder and operators into operators folders.
In a customer environment there will be more DAGs and operators and there are chances where the python names can conflict with the existing names mentioned in this repository.
Can we move the DAGs, operators and hooks into osdu folder or any other folder name so that it will be easy manage. This has to be done in the repository only because the dags use import statements for operators and libs which will fail if someone wanted to put these under different folder structure.
Benifits
- Avoids naming conflict
- Easy to propagate updates from this repository into airflow. We can replace the entire folder in the destination