Change Python SDK installation in the base image
Hello @spencer!
The idea to move the code from dags.libs to Python SDK and install Python SDK via pip was really great.
However, we have some problems with this https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics-osdu-integration/-/blob/master/build/Dockerfile#L36. As I can see from build/providers/aws/buildspec.yaml
, you put Python SDK's tar.gz package into build
folder to install it in main Dockerfile. But this approach affects our build workflow, as we use GitLab Container Registry, which works with the files only from this repository, and don't have osdu_api-0.0.5.tar.gz
in this repository.
What if we stick to another approach and install the package from Python SDK repo directly in the base image? Example: https://community.opengroup.org/osdu/platform/data-flow/ingestion/energistics-osdu-integration/-/merge_requests/17/diffs?commit_id=600686e6d4b5945aafc0728272f7c3eda34629e1#5850b4551d79f846690db166033014da0368c760_37_37
Or, even better, we can register Python SDK in the GitLab Package Registry and add --extra-index-url <registry_url>
to the requirements file.