Commit 7bccdab8 authored by Daniel Scholl's avatar Daniel Scholl
Browse files

Added CSV Parser Directions

parent 0ed72004
Pipeline #31574 passed with stages
in 37 seconds
......@@ -29,3 +29,35 @@ EOF
# Execute container to load the schema's
docker run --env-file .env
## CSV Parser DAG Loading
The CSV Ingestion is an Airflow DAG that has to be loaded. The images are created as part of the [csv-parser]( project and are tied to a release.
# Setup Variables
UNIQUE="<your_osdu_unique>" # ie: demo
DNS_HOST="<your_osdu_fqdn>" # ie: osdu-$
DATA_PARTITION="<your_partition>" # ie:opendes
ACR_REGISTRY="<repository>" # ie:
TAG="<app_version>" # ie: 0.7.0
# This logs your local Azure CLI in using the configured service principal.
az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
GROUP=$(az group list --query "[?contains(name, 'cr${UNIQUE}')].name" -otsv)
ENV_VAULT=$(az keyvault list --resource-group $GROUP --query [].name -otsv)
cat > .env << EOF
AZURE_AD_APP_RESOURCE_ID=$(az keyvault secret show --id https://${ENV_VAULT} --query value -otsv)
AZURE_CLIENT_ID=$(az keyvault secret show --id https://${ENV_VAULT} --query value -otsv)
AZURE_CLIENT_SECRET=$(az keyvault secret show --id https://${ENV_VAULT} --query value -otsv)
docker run -it --env-file .env $ACR_REGISTRY/csv-parser-dag:$TAG
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment