Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
harshit aggarwal
Ingestion DAGs
Commits
780df152
Commit
780df152
authored
Aug 27, 2021
by
harshit aggarwal
Browse files
init
parent
0fb19013
Pipeline
#61246
canceled with stage
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
.gitlab-ci.yml
View file @
780df152
# Copyright 2020 Google LLC
# Copyright 2020 EPAM Systems
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
image
:
google/cloud-sdk:alpine
stages
:
-
linters
-
unit_tests
-
test_dags
-
containerize
-
deploy
-
bootstrap
-
testDag
include
:
-
project
:
"
osdu/platform/ci-cd-pipelines"
ref
:
'
master'
file
:
"
cloud-providers/azure_dag.yml"
pylint
:
image
:
eu.gcr.io/osdu-cicd-epam/airflow-python-dags/airflow-python-dags:latest
stage
:
linters
allow_failure
:
true
script
:
-
python -m pip install setuptools pylint pylint_quotes pylint-exit
-
tests/./set_airflow_env.sh
-
pylint --rcfile=.pylintrc src/*/*.py || EXIT_CODE=$?
-
exit ${EXIT_CODE}
isort
:
image
:
eu.gcr.io/osdu-cicd-epam/airflow-python-dags/airflow-python-dags:latest
allow_failure
:
true
stage
:
linters
script
:
-
python -m pip install setuptools isort
-
isort -c -v src/*/*.py || EXIT_CODE=$?
-
exit ${EXIT_CODE}
azure_test_dag
:
tags
:
[
"
docker-runner"
]
stage
:
unit_tests
needs
:
[
"
isort"
]
-
containerize
azure_containerize
:
tags
:
[
"
osdu-medium"
]
image
:
danielscholl/azure-build-image
stage
:
containerize
variables
:
AIRFLOW_IMAGE
:
airflow-image:${CI_COMMIT_SHA}
before_script
:
-
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script
:
-
|
echo "hello"
azure_create_dag
:
tags
:
[
"
docker-runner"
]
azure_copy_dag
:
tags
:
[
"
docker-runner"
]
# Gitlab Container Registry
-
docker build . --build-arg PYTHON_MAJOR_MINOR_VERSION=3.6 --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.12"
--build-arg AIRFLOW_BRANCH="v1-10-test" -t $CI_REGISTRY_IMAGE/$AIRFLOW_IMAGE --file /airflow_Stuff/Dockerfile .
azure_register_dag
:
tags
:
[
"
docker-runner"
]
-
echo $CI_REGISTRY_IMAGE/$AIRFLOW_IMAGE
-
docker push $CI_REGISTRY_IMAGE/$AIRFLOW_IMAGE
only
:
variables
:
-
$AZURE == 'true'
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment