Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in / Register
Toggle navigation
Menu
Open sidebar
Open Subsurface Data Universe Software
Platform
System
SDKs
Python SDK
Commits
12604e9c
Commit
12604e9c
authored
Mar 03, 2021
by
Aliaksandr Lubouski (EPAM)
🥑
Committed by
Siarhei Khaletski (EPAM)
Mar 03, 2021
Browse files
GONRG-1788: python sdk added cloudbuild dir with AirFlow Dockerfile and cloudbuild.yaml
parent
63adbbf2
Changes
2
Hide whitespace changes
Inline
Side-by-side
cloudbuild/Dockerfile.airflow
0 → 100644
View file @
12604e9c
FROM python:3.6-slim-buster
# Never prompt the user for choices on installation/configuration of packages
ENV DEBIAN_FRONTEND noninteractive
ENV TERM linux
# Airflow
ARG AIRFLOW_VERSION=1.10.14
ARG AIRFLOW_USER_HOME=/usr/local/airflow
ARG AIRFLOW_DEPS=""
ARG PYTHON_DEPS=""
ENV AIRFLOW_HOME=${AIRFLOW_USER_HOME}
# Define en_US.
ENV LANGUAGE en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV LC_CTYPE en_US.UTF-8
ENV LC_MESSAGES en_US.UTF-8
# Disable noisy "Handling signal" log messages:
# ENV GUNICORN_CMD_ARGS --log-level WARNING
RUN export SLUGIFY_USES_TEXT_UNIDECODE=yes \
&& set -ex \
&& buildDeps=' \
freetds-dev \
libkrb5-dev \
libsasl2-dev \
libssl-dev \
libffi-dev \
libpq-dev \
git \
' \
&& apt-get update -yqq \
&& apt-get upgrade -yqq \
&& apt-get install -yqq --no-install-recommends \
$buildDeps \
freetds-bin \
build-essential \
default-libmysqlclient-dev \
apt-utils \
curl \
rsync \
netcat \
locales \
&& sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen \
&& locale-gen \
&& update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 \
&& useradd -ms /bin/bash -d ${AIRFLOW_USER_HOME} airflow \
&& pip install -U pip setuptools wheel \
&& pip install pytz \
&& pip install pyOpenSSL \
&& pip install ndg-httpsclient \
&& pip install pyasn1 \
&& pip install apache-airflow[crypto,celery,postgres,hive,jdbc,mysql,ssh${AIRFLOW_DEPS:+,}${AIRFLOW_DEPS}]==${AIRFLOW_VERSION} \
&& pip install 'redis==3.2' \
&& if [ -n "${PYTHON_DEPS}" ]; then pip install ${PYTHON_DEPS}; fi \
&& apt-get purge --auto-remove -yqq $buildDeps \
&& apt-get autoremove -yqq --purge \
&& apt-get clean \
&& rm -rf \
/var/lib/apt/lists/* \
/tmp/* \
/var/tmp/* \
/usr/share/man \
/usr/share/doc \
/usr/share/doc-base
RUN chown -R airflow: ${AIRFLOW_USER_HOME}
COPY python-sdk /python-sdk
RUN python3 -m pip install --user --upgrade setuptools wheel
RUN cd /python-sdk && python3 setup.py sdist bdist_wheel && python3 -m pip install dist/osdu_api-0.0.4-py3-none-any.whl
cloudbuild/cloudbuild.airflow.yaml
0 → 100644
View file @
12604e9c
# Copyright 2021 Google LLC
# Copyright 2021 EPAM
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
steps
:
-
name
:
'
gcr.io/cloud-builders/docker'
args
:
[
'
build'
,
'
-t'
,
'
eu.gcr.io/osdu-cicd-epam/airflow-python-dags/airflow-python-dags:latest'
,
'
-t'
,
'
eu.gcr.io/osdu-cicd-epam/airflow-python-dags/airflow-python-dags:${_SHORT_SHA}'
,
'
-f'
,
'
Dockerfile.airflow'
,
'
.'
]
images
:
-
'
eu.gcr.io/osdu-cicd-epam/airflow-python-dags/airflow-python-dags'
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment