Skip to content
GitLab
Commit 8626a7f1 authored by Daniel Scholl's avatar Daniel Scholl
Browse files

Merge branch 'feature/dperez50/seismic-store-service' into 'master' 

Feature/dperez50/seismic store service

See merge request osdu/platform/deployment-and-operations/infra-azure-provisioning!169
parents 258a2ce0 8a7d0fde
Hide whitespace changes
Inline Side-by-side
charts/README.md
View file @ 8626a7f1
......@@ -247,6 +247,7 @@ git clone https://community.opengroup.org/osdu/platform/data-flow/enrichment/wks
git clone https://community.opengroup.org/osdu/platform/system/register.git $SRC_DIR/register
git clone https://community.opengroup.org/osdu/platform/system/schema-service.git $SRC_DIR/schema-service
git clonehttps://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow.git $SRC_DIR/ingestion-workflow
git clone https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service.git $SRC_DIR/seismic-store-service
```
__Additional Manual Steps__
......
docs/code-mirroring.md
View file @ 8626a7f1
......@@ -25,6 +25,7 @@ Empty repositories need to be created that will be used by a pipeline to mirror
| notification | https://community.opengroup.org/osdu/platform/system/notification.git|
| schema-service | https://community.opengroup.org/osdu/platform/system/schema-service.git|
| ingestion-workflow | https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow.git |
| seismic-store-service | https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service.git |
```bash
export ADO_ORGANIZATION=<organization_name>
......@@ -88,6 +89,7 @@ Variable Group Name: `Mirror Variables`
| NOTIFICATION_REPO | https://dev.azure.com/osdu-demo/osdu/_git/notification |
| SCHEMA_REPO | https://dev.azure.com/osdu-demo/osdu/_git/schema-service |
| INGESTION_WORKFLOW_REPO | https://dev.azure.com/osdu-demo/osdu/_git/ingestion-workflow |
| SEISMIC_STORE_SERVICE_REPO | https://dev.azure.com/osdu-demo/osdu/_git/seismic-store-service |
| ACCESS_TOKEN | <your_personal_access_token> |
......@@ -120,6 +122,7 @@ az pipelines variable-group create \
NOTIFICATION_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/notification \
SCHEMA_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/schema-service \
INGESTION_WORKFLOW_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/ingestion-workflow \
SEISMIC_STORE_SERVICE_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/seismic-store-service
ACCESS_TOKEN=$ACCESS_TOKEN \
-ojson
```
......@@ -299,6 +302,13 @@ jobs:
sourceGitRepositoryUri: 'https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow.git'
destinationGitRepositoryUri: '$(INGESTION_WORKFLOW_REPO)'
destinationGitRepositoryPersonalAccessToken: $(ACCESS_TOKEN)
- task: swellaby.mirror-git-repository.mirror-git-repository-vsts-task.mirror-git-repository-vsts-task@1
displayName: 'seismic-store-service'
inputs:
sourceGitRepositoryUri: 'https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service.git'
destinationGitRepositoryUri: '$(SEISMIC_STORE_SERVICE_REPO)'
destinationGitRepositoryPersonalAccessToken: $(ACCESS_TOKEN)
EOF
......
docs/service-automation.md
View file @ 8626a7f1
......@@ -43,6 +43,7 @@ This variable group will be used to hold the common values for the services to b
| NOTIFICATION_REGISTER_BASE_URL | `https://<your_fqdn>` |
| NOTIFICATION_BASE_URL | `https://<your_fqdn>/api/notification/v1/` |
| REGISTER_CUSTOM_PUSH_URL_HMAC | `https://<your_fqdn>/api/register/v1/test/challenge/1`|
| AGENT_IMAGE | `ubuntu-latest` |
```bash
......@@ -89,8 +90,9 @@ az pipelines variable-group create \
VENDOR="azure" \
LEGAL_TAG="opendes-public-usa-dataset-7643990" \
NOTIFICATION_REGISTER_BASE_URL="https://${DNS_HOST}" \
NOTIFICATION_BASE_URL="https://${DNS_HOST}/api/notification/v1/"
REGISTER_CUSTOM_PUSH_URL_HMAC="https://${DNS_HOST}/api/register/v1/test/challenge/1"
NOTIFICATION_BASE_URL="https://${DNS_HOST}/api/notification/v1/" \
REGISTER_CUSTOM_PUSH_URL_HMAC="https://${DNS_HOST}/api/register/v1/test/challenge/1" \
AGENT_IMAGE="ubuntu-latest"
-ojson
```
......@@ -130,9 +132,12 @@ This variable group will be used to hold the specific environment values necessa
| ELASTIC_ENDPOINT | `$(opendes-elastic-endpoint)` |
| ELASTIC_USERNAME | `$(opendes-elastic-username)` |
| ELASTIC_PASSWORD | `$(opendes-elastic-password)` |
| ENVIRONMENT_NAME | <your_environment_name_or_identifier> |
| IDENTITY_CLIENT_ID | `$(osdu-identity-id)` |
| INTEGRATION_TESTER | `$(app-dev-sp-username)` |
| MY_TENANT | `opendes` |
| PROVIDER_NAME | `azure` |
| REDIS_PORT | `6380` |
| STORAGE_ACCOUNT | `$(opendes-storage)` |
| STORAGE_ACCOUNT_KEY | `$(opendes-storage-key)` |
| AZURE_EVENT_SUBSCRIBER_SECRET | Subscriber Secret used while performing handshake |
......@@ -147,6 +152,10 @@ This variable group will be used to hold the specific environment values necessa
```bash
DATA_PARTITION_NAME=opendes
DNS_HOST="<your_ingress_hostname>" # ie: osdu.contoso.com
ENVIRONMENT_NAME="<your_environment_name_or_identifier>" # ie: dev-sp-id
PROVIDER_NAME=azure
REDIS_PORT="6380"
az pipelines variable-group create \
--name "Azure Target Env - ${UNIQUE}" \
......@@ -163,9 +172,12 @@ az pipelines variable-group create \
ELASTIC_ENDPOINT='$('${DATA_PARTITION_NAME}'-elastic-endpoint)' \
ELASTIC_USERNAME='$('${DATA_PARTITION_NAME}'-elastic-username)' \
ELASTIC_PASSWORD='$('${DATA_PARTITION_NAME}'-elastic-password)' \
ENVIRONMENT_NAME="$ENVIRONMENT_NAME" \
IDENTITY_CLIENT_ID='$(identity_id)' \
INTEGRATION_TESTER='$(app-dev-sp-username)' \
MY_TENANT="$DATA_PARTITION_NAME" \
PROVIDER_NAME="$PROVIDER_NAME" \
REDIS_PORT="$REDIS_PORT" \
STORAGE_ACCOUNT='$('${DATA_PARTITION_NAME}'-storage)' \
STORAGE_ACCOUNT_KEY='$('${DATA_PARTITION_NAME}'-storage-key)' \
AZURE_EVENT_SUBSCRIBER_SECRET="secret" \
......@@ -572,6 +584,57 @@ az pipelines variable-group create \
-ojson
```
__Setup and Configure the ADO Library `Azure Service Release - seismic-store-service`__
This variable group is the service specific variables necessary for testing and deploying the `seismic-store-service` service.
| Variable | Value |
|----------------------------------|-----------------------------------------------------------------------------------------|
| e2eAdminEmail | <your_sslcert_admin_email> |
| e2eDataPartition | `opendes` |
| e2eLegaltag01 | `opendes-public-usa-dataset-7643990` |
| e2eLegaltag02 | `opendes-dps-integration-test-valid2-legal-tag` |
| e2eSubproject | `demo` |
| e2eSubprojectLongname | `looooooooooooooooooooooooooooooooooooooooooooooooooooongnaaaaaaaaaaaaaaaaaaaameeeeeee` |
| e2eTenant | `opendes` |
| PORT | `80` |
| REPLICA_COUNT | `1` |
| serviceUrlSuffix | `seistore-svc/api/v3` |
| utest.mount.dir | `/service` |
| utest.runtime.image | `seistore-svc-runtime` |
```bash
e2eAdminEmail="<your_cert_admin>" # ie: admin@email.com
e2eDataPartition=opendes
e2eLegaltag01=opendes-public-usa-dataset-7643990
e2eLegaltag02=opendes-dps-integration-test-valid2-legal-tag
e2eSubproject=demo
e2eSubprojectLongname=looooooooooooooooooooooooooooooooooooooooooooooooooooongnaaaaaaaaaaaaaaaaaaaameeeeeee
e2eTenant=opendes
PORT="80"
REPLICA_COUNT="1"
serviceUrlSuffix="seistore-svc/api/v3"
utest.mount.dir="/service"
utest.runtime.image=seistore-svc-runtime
az pipelines variable-group create \
--name "Azure Service Release - seismic-store-service" \
--authorize true \
--variables \
e2eAdminEmail=${e2eAdminEmail} \
e2eDataPartition=${e2eDataPartition} \
e2eLegaltag01=${e2eLegaltag01} \
e2eLegaltag02=${e2eLegaltag02} \
e2eSubproject=${e2eSubproject} \
e2eSubprojectLongname=${e2eSubprojectLongname} \
e2eTenant=${e2eTenant} \
PORT='${PORT}' \
REPLICA_COUNT='${REPLICA_COUNT}' \
serviceUrlSuffix='${serviceUrlSuffix}' \
utest.mount.dir='${utest.mount.dir}' \
utest.runtime.image=${utest.runtime.image} \
-ojson
```
__Create the Chart Pipelines__
......@@ -923,3 +986,18 @@ az pipelines create \
--yaml-path /devops/azure/pipeline.yml \
-ojson
```
16. Add a Pipeline for __seismic-store-service__ to deploy the Seismic Store Service.
_Repo:_ `seismic-store-service`
_Path:_ `/devops/azure/pipeline.yml`
_Validate:_ https://<your_dns_name>//seistore-svc/api/v3/svcstatus is alive.
```bash
az pipelines create \
--name 'service-seismic-store' \
--repository seismic-store-service \
--branch master \
--repository-type tfsgit \
--yaml-path /devops/azure/pipeline.yml \
-ojson
```
tools/rest/partition.http
View file @ 8626a7f1
......@@ -111,6 +111,14 @@ data-partition-id: {{data_partition_id}}
"encryption-key-identifier": {
"sensitive": true,
"value": "encryption-key-identifier"
},
"sdms-storage-account-name": {
"sensitive": true,
"value": "sdms-storage"
},
"sdms-storage-account-key": {
"sensitive": true,
"value": "sdms-storage-key"
}
}
}
......
tools/rest/seismic_store_service.http 0 → 100644
View file @ 8626a7f1
# -------HTTP REST CLIENT -------
# https://marketplace.visualstudio.com/items?itemName=humao.rest-client
## This script will register an opendes paritition to the Seismic Store Service.
@login_base = login.microsoftonline.com/{{TENANT_ID}}
@oauth_token_host = {{login_base}}/oauth2/v2.0/token
# -----------------------
# OAUTH authorization_code
# -----------------------
###
# @name login
@login_base = login.microsoftonline.com/{{TENANT_ID}}
@oauth_token_host = {{login_base}}/oauth2/token
POST https://{{oauth_token_host}} HTTP/1.1
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
&client_id={{PRINCIPAL_ID}}
&client_secret={{PRINCIPAL_SECRET}}
&resource={{CLIENT_ID}}
# -----------------------
# API (Variables)
# -----------------------
###
@access_token = {{login.response.body.access_token}}
@ENDPOINT = https://{{OSDU_HOST}}
@SEISMIC_STORE_SVC_HOST = {{ENDPOINT}}/seistore-svc/api/v3
@data_partition_id = opendes
@esd = opendes.contoso.com
@default_acls = users.datalake.admins@opendes.contoso.com
###
# @name tenant
POST {{SEISMIC_STORE_SVC_HOST}}/tenant/{{data_partition_id}}
Authorization: Bearer {{access_token}}
Content-Type: application/json
{
gcpid: {{data_partition_id}},
esd: {{esd}},
default_acls: {{default_acls}}
}
\ No newline at end of file
tools/test_data/tenant_info_1.json
View file @ 8626a7f1
......@@ -20,6 +20,7 @@
"service.workflow.creator",
"service.workflow.viewer",
"service.workflow.admin",
"users.datalake.admins",
"users.datalake.ops",
"cron.job",
"notification.pubsub",
......
tools/test_data/user_info_1.json
View file @ 8626a7f1
......@@ -50,6 +50,7 @@
"service.workflow.creator",
"service.workflow.viewer",
"service.workflow.admin",
"users.datalake.admins",
"users.datalake.ops",
"cron.job",
"notification.pubsub",
......
tools/variables/seismic-store-service.sh 0 → 100644
View file @ 8626a7f1
#!/usr/bin/env bash
#
# Purpose: Create the Developer Environment Variables.
# Usage:
# seismic-store-service.sh
###############################
## ARGUMENT INPUT ##
###############################
usage() { echo "Usage: DNS_HOST=<your_host> INVALID_JWT=<your_token> crs-conversion-service.sh " 1>&2; exit 1; }
SERVICE="seismic-store-service"
if [ -z $UNIQUE ]; then
tput setaf 1; echo 'ERROR: UNIQUE not provided' ; tput sgr0
usage;
fi
if [ -z $DNS_HOST ]; then
tput setaf 1; echo 'ERROR: DNS_HOST not provided' ; tput sgr0
usage;
fi
if [ -z $ENV_NAME ]; then
tput setaf 1; echo 'ERROR: ENV_NAME not provided' ; tput sgr0
usage;
fi
if [ -z $PORT ]; then
tput setaf 1; echo 'ERROR: PORT not provided' ; tput sgr0
usage;
fi
if [ -f ./settings_common.env ]; then
source ./settings_common.env;
else
tput setaf 1; echo 'ERROR: common.env not found' ; tput sgr0
fi
if [ -f ./settings_environment.env ]; then
source ./settings_environment.env;
else
tput setaf 1; echo 'ERROR: environment.env not found' ; tput sgr0
fi
if [ ! -d $UNIQUE ]; then mkdir $UNIQUE; fi
# ------------------------------------------------------------------------------------------------------
# LocalHost Run Settings
# ------------------------------------------------------------------------------------------------------
CLOUDPROVIDER="azure"
DES_SERVICE_HOST="https://${DNS_HOST}/entitlements/v1"
REDIS_INSTANCE_PORT="6379"
APP_ENVIRONMENT_IDENTIFIER="${ENV_NAME}"
PORT="${PORT}"
KEYVAULT_URL=$ENV_KEYVAULT
# ------------------------------------------------------------------------------------------------------
# Integration Test Settings
# ------------------------------------------------------------------------------------------------------
# ------------------------------------------------------------------------------------------------------
# Create envrc file
# ------------------------------------------------------------------------------------------------------
cat > ${UNIQUE}/${SERVICE}.envrc <<LOCALENV
# ------------------------------------------------------------------------------------------------------
# Common Settings
# ------------------------------------------------------------------------------------------------------
export CLOUDPROVIDER=$CLOUDPROVIDER
# ------------------------------------------------------------------------------------------------------
# Environment Settings
# ------------------------------------------------------------------------------------------------------
export ENV_SUBSCRIPTION_NAME=$ENV_SUBSCRIPTION_NAME
export ENV_APP_ID=$ENV_APP_ID
export ENV_PRINCIPAL_ID=$ENV_PRINCIPAL_ID
export ENV_PRINCIPAL_SECRET=$ENV_PRINCIPAL_SECRET
export DES_SERVICE_HOST=$DES_SERVICE_HOST
export REDIS_INSTANCE_PORT=$REDIS_INSTANCE_PORT
export APP_ENVIRONMENT_IDENTIFIER=$APP_ENVIRONMENT_IDENTIFIER
expor PORT=$PORT
export KEYVAULT_URL=$KEYVAULT_URL
# ------------------------------------------------------------------------------------------------------
# LocalHost Run Settings
# ------------------------------------------------------------------------------------------------------
export AZURE_TENANT_ID="${TENANT_ID}"
export AZURE_CLIENT_ID="${ENV_PRINCIPAL_ID}"
export AZURE_CLIENT_SECRET="${ENV_PRINCIPAL_SECRET}"
export KEYVAULT_URI="${ENV_KEYVAULT}"
export aad_client_id="${ENV_APP_ID}"
# ------------------------------------------------------------------------------------------------------
# Integration Test Settings
# ------------------------------------------------------------------------------------------------------
LOCALENV
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment