Skip to content

GitLab

Commit 9d64e42a authored by Jason's avatar Jason Committed by Daniel Scholl
Browse files

Adding Documentation for Delivery and File Service

parent 052d6f46
Hide whitespace changes
Inline Side-by-side
charts/README.md
View file @ 9d64e42a
......@@ -222,6 +222,8 @@ git clone https://community.opengroup.org/osdu/platform/system/storage.git $SRC_
git clone https://community.opengroup.org/osdu/platform/system/indexer-queue.git $SRC_DIR/indexer-queue
git clone https://community.opengroup.org/osdu/platform/system/indexer-service.git $SRC_DIR/indexer-service
git clone https://community.opengroup.org/osdu/platform/system/search-service.git $SRC_DIR/search-service
git clone https://community.opengroup.org/osdu/platform/system/file.git $SRC_DIR/file-service
git clone https://community.opengroup.org/osdu/platform/system/delivery.git $SRC_DIR/delivery
```
......@@ -299,7 +301,7 @@ helm template airflow ${INFRA_SRC}/charts/airflow -f ${INFRA_SRC}/charts/config_
&& git push origin $UNIQUE)
# Extract manifests from each service chart.
for SERVICE in partition entitlements-azure legal storage indexer-queue indexer-service search-service;
for SERVICE in partition entitlements-azure legal storage indexer-queue indexer-service search-service file-service delivery;
do
helm template $SERVICE ${SRC_DIR}/$SERVICE/devops/azure/chart --set image.branch=$BRANCH --set image.tag=$TAG > ${FLUX_SRC}/providers/azure/hld-registry/$SERVICE.yaml
done
......
docs/code-mirroring.md
View file @ 9d64e42a
......@@ -15,6 +15,7 @@ Empty repositories need to be created that will be used by a pipeline to mirror
| indexer-service | https://community.opengroup.org/osdu/platform/system/indexer-service.git |
| search-service | https://community.opengroup.org/osdu/platform/system/search-service.git |
| delivery | https://community.opengroup.org/osdu/platform/system/delivery.git |
| file | https://community.opengroup.org/osdu/platform/system/file.git |
```bash
export ADO_ORGANIZATION=<organization_name>
......@@ -23,7 +24,7 @@ export ADO_PROJECT=osdu-mvp
az devops configure --defaults organization=https://dev.azure.com/$ADO_ORGANIZATION project=$ADO_PROJECT
# Create required ADO Repositories
for SERVICE in infra-azure-provisioning partition entitlements-azure legal storage indexer-queue indexer-service search-service delivery;
for SERVICE in infra-azure-provisioning partition entitlements-azure legal storage indexer-queue indexer-service search-service delivery file;
do
az repos create --name $SERVICE --organization https://dev.azure.com/${ADO_ORGANIZATION} --project $ADO_PROJECT -ojson
done
......@@ -48,6 +49,7 @@ Variable Group Name: `Mirror Variables`
| INDEXER_REPO | https://dev.azure.com/osdu-demo/osdu/_git/indexer-service |
| SEARCH_REPO | https://dev.azure.com/osdu-demo/osdu/_git/search-service |
| DELIVERY_REPO | https://dev.azure.com/osdu-demo/osdu/_git/delivery |
| FILE_REPO | https://dev.azure.com/osdu-demo/osdu/_git/file |
| ACCESS_TOKEN | <your_personal_access_token> |
......@@ -70,6 +72,7 @@ az pipelines variable-group create \
INDEXER_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/indexer-service \
SEARCH_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/search-service \
DELIVERY_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/delivery \
FILE_REPO=https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/file \
ACCESS_TOKEN=$ACCESS_TOKEN \
-ojson
```
......@@ -180,6 +183,13 @@ jobs:
sourceGitRepositoryUri: 'https://community.opengroup.org/osdu/platform/system/delivery.git'
destinationGitRepositoryUri: '$(DELIVERY_REPO)'
destinationGitRepositoryPersonalAccessToken: $(ACCESS_TOKEN)
- task: swellaby.mirror-git-repository.mirror-git-repository-vsts-task.mirror-git-repository-vsts-task@1
displayName: 'file'
inputs:
sourceGitRepositoryUri: 'https://community.opengroup.org/osdu/platform/system/file.git'
destinationGitRepositoryUri: '$(FILE_REPO)'
destinationGitRepositoryPersonalAccessToken: $(ACCESS_TOKEN)
EOF
(cd ${ADO_PROJECT} && git add -A && git commit -m "pipeline" && git push)
......
docs/service-automation.md
View file @ 9d64e42a
......@@ -26,6 +26,8 @@ This variable group will be used to hold the common values for the services to b
| PUBSUB_TOKEN | `az` |
| SERVICE_CONNECTION_NAME | <your_service_connection_name> |
| GOOGLE_CLOUD_PROJECT | `opendes` |
| FILE_URL | `https://<your_fqdn>/api/file/v2` |
| DELIVERY_URL | `https://<your_fqdn>/api/delivery/v2` |
```bash
......@@ -58,6 +60,8 @@ az pipelines variable-group create \
PUBSUB_TOKEN="az" \
SERVICE_CONNECTION_NAME=$SERVICE_CONNECTION_NAME \
GOOGLE_CLOUD_PROJECT="opendes" \
FILE_URL="https://${DNS_HOST}/api/file/v2" \
DELIVERY_URL="https://${DNS_HOST}/api/delivery/v2/" \
-ojson
```
......@@ -281,7 +285,7 @@ az pipelines variable-group create \
__Setup and Configure the ADO Library `Azure Service Release - search-service`__
This variable group is the service specific variables necessary for testing and deploying the `indexer-service` service.
This variable group is the service specific variables necessary for testing and deploying the `search` service.
| Variable | Value |
|----------|-------|
......@@ -302,6 +306,52 @@ az pipelines variable-group create \
-ojson
```
__Setup and Configure the ADO Library `Azure Service Release - file`__
This variable group is the service specific variables necessary for testing and deploying the `file` service.
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/file-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DFILE_SERVICE_HOST=$(FILE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DDATA_PARTITION_ID=$(MY_TENANT) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/file-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_FILE_SERVICE_NAME)` |
```bash
az pipelines variable-group create \
--name "Azure Service Release - file" \
--authorize true \
--variables \
MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/file-azure" \
MAVEN_INTEGRATION_TEST_OPTIONS='-DFILE_SERVICE_HOST=$(FILE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DDATA_PARTITION_ID=$(MY_TENANT) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0"' \
MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/file-test-azure" \
SERVICE_RESOURCE_NAME='$(AZURE_FILE_SERVICE_NAME)' \
-ojson
```
__Setup and Configure the ADO Library `Azure Service Release - delivery`__
This variable group is the service specific variables necessary for testing and deploying the `delivery` service.
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/delivery-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DDOMAIN=$(DOMAIN) -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DLEGAL_HOST=$(LEGAL_URL) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=common -DOTHER_RELEVANT_DATA_COUNTRIES=US -DLEGAL_TAG=opendes-public-usa-dataset-1 -DSEARCH_HOST=$(SEARCH_URL) -DSTORAGE_HOST=$(STORAGE_URL) -DDELIVERY_HOST=$(DELIVERY_URL)"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/delivery-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_DELIVERY_SERVICE_NAME)` |
```bash
az pipelines variable-group create \
--name "Azure Service Release - delivery" \
--authorize true \
--variables \
MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/delivery-azure" \
MAVEN_INTEGRATION_TEST_OPTIONS='-DargLine="-DDOMAIN=$(DOMAIN) -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DLEGAL_HOST=$(LEGAL_URL) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=common -DOTHER_RELEVANT_DATA_COUNTRIES=US -DLEGAL_TAG=opendes-public-usa-dataset-1 -DSEARCH_HOST=$(SEARCH_URL) -DSTORAGE_HOST=$(STORAGE_URL) -DDELIVERY_HOST=$(DELIVERY_URL)"' \
MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/delivery-test-azure/pom.xml" \
SERVICE_RESOURCE_NAME='$(AZURE_DELIVERY_SERVICE_NAME)' \
-ojson
```
__Create the Chart Pipelines__
......@@ -490,3 +540,35 @@ az pipelines create \
--yaml-path /devops/azure/pipeline.yml \
-ojson
```
7. Add a Pipeline for __file__ to deploy the File Service.
_Repo:_ `file`
_Path:_ `/devops/azure/pipeline.yml`
_Validate:_ https://<your_dns_name>/api/file/v2/swagger-ui.html is alive.
```bash
az pipelines create \
--name 'service-file' \
--repository file \
--branch master \
--repository-type tfsgit \
--yaml-path /devops/azure/pipeline.yml \
-ojson
```
8. Add a Pipeline for __delivery__ to deploy the Delivery Service.
_Repo:_ `delivery`
_Path:_ `/devops/azure/pipeline.yml`
_Validate:_ https://<your_dns_name>/api/delivery/v2/swagger-ui.html is alive.
```bash
az pipelines create \
--name 'service-delivery' \
--repository delivery \
--branch master \
--repository-type tfsgit \
--yaml-path /devops/azure/pipeline.yml \
-ojson
```
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment