Commit b0cd4e25 authored by harshit aggarwal's avatar harshit aggarwal
Browse files

Merge branch 'master' into haaggarw/AddingActuatorHealthEndpoint

parents 06646396 58c84ad5
## Type
<!-- Please choose the type of ticket. -->
- [ ] Feature Request
- [ ] Bug Report
## Priority
- [ ] High
- [ ] Medium
- [ ] Low
------------------------
------------------------
## Feature Request
<!-- If this is a feature request, fill up the following -->
__Why is this change needed?__
<!-- Please add relevant details. -->
__Current behavior__
<!-- Please describe the current behavior you observe -->
__Expected behavior__
<!-- Please describe the behavior you anticipate -->
----------------------------
--------------------------
## Bug Report
<!-- If this is a bug report, fill up the following -->
__Breaking__
<!-- Is the bug breaking something. -->
- [ ] YES
- [ ] NO
__Attached Logs?__
<!-- Please attach relevant logs. -->
- [ ] YES
- [ ] NO
__Reproduction__
<!-- Please mention how often can you reproduce it. -->
__Current behavior__
<!-- Please describe the current behavior you observe -->
__Expected behavior__
<!-- Please describe the behavior you anticipate -->
__Steps to reproduce__
<!-- Please add how to reproduce the bug -->
--------------------------
--------------------------
## Other information
<!-- Any other information that is important to this PR such as screenshots of how the component looks before and after the change. -->
# Microsoft Open Source Code of Conduct
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
Resources:
- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns
# Contributing to OSDU on Azure infrastructure
## Quicklinks
* [Code of Conduct](CODE_OF_CONDUCT.md)
* [Current Process](#current-process)
## Getting Started
We are trying to formalize a process for contributing to the repo which is a work in process and will continue to evolve.
Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:
- Search for existing Issues and PRs before creating your own.
- We work hard to make sure issues are handled but, it could take a while when dealing with infrastructure to investigate the impact of a change and determine the proper manner of making the change.
- Our goal is to try to now severly reduce the breaking changes necessary.
- As we move forward it is very helpful to have Merge Requests that relate to an Issue Tracking item where conversation can occur and the implementation has been discussed prior to code being submitted.
## Current Process
1. Create an Issue that describes what is necessary.
- An issue could define the overall story of what is being accomplished.
- An issue should define the acceptance criteria of what done means.
- An issue should be marked as approved by a maintainer when fully understood and agreed to.
2. Work the Issue and perform an implementation.
- Issues should be estimated as to how long it is thought it would take to actually implement.
- Issues should be added to milestones when it is planned to actually commit to the work.
- Issues can be worked by developers as long as a maintainer is overseeing the effort.
- When issues are being worked a representative should be communicating to the Maintainer team frequently.
3. Submit the MR for Maintainer approval.
- Merge Requests are requested to be as small as possible to reduce the risk and impact.
- There can be multiple Merge Request for an Issue.
- Stake holders should be defined and tagged to the MR.
- An approval needs to be given by a Maintainer and any Stakeholders identified.
- For some things a Project Owner Approval may be necessary.
......@@ -159,10 +159,9 @@ The script creates some local files to be used.
2. .envrc -- This file is used directory by direnv and requires `direnv allow` to be run to access variables.
3. ~/.ssh/osdu_{UNIQUE}/azure-aks-gitops-ssh-key -- SSH key used by flux.
4. ~/.ssh/osdu_{UNIQUE}/azure-aks-gitops-key.pub -- SSH Public Key used by flux.
5. ~/.ssh/osdu_{UNIQUE}/azure-aks-gitops-key.passphrase -- SSH Key Passphrase used by flux.
6. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key -- SSH Key used by AKS
7. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key.pub -- SSH Public Key used by AKS
8. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key.passphrase -- SSH Key Passphrase used by AKS
5. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key -- SSH Key used by AKS
6. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key.pub -- SSH Public Key used by AKS
7. ~/.ssh/osdu_{UNIQUE}/azure-aks-node-ssh-key.passphrase -- SSH Key Passphrase used by AKS
> Ensure environment variables are loaded `direnv allow`
......@@ -172,9 +171,9 @@ __Installed Azure Resources__
2. Storage Account
3. Key Vault
4. A principal to be used for Terraform _(Requires Grant Admin Approval)_
5. A principal to be used for the OSDU environment.
6. An application to be used for the OSDU environment. _(future)_
7. An application to be used for negative integration testing.
5. A principal to be used for the OSDU environment _(Requires Grant Admin Approval)_
6. An application to be used for the OSDU environment _(future)_
7. An application to be used for negative integration testing
> Removal would require deletion of all AD elements `osdu-mvp-{UNIQUE}-*`, unlocking and deleting the resource group then purging the KV.
......@@ -249,9 +248,11 @@ __Manual Installation__
3. Upload the Integration Test Data following directions [here](./tools/test_data).
3. Deploy the application helm charts following the directions [here](./charts).
4. Deploy the application helm charts following the directions [here](./charts).
5. Setup Environment Variables for IDE Development and Integration Testing.
5. Register the Data Partition API using the API request located [here](../tools/rest/partition.http).
6. Setup Environment Variables for IDE Development and Integration Testing.
__Automated Pipeline Installation__
......
# Helm Installation Instructions
__DNS Record Setup__
Manually update your DNS A Records to point to the Public IP Address for the environment.
```bash
# Get IP Address
RESOURCE_GROUP=$(az group list --query "[?contains(name, '${UNIQUE}sr')].name" -otsv |grep -v MC)
az network public-ip list --resource-group $RESOURCE_GROUP --query [].ipAddress -otsv
```
__CLI Login__
Login to Azure CLI using the OSDU Environment Service Principal.
......@@ -33,11 +23,10 @@ Create the helm chart values file necessary to install charts.
```bash
# Setup Variables
ISTIO_DASH="<your_dash_login>" # ie: admin
ADMIN_EMAIL="<your_cert_admin>" # ie: admin@email.com
DNS_HOST="<your_ingress_hostname>" # ie: osdu.contoso.com
GROUP=$(az group list --query "[?contains(name, '${UNIQUE}cr')].name" -otsv)
GROUP=$(az group list --query "[?contains(name, 'cr${UNIQUE}')].name" -otsv)
ENV_VAULT=$(az keyvault list --resource-group $GROUP --query [].name -otsv)
# Translate Values File
......@@ -78,8 +67,8 @@ global:
# based64 encoded username and password
#
istio:
username: $(echo $ISTIO_DASH | base64)
password: $(echo $ISTIO_DASH | base64)
username: $(az keyvault secret show --id https://${COMMON_VAULT}.vault.azure.net/secrets/istio-username --query value -otsv)
password: $(az keyvault secret show --id https://${COMMON_VAULT}.vault.azure.net/secrets/istio-password --query value -otsv)
EOF
```
......@@ -97,7 +86,6 @@ git clone https://community.opengroup.org/osdu/platform/system/storage.git $SRC_
git clone https://community.opengroup.org/osdu/platform/system/indexer-queue.git $SRC_DIR/indexer-queue
git clone https://community.opengroup.org/osdu/platform/system/indexer-service.git $SRC_DIR/indexer-service
git clone https://community.opengroup.org/osdu/platform/system/search-service.git $SRC_DIR/search-service
git clone https://community.opengroup.org/osdu/platform/system/delivery.git $SRC_DIR/delivery
```
......@@ -108,7 +96,7 @@ __Kubernetes API Access__
It can often be helpful to be able to retrieve the cluster context and execute queries directly against the Kubernetes API.
```bash
BASE_NAME=$(az group list --query "[?contains(name, '${UNIQUE}sr')].name" -otsv |grep -v MC | rev | cut -c 3- | rev)
BASE_NAME=$(az group list --query "[?contains(name, 'sr${UNIQUE}')].name" -otsv |grep -v MC | rev | cut -c 3- | rev)
az aks get-credentials -n ${BASE_NAME}aks -g ${BASE_NAME}rg
```
......
......@@ -4,6 +4,6 @@ Manually update your DNS A Records to point to the Public IP Address for the env
```bash
# Get IP Address
RESOURCE_GROUP=$(az group list --query "[?contains(name, '${UNIQUE}sr')].name" -otsv |grep -v MC)
RESOURCE_GROUP=$(az group list --query "[?contains(name, 'sr${UNIQUE}')].name" -otsv |grep -v MC)
az network public-ip list --resource-group $RESOURCE_GROUP --query [].ipAddress -otsv
```
```
\ No newline at end of file
......@@ -20,6 +20,7 @@ This variable group will be used to hold the common values for the services to b
| EXPIRED_TOKEN | <an_expired_token> |
| HOST_URL | `https://<your_fqdn>/` |
| LEGAL_URL | `https://<your_fqdn>/api/legal/v1/` |
| STORAGE_URL | `https://<your_fqdn>/api/storage/v2/` |
| NO_DATA_ACCESS_TESTER | `$(osdu-mvp-<your_unique>-noaccess-clientid)` |
| NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET | `$(osdu-mvp-<your_unique>-noaccess-secret)` |
| PUBSUB_TOKEN | `az` |
......@@ -51,6 +52,7 @@ az pipelines variable-group create \
EXPIRED_TOKEN=$INVALID_TOKEN \
HOST_URL="https://${DNS_HOST}/" \
LEGAL_URL="https://${DNS_HOST}/api/legal/v1/" \
STORAGE_URL="https://${DNS_HOST}/api/storage/v2/" \
NO_DATA_ACCESS_TESTER='$(osdu-mvp-'${UNIQUE}'-noaccess-clientid)' \
NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET='$(osdu-mvp-'${UNIQUE}'-noaccess-secret))' \
PUBSUB_TOKEN="az" \
......@@ -72,6 +74,7 @@ This variable group is a linked variable group that links to the Common Key Vaul
- istio-password
- osdu-mvp-{unique}-application-clientid
- osdu-mvp-{unique}-application-secret
- osdu-mvp-{unique}-application-oid
- osdu-infra-{unique}-noaccess-clientid
- osdu-infra-{unique}-noaccess-oid
......@@ -91,6 +94,8 @@ This variable group will be used to hold the specific environment values necessa
| DNS_HOST | <your_FQDN> |
| DOMAIN | `contoso.com` |
| ELASTIC_ENDPOINT | `$(opendes-elastic-endpoint)` |
| ELASTIC_USERNAME | `$(opendes-elastic-username)` |
| ELASTIC_PASSWORD | `$(opendes-elastic-password)` |
| IDENTITY_CLIENT_ID | `$(identity_id)` |
| INTEGRATION_TESTER | `$(app-dev-sp-username)` |
| MY_TENANT | `opendes` |
......@@ -115,6 +120,8 @@ az pipelines variable-group create \
DNS_HOST="$DNS_HOST" \
DOMAIN="contoso.com" \
ELASTIC_ENDPOINT='$('${DATA_PARTITION_NAME}'-elastic-endpoint)' \
ELASTIC_USERNAME='$('${DATA_PARTITION_NAME}'-elastic-username)' \
ELASTIC_PASSWORD='$('${DATA_PARTITION_NAME}'-elastic-password)' \
IDENTITY_CLIENT_ID='$(identity_id)' \
INTEGRATION_TESTER='$(app-dev-sp-username)' \
MY_TENANT="$DATA_PARTITION_NAME" \
......@@ -142,6 +149,10 @@ This variable group is a linked variable group that links to the Environment Key
- {partition-name}-elastic-endpoint
- {partition-name}-elastic-password
- {partition-name}-elastic-username
- {partition-name}-storage
- {partition-name}-storage-key
- {partition-name}-sb-connection
- {partition-name}-sb-namespace
- osdu-identity-id
- subscription-id
- tenant-id
......@@ -225,7 +236,7 @@ This variable group is the service specific variables necessary for testing and
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/storage-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=$(DEPLOY_ENV)` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=empty` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/storage-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_STORAGE_SERVICE_NAME)` |
......@@ -235,7 +246,7 @@ az pipelines variable-group create \
--authorize true \
--variables \
MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/storage-azure" \
MAVEN_INTEGRATION_TEST_OPTIONS='-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=$(DEPLOY_ENV)' \
MAVEN_INTEGRATION_TEST_OPTIONS='-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=empty' \
MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/storage-test-azure" \
SERVICE_RESOURCE_NAME='$(AZURE_STORAGE_SERVICE_NAME)' \
-ojson
......@@ -249,7 +260,7 @@ This variable group is the service specific variables necessary for testing and
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/indexer-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=contoso.com -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/indexer-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_INDEXER_SERVICE_NAME)` |
......@@ -259,7 +270,7 @@ az pipelines variable-group create \
--authorize true \
--variables \
MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/indexer-azure" \
MAVEN_INTEGRATION_TEST_OPTIONS='-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=contoso.com -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US' \
MAVEN_INTEGRATION_TEST_OPTIONS='-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US' \
MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/indexer-test-azure" \
SERVICE_RESOURCE_NAME='$(AZURE_INDEXER_SERVICE_NAME)' \
-ojson
......@@ -274,7 +285,7 @@ This variable group is the service specific variables necessary for testing and
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/search-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DINDEXER_HOST=$() -DSTORAGE_HOST=$() -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/integration-tests/search-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_SEARCH_SERVICE_NAME)` |
......@@ -284,7 +295,7 @@ az pipelines variable-group create \
--authorize true \
--variables \
MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/search-azure" \
MAVEN_INTEGRATION_TEST_OPTIONS='-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DINDEXER_HOST=$() -DSTORAGE_HOST=$() -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)' \
MAVEN_INTEGRATION_TEST_OPTIONS='-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)' \
MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/integration-tests/search-test-azure" \
SERVICE_RESOURCE_NAME='$(AZURE_SEARCH_SERVICE_NAME)' \
-ojson
......
......@@ -212,7 +212,7 @@ function CreateADApplication() {
--display-name $1 \
--query [].appId -otsv)
APP_OID=$(az ad app list \
APP_OID=$(az ad sp list \
--display-name $1 \
--query [].objectId -otsv)
......
......@@ -43,6 +43,6 @@ EOF
__Retrieve Additional Values__
The `INITIAL_TOKEN` is an open id token. Follow the directions in osduauth to obtain a token and once obtained save the value in settings.
The `INITIAL_TOKEN` is an open id token. Follow the directions in [osduauth](./osduauth) to obtain a token and once obtained save the value in settings.
The `OSDU_HOST` is your FQDN for your osdu environment.
......@@ -7,7 +7,7 @@ __Create the HTML File__
# This logs your local Azure CLI in using the configured service principal.
az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
GROUP=$(az group list --query "[?contains(name, '${UNIQUE}cr')].name" -otsv)
GROUP=$(az group list --query "[?contains(name, 'cr${UNIQUE}')].name" -otsv)
ENV_VAULT=$(az keyvault list --resource-group $GROUP --query [].name -otsv)
CLIENT_ID=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/aad-client-id --query value -otsv)
......
......@@ -61,15 +61,16 @@ These files need to be uploaded into the proper Cosmos Collections with the requ
- storage_schema_10.json
- storage_schema_11.json
> NOTE: If you are doing a manual deployment, your partition name is "opendes" by default
```bash
# Retrieve Values from Common Key Vault
export NO_DATA_ACCESS_TESTER=$(az keyvault secret show --id https://$COMMON_VAULT.vault.azure.net/secrets/osdu-mvp-demo-noaccess-clientid --query value -otsv)
export NO_DATA_ACCESS_TESTER=$(az keyvault secret show --id https://$COMMON_VAULT.vault.azure.net/secrets/osdu-mvp-${UNIQUE}-noaccess-oid
--query value -otsv)
# Retrieve Values from Environment Key Vault
export COSMOS_ENDPOINT=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/${PARTITION_NAME}-cosmos-endpoint --query value -otsv)
export COSMOS_KEY=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/${PARTITION_NAME}-cosmos-primary-key --query value -otsv)
export COSMOS_KEY=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/${PARTITION_NAME}-cosmos-primary-key --query value -otsv)
export SERVICE_PRINCIPAL_ID=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/app-dev-sp-username --query value -otsv)
export SERVICE_PRINCIPAL_OID=$(az keyvault secret show --id https://${ENV_VAULT}.vault.azure.net/secrets/app-dev-sp-id --query value -otsv)
......
cat > local.yaml <<LOCALRUN
AZURE_TENANT_ID: "${TENANT_ID}"
AZURE_CLIENT_ID: "${ENV_PRINCIPAL_ID}"
AZURE_CLIENT_SECRET: "${ENV_PRINCIPAL_SECRET}"
azure_activedirectory_session_stateless: "true"
azure_activedirectory_AppIdUri: "api://${ENV_APP_ID}"
aad_client_id: "${ENV_APP_ID}"
appinsights_key: "${ENV_APPINSIGHTS_KEY}"
KEYVAULT_URI: "${ENV_KEYVAULT}"
cosmosdb_database: "${COSMOS_DB_NAME}"
spring_application_name: "entitlements-azure"
service_domain_name: "${COMPANY_DOMAIN}"
server_port: "8080"
LOCALRUN
cat > test/local.yaml <<LOCALTEST
ENTITLEMENT_URL: "http://localhost:${server_port}/entitlements/v1/"
MY_TENANT: "${OSDU_TENANT}"
AZURE_AD_TENANT_ID: "${TENANT_ID}"
INTEGRATION_TESTER: "${ENV_PRINCIPAL_ID}"
ENTITLEMENT_MEMBER_NAME_VALID: "${ENV_PRINCIPAL_ID}"
AZURE_TESTER_SERVICEPRINCIPAL_SECRET: "${ENV_PRINCIPAL_SECRET}"
AZURE_AD_APP_RESOURCE_ID: "${ENV_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_ID: "${OTHER_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_OID: "${OTHER_APP_OID}"
DOMAIN: "${COMPANY_DOMAIN}"
EXPIRED_TOKEN: "${INVALID_JWT}"
AZURE_AD_USER_EMAIL: "${AD_USER_EMAIL}"
AZURE_AD_USER_OID: "${AD_USER_OID}"
AZURE_AD_GUEST_EMAIL: "${AD_GUEST_EMAIL}"
AZURE_AD_GUEST_OID: "${AD_GUEST_OID}"
ENTITLEMENT_GROUP_NAME_VALID: "integ.test.data.creator"
ENTITLEMENT_MEMBER_NAME_INVALID: "InvalidTestAdmin"
LOCALTEST
cat > test/hosted.yaml <<DEVTEST
ENTITLEMENT_URL: "https://${ENV_HOST}/entitlements/v1/" # Test Against Environment
MY_TENANT: "${OSDU_TENANT}"
AZURE_AD_TENANT_ID: "${TENANT_ID}"
INTEGRATION_TESTER: "${ENV_PRINCIPAL_ID}"
ENTITLEMENT_MEMBER_NAME_VALID: "${ENV_PRINCIPAL_ID}"
AZURE_TESTER_SERVICEPRINCIPAL_SECRET: "${ENV_PRINCIPAL_SECRET}"
AZURE_AD_APP_RESOURCE_ID: "${ENV_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_ID: "${OTHER_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_OID: "${OTHER_APP_OID}"
DOMAIN: "${COMPANY_DOMAIN}"
EXPIRED_TOKEN: "${INVALID_JWT}"
AZURE_AD_USER_EMAIL: "${AD_USER_EMAIL}"
AZURE_AD_USER_OID: "${AD_USER_OID}"
AZURE_AD_GUEST_EMAIL: "${AD_GUEST_EMAIL}"
AZURE_AD_GUEST_OID: "${AD_GUEST_OID}"
ENTITLEMENT_GROUP_NAME_VALID: "integ.test.data.creator"
ENTITLEMENT_MEMBER_NAME_INVALID: "InvalidTestAdmin"
DEVTEST
#!/usr/bin/env bash
#
# Purpose: Create the Developer Environment Variables.
# Usage:
# entitlements.sh
###############################
## ARGUMENT INPUT ##
###############################
usage() { echo "Usage: DNS_HOST=<your_host> INVALID_JWT=<your_token> entitlements.sh " 1>&2; exit 1; }
SERVICE="entitlements"
if [ -z $UNIQUE ]; then
tput setaf 1; echo 'ERROR: UNIQUE not provided' ; tput sgr0
usage;
fi
if [ -z $DNS_HOST ]; then
tput setaf 1; echo 'ERROR: DNS_HOST not provided' ; tput sgr0
usage;
fi
if [ -z $INVALID_JWT ]; then
tput setaf 1; echo 'ERROR: INVALID_JWT not provided' ; tput sgr0
usage;
fi
if [ -f ./settings_common.env ]; then
source ./settings_common.env;
else
tput setaf 1; echo 'ERROR: common.env not found' ; tput sgr0
fi
if [ -f ./settings_environment.env ]; then
source ./settings_environment.env;
else
tput setaf 1; echo 'ERROR: environment.env not found' ; tput sgr0
fi
if [ ! -d $UNIQUE ]; then mkdir $UNIQUE; fi
# ------------------------------------------------------------------------------------------------------
# LocalHost Run Settings
# ------------------------------------------------------------------------------------------------------
AZURE_TENANT_ID="${TENANT_ID}"
AZURE_CLIENT_ID="${ENV_PRINCIPAL_ID}"
AZURE_CLIENT_SECRET="${ENV_PRINCIPAL_SECRET}"
azure_activedirectory_session_stateless="true"
azure_activedirectory_AppIdUri="api://${ENV_APP_ID}"
aad_client_id="${ENV_APP_ID}"
appinsights_key="${ENV_APPINSIGHTS_KEY}"
KEYVAULT_URI="${ENV_KEYVAULT}"
cosmosdb_database="${COSMOS_DB_NAME}"
spring_application_name="entitlements-azure"
service_domain_name="${COMPANY_DOMAIN}"
partition_service_endpoint="https://${ENV_HOST}/api/partition/v1/"
azure_istioauth_enabled="true"
server_port="8080"
# ------------------------------------------------------------------------------------------------------
# Integration Test Settings
# ------------------------------------------------------------------------------------------------------
ENTITLEMENT_URL="https://${ENV_HOST}/entitlements/v1/"
MY_TENANT="${OSDU_TENANT}"
AZURE_AD_TENANT_ID="${TENANT_ID}"
INTEGRATION_TESTER="${ENV_PRINCIPAL_ID}"
ENTITLEMENT_MEMBER_NAME_VALID="${ENV_PRINCIPAL_ID}"
AZURE_TESTER_SERVICEPRINCIPAL_SECRET="${ENV_PRINCIPAL_SECRET}"
AZURE_AD_APP_RESOURCE_ID="${ENV_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_ID="${OTHER_APP_ID}"
AZURE_AD_OTHER_APP_RESOURCE_OID="${OTHER_APP_OID}"
DOMAIN="${COMPANY_DOMAIN}"
EXPIRED_TOKEN="${INVALID_JWT}"
ENTITLEMENT_GROUP_NAME_VALID="integ.test.data.creator"
ENTITLEMENT_MEMBER_NAME_INVALID="InvalidTestAdmin"
AZURE_AD_USER_EMAIL="${AD_USER_EMAIL}"
AZURE_AD_USER_OID="${AD_USER_OID}"
AZURE_AD_GUEST_EMAIL="${AD_GUEST_EMAIL}"
AZURE_AD_GUEST_OID="${AD_GUEST_OID}"
cat > ${UNIQUE}/${SERVICE}.envrc <<LOCALENV
# ------------------------------------------------------------------------------------------------------
# Common Settings
# ------------------------------------------------------------------------------------------------------
export OSDU_TENANT=$OSDU_TENANT
export OSDU_TENANT2=$OSDU_TENANT2
export OSDU_TENANT3=$OSDU_TENANT3
export COMPANY_DOMAIN=$COMPANY_DOMAIN
export COSMOS_DB_NAME=$COSMOS_DB_NAME
export LEGAL_SERVICE_BUS_TOPIC=$LEGAL_SERVICE_BUS_TOPIC
export RECORD_SERVICE_BUS_TOPIC=$RECORD_SERVICE_BUS_TOPIC
export LEGAL_STORAGE_CONTAINER=$LEGAL_STORAGE_CONTAINER
export TENANT_ID=$TENANT_ID
export INVALID_JWT=$INVALID_JWT
export NO_ACCESS_ID=$NO_ACCESS_ID
export NO_ACCESS_SECRET=$NO_ACCESS_SECRET
export OTHER_APP_ID=$OTHER_APP_ID
export OTHER_APP_OID=$OTHER_APP_OID
export AD_USER_EMAIL=$AD_USER_EMAIL
export AD_USER_OID=$AD_USER_OID
export AD_GUEST_EMAIL=$AD_GUEST_EMAIL
export AD_GUEST_OID=$AD_GUEST_OID
# ------------------------------------------------------------------------------------------------------
# Environment Settings
# ------------------------------------------------------------------------------------------------------
export ENV_SUBSCRIPTION_NAME=$ENV_SUBSCRIPTION_NAME
export ENV_APP_ID=$ENV_APP_ID
export ENV_PRINCIPAL_ID=$ENV_PRINCIPAL_ID
export ENV_PRINCIPAL_SECRET=$ENV_PRINCIPAL_SECRET
export ENV_APPINSIGHTS_KEY=$ENV_APPINSIGHTS_KEY
export ENV_REGISTRY=$ENV_REGISTRY
export ENV_STORAGE=$ENV_STORAGE
export ENV_STORAGE_KEY=$ENV_STORAGE_KEY
export ENV_STORAGE_CONNECTION=$ENV_STORAGE_CONNECTION
export ENV_COSMOSDB_HOST=$ENV_COSMOSDB_HOST
export ENV_COSMOSDB_KEY=$ENV_COSMOSDB_KEY
export ENV_SERVICEBUS_NAMESPACE=$ENV_SERVICEBUS_NAMESPACE
export ENV_SERVICEBUS_CONNECTION=$ENV_SERVICEBUS_CONNECTION
export ENV_KEYVAULT=$ENV_KEYVAULT
export ENV_HOST=$ENV_HOST
export ENV_REGION=$ENV_REGION
export ENV_ELASTIC_HOST=$ENV_ELASTIC_HOST
export ENV_ELASTIC_PORT=$ENV_ELASTIC_PORT
export ENV_ELASTIC_USERNAME=$ENV_ELASTIC_USERNAME
export ENV_ELASTIC_PASSWORD=$ENV_ELASTIC_PASSWORD
# ------------------------------------------------------------------------------------------------------
# LocalHost Run Settings
# ------------------------------------------------------------------------------------------------------
export AZURE_TENANT_ID="${TENANT_ID}"
export AZURE_CLIENT_ID="${ENV_PRINCIPAL_ID}"
export AZURE_CLIENT_SECRET="${ENV_PRINCIPAL_SECRET}"
export azure_activedirectory_session_stateless="true"
export azure_activedirectory_AppIdUri="api://${ENV_APP_ID}"
export aad_client_id="${ENV_APP_ID}"
export appinsights_key="${ENV_APPINSIGHTS_KEY}"
export KEYVAULT_URI="${ENV_KEYVAULT}"
export cosmosdb_database="${COSMOS_DB_NAME}"
export spring_application_name="entitlements-azure"
export service_domain_name="${COMPANY_DOMAIN}"
export partition_service_endpoint="https://${ENV_HOST}/api/partition/v1/"
export azure_istioauth_enabled="true"
export server_port="8080"
# ------------------------------------------------------------------------------------------------------
# Integration Test Settings
# ------------------------------------------------------------------------------------------------------
export ENTITLEMENT_URL="https://${ENV_HOST}/entitlements/v1/"
export MY_TENANT="${OSDU_TENANT}"
export AZURE_AD_TENANT_ID="${TENANT_ID}"
export INTEGRATION_TESTER="${ENV_PRINCIPAL_ID}"
export ENTITLEMENT_MEMBER_NAME_VALID="${ENV_PRINCIPAL_ID}"
export AZURE_TESTER_SERVICEPRINCIPAL_SECRET="${ENV_PRINCIPAL_SECRET}"
export AZURE_AD_APP_RESOURCE_ID="${ENV_APP_ID}"
export AZURE_AD_OTHER_APP_RESOURCE_ID="${OTHER_APP_ID}"
export AZURE_AD_OTHER_APP_RESOURCE_OID="${OTHER_APP_OID}"
export DOMAIN="${COMPANY_DOMAIN}"
export EXPIRED_TOKEN="${INVALID_JWT}"
export ENTITLEMENT_GROUP_NAME_VALID="integ.test.data.creator"
export ENTITLEMENT_MEMBER_NAME_INVALID="InvalidTestAdmin"
export AZURE_AD_USER_EMAIL="${AD_USER_EMAIL}"
export AZURE_AD_USER_OID="${AD_USER_OID}"
export AZURE_AD_GUEST_EMAIL="${AD_GUEST_EMAIL}"
export AZURE_AD_GUEST_OID="${AD_GUEST_OID}"
LOCALENV