Commit d9d93f1f authored by Aman Verma's avatar Aman Verma
Browse files

mergin with latest master

parents 6566cd3e bcf2513a
Pipeline #67064 passed with stages
in 49 seconds
...@@ -2,6 +2,27 @@ ...@@ -2,6 +2,27 @@
# Current Master # Current Master
# v0.11.0 (2021-9-1)
__Branch__ https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/release/0.11
__Tag__ https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tags/v0.11.2
__Infra Changes__
- [Keda Upgrade to 2.x](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/128) - Follow this Documentation to do it: [Keda Upgrade](docs/keda-upgrade.md)
- [BYOAD Enable](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/197)
__Service Onboarded__
- [Dataset Service](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/153)
__Feature Changes__
- [Add multi-partition support in Indexer Queue](https://community.opengroup.org/osdu/platform/system/indexer-queue/-/issues/6)
- [Notification Service to use Service Bus](https://community.opengroup.org/osdu/platform/system/notification/-/issues/25)
- [Notification loss, if migration not done](https://community.opengroup.org/osdu/platform/system/notification/-/blob/master/provider/notification-azure/docs/MIGRATION.md)
__Deprecation Notes__
- The Notification service has started functioning on Service Bus as the underlying PubSub. Starting v0.12.0, Event Grid will be deprecated. In v0.11.0, the service will have two deployments - one using Event Grid and one uses Service Bus.
# v0.10.0 (2021-8-8) # v0.10.0 (2021-8-8)
__Infra Changes__ __Infra Changes__
......
...@@ -174,24 +174,13 @@ __Installed Azure Resources__ ...@@ -174,24 +174,13 @@ __Installed Azure Resources__
1. Resource Group 1. Resource Group
2. Storage Account 2. Storage Account
3. Key Vault 3. Key Vault
4. A principal to be used by Terraform to create all resources for an OSDU Environment. _(Requires Grant Admin Approval)_ 4. A principal to be used by Terraform to create all resources for an OSDU Environment.
5. A principal required by an OSDU environment deployment that will have root level access to that environment. _(Requires Grant Admin Approval)_ 5. A principal required by an OSDU environment deployment.
6. An AD application to be leveraged in the future that defines and controls access to the OSDU Environment for AD Identity. _(future)_ 6. An AD application to be leveraged that defines and controls access to the OSDU Environment for AD Identity.
7. An AD application to be used for negative integration testing 7. An AD application to be used for negative integration testing
> Removal would require deletion of all AD elements `osdu-mvp-{UNIQUE}-*`, unlocking and deleting the resource group. > Removal would require deletion of all AD elements `osdu-mvp-{UNIQUE}-*`, unlocking and deleting the resource group.
__Azure AD Admin Consent__
2 service principals have been created that need to have an AD Admin `grant admin consent` on.
1. osdu-mvp-{UNIQUE}-terraform _(Azure AD Application Graph - Application.ReadWrite.OwnedBy)_
2. osdu-mvp-{UNIQUE}-principal _(Microsoft Graph - Directory.Read.All)_
For more information on Azure identity and authorization, see the official Microsoft documentation [here](https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-permissions-and-consent).
## Elastic Search Setup ## Elastic Search Setup
Infrastructure requires a bring your own Elastic Search Instance of a version of 7.x (ie: 7.11.1) with a valid https endpoint and the access information must now be stored in the Common KeyVault. The recommended method of Elastic Search is to use the [Elastic Cloud Managed Service from the Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/elastic.ec-azure?tab=Overview). Infrastructure requires a bring your own Elastic Search Instance of a version of 7.x (ie: 7.11.1) with a valid https endpoint and the access information must now be stored in the Common KeyVault. The recommended method of Elastic Search is to use the [Elastic Cloud Managed Service from the Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/elastic.ec-azure?tab=Overview).
...@@ -282,6 +271,9 @@ __Automated Pipeline Installation__ ...@@ -282,6 +271,9 @@ __Automated Pipeline Installation__
1. Setup Airflow DNS to point the deployed airflow in data partition followin directions [here](./docs/dp-airflow-dns-setup.md) 1. Setup Airflow DNS to point the deployed airflow in data partition followin directions [here](./docs/dp-airflow-dns-setup.md)
__Steps to load TNO Data__
https://community.opengroup.org/osdu/platform/data-flow/data-loading/open-test-data/-/blob/master/rc--3.0.0/6-data-load-scripts/README.md
__Data Migration for Entitlements from Milestone 4(v0.7.0) or lower, to Milestone 5(v0.8.0) or higher__ __Data Migration for Entitlements from Milestone 4(v0.7.0) or lower, to Milestone 5(v0.8.0) or higher__
...@@ -289,6 +281,14 @@ Milestone 5(v0.8.0) introduced a breaking changed for Entitlements, which requir ...@@ -289,6 +281,14 @@ Milestone 5(v0.8.0) introduced a breaking changed for Entitlements, which requir
[here](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/tree/master/data-migration). [here](https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/tree/master/data-migration).
The script should be run whenever you update OSDU installation from less than Milestone 5(v0.8.0) to equivalent or higher. The script should be run whenever you update OSDU installation from less than Milestone 5(v0.8.0) to equivalent or higher.
__Migration scripts for Notification from Milestone 7(v0.10.0) or lower, to Milestone 8(v0.11.0) or higher__
https://community.opengroup.org/osdu/platform/system/notification/-/blob/master/provider/notification-azure/docs/MIGRATION.md
__KEDA upgrade steps from Milestone 7(v0.10.0) or lower, to Milestone 8(v0.11.0) or higher (Not Mandatory)__
https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/keda-upgrade.md
## How to enable Policy based authorization (optional) ## How to enable Policy based authorization (optional)
Follow the steps in the [link](https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/docs/tutorial/PolicyService-Integration.md) to enbale policy based authoorization. Follow the steps in the [link](https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/docs/tutorial/PolicyService-Integration.md) to enbale policy based authoorization.
......
...@@ -298,7 +298,9 @@ airflow: ...@@ -298,7 +298,9 @@ airflow:
value: "/opt/celery" value: "/opt/celery"
# Needed for installing python osdu python sdk. In future this will be changed # Needed for installing python osdu python sdk. In future this will be changed
- name: CI_COMMIT_TAG - name: CI_COMMIT_TAG
value: "v0.11.0" value: "v0.12.0"
- name: BUILD_TAG
value: "v0.12.0"
extraConfigmapMounts: extraConfigmapMounts:
- name: remote-log-config - name: remote-log-config
mountPath: /opt/airflow/config mountPath: /opt/airflow/config
...@@ -328,7 +330,7 @@ airflow: ...@@ -328,7 +330,7 @@ airflow:
"requests==2.25.1", "requests==2.25.1",
"tenacity==8.0.1", "tenacity==8.0.1",
"https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_api-0.11.0.tar.gz", "https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_api-0.11.0.tar.gz",
"https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_airflow-0.0.1.tar.gz" "https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_airflow-0.0.1.dev31+59e58330.tar.gz"
] ]
extraVolumeMounts: extraVolumeMounts:
- name: azure-keyvault - name: azure-keyvault
......
...@@ -313,7 +313,9 @@ airflow: ...@@ -313,7 +313,9 @@ airflow:
value: "/opt/celery" value: "/opt/celery"
# Needed for installing python osdu python sdk. In future this will be changed # Needed for installing python osdu python sdk. In future this will be changed
- name: CI_COMMIT_TAG - name: CI_COMMIT_TAG
value: "v0.11.0" value: "v0.12.0"
- name: BUILD_TAG
value: "v0.12.0"
extraConfigmapMounts: extraConfigmapMounts:
- name: remote-log-config - name: remote-log-config
mountPath: /opt/airflow/config mountPath: /opt/airflow/config
...@@ -343,7 +345,7 @@ airflow: ...@@ -343,7 +345,7 @@ airflow:
"requests==2.25.1", "requests==2.25.1",
"tenacity==8.0.1", "tenacity==8.0.1",
"https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_api-0.11.0.tar.gz", "https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_api-0.11.0.tar.gz",
"https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_airflow-0.0.1.tar.gz" "https://azglobalosdutestlake.blob.core.windows.net/pythonsdk/osdu_airflow-0.0.1.dev31+59e58330.tar.gz"
] ]
extraVolumeMounts: extraVolumeMounts:
- name: azure-keyvault - name: azure-keyvault
......
...@@ -121,3 +121,7 @@ spec: ...@@ -121,3 +121,7 @@ spec:
serviceName: dataset serviceName: dataset
servicePort: 80 servicePort: 80
path: /api/dataset/v1/* path: /api/dataset/v1/*
- backend:
serviceName: seismic-dms-file-metadata-service
servicePort: 80
path: /seismic-file-metadata/api/v1/*
...@@ -51,6 +51,11 @@ spec: ...@@ -51,6 +51,11 @@ spec:
configMapKeyRef: configMapKeyRef:
name: {{ .Values.global.job.configmap_name }} name: {{ .Values.global.job.configmap_name }}
key: ENV_CLUSTER_NAME key: ENV_CLUSTER_NAME
- name: ENV_APPGW_NAME
valueFrom:
configMapKeyRef:
name: {{ .Values.global.job.configmap_name }}
key: ENV_APPGW_NAME
command: command:
- /bin/sh - /bin/sh
args: args:
...@@ -62,6 +67,7 @@ spec: ...@@ -62,6 +67,7 @@ spec:
# Compare expire dates of certificates in Key Vault and in istio-system namespaces # Compare expire dates of certificates in Key Vault and in istio-system namespaces
function check_expire_date() { function check_expire_date() {
echo "Compare expire dates of certificates in Key Vault and in istio-system namespaces"
az keyvault certificate download --vault-name ${ENV_KEYVAULT_NAME} -n ${KV_CERT_NAME} --file ${KV_CERT_NAME}.pem az keyvault certificate download --vault-name ${ENV_KEYVAULT_NAME} -n ${KV_CERT_NAME} --file ${KV_CERT_NAME}.pem
KV_CERT_EXPIREDATE=$(openssl x509 -in ${KV_CERT_NAME}.pem -enddate -noout | cut -d '=' -f2) KV_CERT_EXPIREDATE=$(openssl x509 -in ${KV_CERT_NAME}.pem -enddate -noout | cut -d '=' -f2)
KV_CERT_EXPIREDATE=$(date "+%Y-%m-%d" --date="${KV_CERT_EXPIREDATE}") KV_CERT_EXPIREDATE=$(date "+%Y-%m-%d" --date="${KV_CERT_EXPIREDATE}")
...@@ -84,7 +90,7 @@ spec: ...@@ -84,7 +90,7 @@ spec:
# Cleanup function # Cleanup function
cleanup() { cleanup() {
echo Clean all existing files echo "Clean all existing files"
rm -f cert.crt cert.key osdu-certificate.pfx ${KV_CERT_NAME}.pem rm -f cert.crt cert.key osdu-certificate.pfx ${KV_CERT_NAME}.pem
curl -X POST "http://localhost:${SIDECAR_PORT}/quitquitquit" curl -X POST "http://localhost:${SIDECAR_PORT}/quitquitquit"
} }
...@@ -111,7 +117,7 @@ spec: ...@@ -111,7 +117,7 @@ spec:
# Log In in Azure # Log In in Azure
az login --service-principal -u ${client_id} -p ${client_secret} --tenant ${tenant_id} az login --service-principal -u ${client_id} -p ${client_secret} --tenant ${tenant_id}
{{- if .Values.global.istio.enableIstioKeyvaultCert }} {{- if .Values.global.istio.enableIstioKeyvaultCert }}
K8S_CERT_SECRET=istio-appgw-ssl-cert K8S_CERT_SECRET=istio-appgw-ssl-cert
K8S_NAMESPACE_NAME=istio-system K8S_NAMESPACE_NAME=istio-system
...@@ -120,8 +126,13 @@ spec: ...@@ -120,8 +126,13 @@ spec:
check_expire_date check_expire_date
# Download BYOC certificate from keyvault # Download BYOC certificate from keyvault
echo "Download BYOC certificate from keyvault"
az keyvault secret download --file ${KV_CERT_NAME}.pfx --vault-name ${ENV_KEYVAULT_NAME} --encoding base64 --name ${KV_CERT_NAME} az keyvault secret download --file ${KV_CERT_NAME}.pfx --vault-name ${ENV_KEYVAULT_NAME} --encoding base64 --name ${KV_CERT_NAME}
# Upload BYOC certificate to appgw
echo "Upload BYOC certificate to appgw"
az network application-gateway ssl-cert update -g ${ENV_SR_GROUP_NAME} --gateway-name ${ENV_APPGW_NAME} -n ${KV_CERT_NAME} --cert-file ${K8S_CERT_SECRET}.pfx --cert-password ""
# Extract key and crt # Extract key and crt
openssl pkcs12 -in ${K8S_CERT_SECRET}.pfx -out cert.pem -passin pass:"" -nodes -passout pass:"" openssl pkcs12 -in ${K8S_CERT_SECRET}.pfx -out cert.pem -passin pass:"" -nodes -passout pass:""
openssl rsa -in cert.pem -out cert.key openssl rsa -in cert.pem -out cert.key
...@@ -135,6 +146,9 @@ spec: ...@@ -135,6 +146,9 @@ spec:
sleep 5 sleep 5
# Restart istio-ingressgateway pods
kubectl rollout restart -n ${K8S_ISTIONAMESPACE_NAME} deployment/istio-ingressgateway
# Check certificate expire date one more time # Check certificate expire date one more time
check_expire_date check_expire_date
......
...@@ -43,6 +43,11 @@ spec: ...@@ -43,6 +43,11 @@ spec:
configMapKeyRef: configMapKeyRef:
name: {{ .Values.global.job.configmap_name }} name: {{ .Values.global.job.configmap_name }}
key: ENV_CLUSTER_NAME key: ENV_CLUSTER_NAME
- name: ENV_APPGW_NAME
valueFrom:
configMapKeyRef:
name: {{ .Values.global.job.configmap_name }}
key: ENV_APPGW_NAME
command: command:
- /bin/sh - /bin/sh
args: args:
...@@ -82,14 +87,19 @@ spec: ...@@ -82,14 +87,19 @@ spec:
az login --service-principal -u ${client_id} -p ${client_secret} --tenant ${tenant_id} az login --service-principal -u ${client_id} -p ${client_secret} --tenant ${tenant_id}
az aks get-credentials --resource-group ${ENV_SR_GROUP_NAME} --name ${ENV_CLUSTER_NAME} az aks get-credentials --resource-group ${ENV_SR_GROUP_NAME} --name ${ENV_CLUSTER_NAME}
{{- if .Values.global.istio.enableIstioKeyvaultCert }} {{- if .Values.global.istio.enableIstioKeyvaultCert }}
K8S_CERT_SECRET=istio-appgw-ssl-cert K8S_CERT_SECRET=istio-appgw-ssl-cert
K8S_NAMESPACE_NAME=istio-system K8S_NAMESPACE_NAME=istio-system
# Download BYOC certificate from keyvault # Download BYOC certificate from keyvault
echo "Download BYOC certificate from keyvault"
az keyvault secret download --file ${KV_CERT_NAME}.pfx --vault-name ${ENV_KEYVAULT_NAME} --encoding base64 --name ${KV_CERT_NAME} az keyvault secret download --file ${KV_CERT_NAME}.pfx --vault-name ${ENV_KEYVAULT_NAME} --encoding base64 --name ${KV_CERT_NAME}
# Upload BYOC certificate to appgw
echo "Upload BYOC certificate to appgw"
az network application-gateway ssl-cert create -g ${ENV_SR_GROUP_NAME} --gateway-name ${ENV_APPGW_NAME} -n ${KV_CERT_NAME} --cert-file ${K8S_CERT_SECRET}.pfx --cert-password ""
# Extract key and crt # Extract key and crt
openssl pkcs12 -in ${K8S_CERT_SECRET}.pfx -out cert.pem -passin pass:"" -nodes -passout pass:"" openssl pkcs12 -in ${K8S_CERT_SECRET}.pfx -out cert.pem -passin pass:"" -nodes -passout pass:""
openssl rsa -in cert.pem -out cert.key openssl rsa -in cert.pem -out cert.key
...@@ -101,6 +111,9 @@ spec: ...@@ -101,6 +111,9 @@ spec:
--from-file=tls.crt=cert.crt --from-file=tls.key=cert.key \ --from-file=tls.crt=cert.crt --from-file=tls.key=cert.key \
-o yaml | kubectl apply -f - -o yaml | kubectl apply -f -
# Restart istio-ingressgateway pods
kubectl rollout restart -n ${K8S_ISTIONAMESPACE_NAME} deployment/istio-ingressgateway
{{ else }} {{ else }}
K8S_CERT_SECRET=osdu-certificate K8S_CERT_SECRET=osdu-certificate
......
# Enable BYOAD
We've added a feature flag (aad_client_id) to enable or disable auto-creation of ad-application in central resources.
# Updating existing infra to have custom AD Application
Doing this will make current auth and refresh codes invalid. They'll need to be generated again.
1. Users with manual deployment, if already not set, set aad_client_id = {{application client id of the custom ad application created}} in custom values file for terraform apply in central resources.
2. Users with automated pipeline -
1. . Go to Pipelines Library in ADO
2. Go to `Infrastructure Pipeline Variables - demo` variable group
3. Add or update the below variable if already not set
| Variable | Value |
|----------|-------|
| TF_VAR_aad_client_id | {{application client id of manually created ad application}} |
3. Users with automated pipeline should now run chart chart-osdu-istio and chart-osdu-istio-auth pipeline.
4. Users with manual deployment need to re-install osdu-istio helm chart with new app-id.
5. Delete all pods in the portal AKS. This will trigger a restart of all pods. Complete steps 5,6 and 7 in quick procession.
6. While all pods are getting restarted, move to configuration in portal AKS. Select secrets tab and choose osdu/osdu-azure namespace.
7. Delete active-directory from the results. This will trigger its recreation.
8. Delete all pods again to make sure that new pods are using new active directory secrets.
9. Run this script with required values substituted - [subscriberCreationRegisterService](./Trouble%20Shooting%20Guides/tsg-scripts/subscriberCreationRegisterService.ps1)
\ No newline at end of file
...@@ -34,10 +34,6 @@ This is likely an issue with the `Application.ReadWrite.OwnedBy` permissions tha ...@@ -34,10 +34,6 @@ This is likely an issue with the `Application.ReadWrite.OwnedBy` permissions tha
The common_prepare.sh script is a helper script that helps to perform the activities necessary to provision OSDU on Azure.  These activities can all be performed manually if desired.  Service Principals are created using the command `az ad sp create-for-rbac` which requires Owner permissions on a subscription to perform. The common_prepare.sh script is a helper script that helps to perform the activities necessary to provision OSDU on Azure.  These activities can all be performed manually if desired.  Service Principals are created using the command `az ad sp create-for-rbac` which requires Owner permissions on a subscription to perform.
   
## Why does the Service Principal used by Terraform to create an OSDU Environment Stamp require Azure AD Graph API access levels of `Application.ReadWrite.OwnedBy`?
Terraform is used to provision an OSDU Environment Stamp a Service Principal is the identity used by Terraform to perform this action.  An OSDU Environment Stamp requires an AD Application used for Identity Management which is currently created by the Terraform Scripts.  In order for a Service Principal to be able to create Applications in AD, the permission of `Application.ReadWrite.OwnedBy` is required for the Azure AD Graph API.
 
## Why does the Service Principal used internally within an OSDU Environment Stamp require MS Graph API  access levels of `Directory.ReadAll`? ## Why does the Service Principal used internally within an OSDU Environment Stamp require MS Graph API  access levels of `Directory.ReadAll`?
The OSDU Entitlement Service integrates with Azure AD.  The defined API spec for the service includes a Create method for which input criteria includes an email address.  This email address is looked up in Azure AD to confirm it exists and retrieve the Object Id of the user to be used as the source of identity which requires the permission of `Directory.ReadAll` for the MS Graph API. The OSDU Entitlement Service integrates with Azure AD.  The defined API spec for the service includes a Create method for which input criteria includes an email address.  This email address is looked up in Azure AD to confirm it exists and retrieve the Object Id of the user to be used as the source of identity which requires the permission of `Directory.ReadAll` for the MS Graph API.
...@@ -59,7 +55,7 @@ This key pair can be used to ssh into an AKS node if needed. ...@@ -59,7 +55,7 @@ This key pair can be used to ssh into an AKS node if needed.
* osdu-mvp-xxx-terraform – A principal identity that can be used by Terraform for creating OSDU Resources * osdu-mvp-xxx-terraform – A principal identity that can be used by Terraform for creating OSDU Resources
* osdu-mvp-xxx-principal – A principal identity that is fed to an OSDU Deployment to be used as the Root Level Identity for that OSDU Environment * osdu-mvp-xxx-principal – A principal identity that is fed to an OSDU Deployment to be used as the Root Level Identity for that OSDU Environment
* osdu-mvp-xxx-noaccess – A negative testing principal identity. * osdu-mvp-xxx-noaccess – A negative testing principal identity.
* osdu-mvp-xxx-application – An AD Application for future use. (Not currently used yet.) * osdu-mvp-xxx-application – An AD Application.
## What AAD Items are created by the central resource template? ## What AAD Items are created by the central resource template?
* osdu-mvp-crxxx-xxxx-app – An AD application that defines the OSDU Environment created. * osdu-mvp-crxxx-xxxx-app – An AD application that defines the OSDU Environment created.
......
...@@ -105,6 +105,7 @@ az pipelines variable-group create \ ...@@ -105,6 +105,7 @@ az pipelines variable-group create \
TF_VAR_principal_password="${TF_VAR_principal_password}" \ TF_VAR_principal_password="${TF_VAR_principal_password}" \
TF_VAR_resource_group_location="${REGION}" \ TF_VAR_resource_group_location="${REGION}" \
TF_VAR_deploy_dp_airflow="false" \ TF_VAR_deploy_dp_airflow="false" \
TF_VAR_aad_client_id="$TF_VAR_application_clientid" \
-ojson -ojson
``` ```
...@@ -121,6 +122,17 @@ To enable airflow multi partition turn on the feature flag by following the belo ...@@ -121,6 +122,17 @@ To enable airflow multi partition turn on the feature flag by following the belo
| TF_VAR_deploy_dp_airflow | true | | TF_VAR_deploy_dp_airflow | true |
| TF_VAR_ssl_challenge_required | true (if not using BYOC) <br> false (if using BYOC) | | TF_VAR_ssl_challenge_required | true (if not using BYOC) <br> false (if using BYOC) |
__Enable BYOAD__
To enable byoad, turn on the feature flag by following the below steps. If you don't want to create your own AD Application, you can skip it.
1. Go to Pipelines Library in ADO
2. Go to `Infrastructure Pipeline Variables - demo` variable group
3. Add or update the below variable
| Variable | Value |
|----------|-------|
| TF_VAR_aad_client_id | {{application client id of manually created ad application}} |
__Setup and Configure the ADO Library `Infrastructure Pipeline Secrets - demo`__ __Setup and Configure the ADO Library `Infrastructure Pipeline Secrets - demo`__
> This should be linked Secrets from Azure Key Vault `osducommon<random>` > This should be linked Secrets from Azure Key Vault `osducommon<random>`
...@@ -161,7 +173,6 @@ az pipelines create \ ...@@ -161,7 +173,6 @@ az pipelines create \
-ojson -ojson
``` ```
2. `infrastructure-data-partition` 2. `infrastructure-data-partition`
> For the first run of the pipeline approvals will need to be made for the 2 secure files and the Service Connection. > For the first run of the pipeline approvals will need to be made for the 2 secure files and the Service Connection.
......
...@@ -13,7 +13,6 @@ DATA_PARTITION="<your_partition>" # ie:opendes ...@@ -13,7 +13,6 @@ DATA_PARTITION="<your_partition>" # ie:opendes
ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io
TAG="<app_version>" # ie: 0.12.0 TAG="<app_version>" # ie: 0.12.0
# This logs your local Azure CLI in using the configured service principal. # This logs your local Azure CLI in using the configured service principal.
az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
...@@ -43,7 +42,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo ...@@ -43,7 +42,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo
AZURE_DNS_NAME="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com AZURE_DNS_NAME="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com
DATA_PARTITION="<your_partition>" # ie:opendes DATA_PARTITION="<your_partition>" # ie:opendes
ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io
TAG="<app_version>" # ie: 0.10.0 TAG="<app_version>" # ie: 0.11.0
# This logs your local Azure CLI in using the configured service principal. # This logs your local Azure CLI in using the configured service principal.
az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
...@@ -75,7 +74,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo ...@@ -75,7 +74,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo
DNS_HOST="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com DNS_HOST="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com
DATA_PARTITION="<your_partition>" # ie:opendes DATA_PARTITION="<your_partition>" # ie:opendes
ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io ACR_REGISTRY="<repository>" # ie: msosdu.azurecr.io
TAG="<app_version>" # ie: 0.10.0 TAG="<app_version>" # ie: 0.11.0
# This logs your local Azure CLI in using the configured service principal. # This logs your local Azure CLI in using the configured service principal.
az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID
...@@ -108,7 +107,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo ...@@ -108,7 +107,7 @@ UNIQUE="<your_osdu_unique>" # ie: demo
AZURE_DNS_NAME="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com AZURE_DNS_NAME="<your_osdu_fqdn>" # ie: osdu-$UNIQUE.contoso.com
DATA_PARTITION="<your_partition>" # ie:opendes DATA_PARTITION="<your_partition>" # ie:opendes
ACR_REGISTRY="<your_acr_fqdn>" # ie: myacr.azurecr.io ACR_REGISTRY="<your_acr_fqdn>" # ie: myacr.azurecr.io
TAG="<app_version>" # ie: 0.10.0 TAG="<app_version>" # ie: 0.11.0
DAG_TASK_IMAGE="segy-to-zgy-conversion-dag" # i.e. name for the image in ACR DAG_TASK_IMAGE="segy-to-zgy-conversion-dag" # i.e. name for the image in ACR
AZURE_TENANT_ID="<azure tenant>" AZURE_TENANT_ID="<azure tenant>"
......
...@@ -13,12 +13,12 @@ ...@@ -13,12 +13,12 @@
// limitations under the License. // limitations under the License.
data "azuread_service_principal" "main" { data "azuread_service_principal" "main" {
count = length(local.api_names) count = var.aad_client_id != "" ? 0 : length(local.api_names)
display_name = local.api_names[count.index] display_name = local.api_names[count.index]
} }
resource "azuread_application" "main" { resource "azuread_application" "main" {
count = var.enable_bring_your_own_ad_app ? 0 : 1 count = var.aad_client_id != "" ? 0 : 1
name = var.name name = var.name
homepage = coalesce(var.homepage, local.homepage) homepage = coalesce(var.homepage, local.homepage)
identifier_uris = local.identifier_uris identifier_uris = local.identifier_uris
...@@ -60,14 +60,14 @@ resource "azuread_application" "main" { ...@@ -60,14 +60,14 @@ resource "azuread_application" "main" {
} }
resource "random_password" "main" { resource "random_password" "main" {
count = !var.enable_bring_your_own_ad_app && var.password == "" ? 1 : 0 count = var.aad_client_id == "" && var.password == "" ? 1 : 0
length = 32 length = 32
special = false special = false
} }
resource "azuread_application_password" "main" { resource "azuread_application_password" "main" {
count = !var.enable_bring_your_own_ad_app && var.password != null ? 1 : 0 count = var.aad_client_id == "" && var.password != null ? 1 : 0
application_object_id = var.enable_bring_your_own_ad_app ? null : azuread_application.main[0].object_id application_object_id = var.aad_client_id != "" ? null : azuread_application.main[0].object_id
value = coalesce(var.password, random_password.main[0].result) value = coalesce(var.password, random_password.main[0].result)
end_date = local.end_date end_date = local.end_date
......
...@@ -13,22 +13,22 @@ ...@@ -13,22 +13,22 @@
// limitations under the License. // limitations under the License.
output "name" { output "name" {
value = var.enable_bring_your_own_ad_app ? null : azuread_application.main[0].name value = var.aad_client_id != "" ? null : azuread_application.main[0].name
description = "The display name of the application." description = "The display name of the application."
} }
output "id" { output "id" {
value = var.enable_bring_your_own_ad_app ? null : azuread_application.main[0].application_id value = var.aad_client_id != "" ? null : azuread_application.main[0].application_id
description = "The ID of the application." description = "The ID of the application."
} }
output "object_id" { output "object_id" {
value = var.enable_bring_your_own_ad_app ? null : azuread_application.main[0].object_id value = var.aad_client_id != "" ? null : azuread_application.main[0].object_id
description = "The object ID of the application." description = "The object ID of the application."
} }
output "roles" { output "roles" {
value = var.enable_bring_your_own_ad_app ? null : { value = var.aad_client_id != "" ? null : {
for r in azuread_application.main[0].app_role : for r in azuread_application.main[0].app_role :
r.display_name => { r.display_name => {
id = r.id id = r.id
...@@ -42,7 +42,7 @@ output "roles" { ...@@ -42,7 +42,7 @@ output "roles" {
} }
output "password" { output "password" {
value = var.enable_bring_your_own_ad_app ? null : azuread_application_password.main.0.value value = var.aad_client_id != "" ? null : azuread_application_password.main.0.value
sensitive = true sensitive = true
description = "The password for the application." description = "The password for the application."
} }
...@@ -12,10 +12,10 @@ ...@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
variable "enable_bring_your_own_ad_app" { variable "aad_client_id" {
description = "Feature flag for BYOA" description = "Existing Application AppId."
default = false type = string
type = bool default = ""
} }
variable "name" {