infra-automation.md 8.3 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Deploy Infrastructure

__Configure Azure DevOps Service Connection__

- Configure an [ARM Resources Service Connection](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/connect-to-azure?view=azure-devops) for the desired subscription.
  - Scope should be to the desired Subscription but do not apply scope to a Resource Group

```bash
SERVICE_CONNECTION_NAME=osdu-mvp-$UNIQUE
export AZURE_DEVOPS_EXT_AZURE_RM_SERVICE_PRINCIPAL_KEY=$ARM_CLIENT_SECRET

az devops service-endpoint azurerm create \
  --name $SERVICE_CONNECTION_NAME \
  --azure-rm-tenant-id $ARM_TENANT_ID \
  --azure-rm-subscription-id $ARM_SUBSCRIPTION_ID \
  --azure-rm-subscription-name $(az account show --subscription $ARM_SUBSCRIPTION_ID --query name -otsv) \
  --azure-rm-service-principal-id $ARM_CLIENT_ID \
  -ojsonc
```


__Setup and Configure the ADO Library `Infrastructure Pipeline Variables`__

This variable group will be used to hold the common values for infrastructure to be built regardless of a specified environment.

  | Variable | Value |
  |----------|-------|
28
  | AGENT_IMAGE | ubuntu-20.04 |
29
30
31
32
33
34
35
36
37
38
39
40
  | BUILD_ARTIFACT_NAME | infra-templates |
  | SERVICE_CONNECTION_NAME | <your_service_connection_name> |
  | TF_VAR_elasticsearch_secrets_keyvault_name | osducommon<your_unique>-kv |
  | TF_VAR_elasticsearch_secrets_keyvault_resource_group | osdu-common-<your_unique> |
  | TF_VAR_remote_state_account | osducommon<your_unique> |
  | TF_VAR_remote_state_container | remote-state-container |

```bash
az pipelines variable-group create \
  --name "Infrastructure Pipeline Variables" \
  --authorize true \
  --variables \
41
  AGENT_IMAGE="ubuntu-20.04" \
42
43
44
45
46
47
48
49
50
51
52
53
54
55
  BUILD_ARTIFACT_NAME="infra-templates" \
  TF_VAR_elasticsearch_secrets_keyvault_name=$COMMON_VAULT  \
  TF_VAR_elasticsearch_secrets_keyvault_resource_group=osdu-common-${UNIQUE} \
  TF_VAR_remote_state_account=$TF_VAR_remote_state_account \
  TF_VAR_remote_state_container="remote-state-container" \
  SERVICE_CONNECTION_NAME=$SERVICE_CONNECTION_NAME \
  -ojson
```


__Setup and Configure the ADO Library `Infrastructure Pipeline Variables - demo`__

This variable group will be used to hold the common values for a specific infrastructure environment to be built. There is an implied naming convention to this Variable group `demo` relates to the environment name.  Additionally you can specify and override the region locations here.

56
57
__TF_VAR_istio_int_load_balancer_ip__ has been set by default in terraform code, however, if you are on brownfield deployments, to avoid downtime, you may want to setup this in the Variable Group as env var or in your `terraform.tfvars` file to match your environment setup.

58
59
60
61
62
63

  | Variable | Value |
  |----------|-------|
  | ARM_SUBSCRIPTION_ID | <your_subscription_id> |
  | TF_VAR_aks_agent_vm_count | 3 |
  | TF_VAR_central_resources_workspace_name | cr-demo |
64
65
  | TF_VAR_service_resources_workspace_name | sr-demo |
  | TF_VAR_data_partition_resources_workspace_name | dp1-demo |
66
67
68
69
  | TF_VAR_cosmosdb_replica_location | eastus2 |
  | TF_VAR_data_partition_name | opendes |
  | TF_VAR_data_resources_workspace_name | dr-demo |
  | TF_VAR_elasticsearch_version | <your_elastic_version> |
Daniel Scholl's avatar
Daniel Scholl committed
70
  | TF_VAR_gitops_branch | <desired_branch> |
71
72
73
74
75
76
77
  | TF_VAR_gitops_path | providers/azure/hld-registry |
  | TF_VAR_gitops_ssh_url | git@<your_flux_repo> |
  | TF_VAR_principal_appId | <your_principal_appId> |
  | TF_VAR_principal_name | <your_principal_name> |
  | TF_VAR_principal_objectId | <your_principal_objectId> |
  | TF_VAR_principal_password | <your_principal_password> |
  | TF_VAR_resource_group_location | centralus |
78
  | TF_VAR_deploy_dp_airflow | false |
79
80
  | TF_VAR_aks_dns_host  | <your_dns_host> |
  | (Optional) TF_VAR_istio_int_load_balancer_ip  | 10.10.255.253 |
81
82
  | (optional) TF_VAR_secret_kv_enabled | false |
  | (optional) TF_VAR_reservoir_ddms | {enabled=false,sku="B_Gen5_1"} |
83
  
84
85
86
87
88
89

```bash
ENVIRONMENT="demo"
REGION="centralus"
REGION_PAIR="eastus2"
PARTITION_NAME="opendes"
Daniel Scholl's avatar
Daniel Scholl committed
90
ELASTIC_VERSION="7.11.1"
91
GIT_REPO=git@ssh.dev.azure.com:v3/${ADO_ORGANIZATION}/${ADO_PROJECT}/k8-gitops-manifests
92
DNS_HOSTNAME=myhostname.contoso.com # Replace for your hostname to be used
93
94
95
96
97
98
99
100

az pipelines variable-group create \
  --name "Infrastructure Pipeline Variables - ${ENVIRONMENT}" \
  --authorize true \
  --variables \
  ARM_SUBSCRIPTION_ID="${ARM_SUBSCRIPTION_ID}" \
  TF_VAR_aks_agent_vm_count=3 \
  TF_VAR_central_resources_workspace_name="cr-${ENVIRONMENT}" \
101
102
  TF_VAR_service_resources_workspace_name="sr-${ENVIRONMENT}" \
  TF_VAR_data_partition_resources_workspace_name="dp1-${ENVIRONMENT}" \
103
104
105
106
  TF_VAR_cosmosdb_replica_location="${REGION_PAIR}" \
  TF_VAR_data_partition_name="${PARTITION_NAME}" \
  TF_VAR_data_resources_workspace_name="dr-${ENVIRONMENT}" \
  TF_VAR_elasticsearch_version="${ELASTIC_VERSION}" \
107
  TF_VAR_gitops_branch="${UNIQUE}" \
108
109
110
111
112
113
114
  TF_VAR_gitops_path="providers/azure/hld-registry" \
  TF_VAR_gitops_ssh_url="${GIT_REPO}" \
  TF_VAR_principal_appId="${TF_VAR_principal_appId}" \
  TF_VAR_principal_name="${TF_VAR_principal_name}" \
  TF_VAR_principal_objectId="${TF_VAR_principal_objectId}" \
  TF_VAR_principal_password="${TF_VAR_principal_password}" \
  TF_VAR_resource_group_location="${REGION}" \
115
  TF_VAR_deploy_dp_airflow="false" \
Vivek Ojha's avatar
Vivek Ojha committed
116
  TF_VAR_aad_client_id="$TF_VAR_application_clientid" \
117
  TF_VAR_secret_kv_enabled="false" \
118
  TF_VAR_aks_dns_host=$DNS_HOSTNAME \
119
120
121
  -ojson
```

122
123
__Enable Airflow Multi partition support__

124
To enable airflow multi partition turn on the feature flag by following the below steps. **This will create infrastructure to support separate airflow cluster for every data partition**. If you don't want to enable it you can skip this step
125

126
127
128
129
130
131
132
1. Go to Pipelines Library in ADO
2. Go to `Infrastructure Pipeline Variables - demo` variable group
3. Add or update the below variable
  
  | Variable | Value |
  |----------|-------|
  | TF_VAR_deploy_dp_airflow | true |
133
  | TF_VAR_ssl_challenge_required | true (if not using BYOC) <br> false (if using BYOC) |
134

135
136
137
138
139
140
141
142
143
__Enable BYOAD__
To enable byoad, turn on the feature flag by following the below steps. If you don't want to create your own AD Application, you can skip it.

1. Go to Pipelines Library in ADO
2. Go to `Infrastructure Pipeline Variables - demo` variable group
3. Add or update the below variable

| Variable | Value |
  |----------|-------|
Vivek Ojha's avatar
Vivek Ojha committed
144
| TF_VAR_aad_client_id | {{application client id of manually created ad application}} |
145

146

147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
__Setup and Configure the ADO Library `Infrastructure Pipeline Secrets - demo`__
> This should be linked Secrets from Azure Key Vault `osducommon<random>`

  | Variable | Value |
  |----------|-------|
  | elastic-endpoint-dp1-demo | `*********` |
  | elastic-username-dp1-demo | `*********` |
  | elastic-password-dp1-demo | `*********` |


__Setup 2 Secure Files__

[Upload the 2 Secure files](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/secure-files?view=azure-devops).


  - ~/.ssh/osdu_$UNIQUE/azure-aks-gitops-ssh-key
  - ~/.ssh/osdu_$UNIQUE/azure-aks-node-ssh-key.pub



__Execute the pipelines in `osdu-infrastructure`__

> These pipelines need to be executed to completion in the specific order.

1. `infrastructure-central-resources`

  > For the first run of the pipeline approvals will need to be made for the 2 secure files and the Service Connection.

```bash
# Create and Deploy the Pipeline
az pipelines create \
  --name 'infrastructure-central-resources'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
Daniel Scholl's avatar
Daniel Scholl committed
182
  --yaml-path /devops/pipelines/infrastructure-central-resources.yml  \
183
184
185
  -ojson
```

Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
186
3. `infrastructure-service-resources`
187
188
189
190
191
192

  > For the first run of the pipeline approvals will need to be made for the 2 secure files and the Service Connection.

```bash
# Create and Deploy the Pipeline
az pipelines create \
Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
193
  --name 'infrastructure-service-resources'  \
194
195
196
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
197
  --yaml-path /devops/pipelines/infrastructure-service-resources.yml  \
198
199
200
  -ojson
```

Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
201
3. `infrastructure-data-partition`
202
203
204
205
206
207

  > For the first run of the pipeline approvals will need to be made for the 2 secure files and the Service Connection.

```bash
# Create and Deploy the Pipeline
az pipelines create \
Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
208
  --name 'infrastructure-data-partition'  \
209
210
211
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
Igor Zimovets (EPAM)'s avatar
Igor Zimovets (EPAM) committed
212
  --yaml-path /devops/pipelines/infrastructure-data-partition.yml  \
213
214
  -ojson
```