service-automation.md 61.8 KB
Newer Older
1
2
3
4
5
6
7
8
# Create OSDU Service Libraries

__Setup and Configure the ADO Library `Azure - OSDU`__

This variable group will be used to hold the common values for the services to be deployed and relate specifically to settings common to integration tests across all services.  Variable values here that are other variables are being pulled from the Library group accessing the common Key Vault.

| Variable                                      | Value |
|-----------------------------------------------|-------|
Sumra Zafar's avatar
Sumra Zafar committed
9
| ADMIN_EMAIL                                   | <your_sslcert_admin_email>                  |
10
| AGENT_IMAGE                                    | `ubuntu-18.04`                        |
Sumra Zafar's avatar
Sumra Zafar committed
11
12
| AZURE_AD_GUEST_EMAIL                          | `$(ad-guest-email)`                         |
| AZURE_AD_GUEST_OID                            | `$(ad-guest-oid)`                           |
13
| AZURE_AD_OTHER_APP_RESOURCE_ID                | `$(osdu-mvp-<your_unique>-application-clientid)` |
Sumra Zafar's avatar
Sumra Zafar committed
14
15
16
17
18
19
20
| AZURE_AD_OTHER_APP_RESOURCE_OID               | `$(osdu-mvp-<your_unique>-application-oid)` |
| AZURE_AD_USER_EMAIL                           | `$(ad-user-email)`                          |
| AZURE_AD_USER_OID                             | `$(ad-user-oid)`                            |
| AZURE_LEGAL_TOPICNAME                         | `legaltags`                                 |
| DEPLOY_ENV                                    | `empty`                                     |
| EXPIRED_TOKEN                                 | <an_expired_token>                          |
| HOST_URL                                      | `https://<your_fqdn>/`                      |
21
| NO_DATA_ACCESS_TESTER                         | `$(osdu-mvp-<your_unique>-noaccess-clientid)`  |
Sumra Zafar's avatar
Sumra Zafar committed
22
23
24
25
| NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET | `$(osdu-mvp-<your_unique>-noaccess-secret)` |
| PUBSUB_TOKEN                                  | `az`                                        |
| SERVICE_CONNECTION_NAME                       | <your_service_connection_name>              |
| GOOGLE_CLOUD_PROJECT                          | `opendes`                                   |
26
| ENTITLEMENT_URL                               | `https://<your_fqdn>/entitlements/v1/`      |
27
| ENTITLEMENT_V2_URL                            | `https://<your_fqdn>/api/entitlements/v2/`  |
28
29
30
| LEGAL_URL                                     | `https://<your_fqdn>/api/legal/v1/`         |
| STORAGE_URL                                   | `https://<your_fqdn>/api/storage/v2/`       |
| SEARCH_URL                                    | `https://<your_fqdn>/api/search/v2/`        |
Sumra Zafar's avatar
Sumra Zafar committed
31
32
| FILE_URL                                      | `https://<your_fqdn>/api/file/v2`           |
| DELIVERY_URL                                  | `https://<your_fqdn>/api/delivery/v2`       |
Nicholas Karsky's avatar
Nicholas Karsky committed
33
34
| UNIT_URL                                      | `https://<your_fqdn>/api/unit/v2`           |
| CRS_CATALOG_URL                               | `https://<your_fqdn>/api/crs/catalog/v2/`   |
Sumra Zafar's avatar
Sumra Zafar committed
35
| CRS_CONVERSION_URL                            | `https://<your_fqdn>/api/crs/converter/v2/` |
Vivek Ojha's avatar
Vivek Ojha committed
36
| DATASET_URL                                   | `https://<your_fqdn>/api/dataset/v1`           |
harshit aggarwal's avatar
harshit aggarwal committed
37
38
39
40
41
42
| REGISTER_BASE_URL                             | `https://<your_fqdn>/`                      |
| ACL_OWNERS                                    | `data.test1`                                |
| ACL_VIEWERS                                   | `data.test1`                                |
| DATA_PARTITION_ID                             | `opendes`                                   |
| TENANT_NAME                                   | `opendes`                                   |
| VENDOR                                        | `azure`                                     |
harshit aggarwal's avatar
harshit aggarwal committed
43
| OSDU_TENANT                                   | `opendes`                                   |
Nicholas Karsky's avatar
Nicholas Karsky committed
44
| NOTIFICATION_REGISTER_BASE_URL                | `https://<your_fqdn>`                       |
harshit aggarwal's avatar
harshit aggarwal committed
45
46
| NOTIFICATION_BASE_URL                         | `https://<your_fqdn>/api/notification/v1/`  |
| REGISTER_CUSTOM_PUSH_URL_HMAC                 | `https://<your_fqdn>/api/register/v1/test/challenge/1`|
47
| AGENT_IMAGE                                   | `ubuntu-latest`                             | 
48
49
| PROVIDER_NAME                                 | `azure` 
| ENABLE_KEYVAULT_CERT                          | `false`              Set this variable to `true` if you want to use your own certificate from Keyvault certificate - appgw-ssl-cert
harshit aggarwal's avatar
harshit aggarwal committed
50

51
52
53
54
55
56

```bash
ADMIN_EMAIL="<your_cert_admin>"     # ie: admin@email.com
DNS_HOST="<your_ingress_hostname>"  # ie: osdu.contoso.com
SERVICE_CONNECTION_NAME=osdu-mvp-$UNIQUE
INVALID_TOKEN="<an_invalid_token>"
57
PROVIDER_NAME=azure
58
59
60
61
62
63

az pipelines variable-group create \
  --name "Azure - OSDU" \
  --authorize true \
  --variables \
  ADMIN_EMAIL=$ADMIN_EMAIL \
64
  AGENT_IMAGE="ubuntu-18.04" \
65
66
67
68
69
70
71
72
73
74
75
  AZURE_AD_GUEST_EMAIL='$(ad-guest-email)' \
  AZURE_AD_GUEST_OID='$(ad-guest-oid)' \
  AZURE_AD_OTHER_APP_RESOURCE_ID='$(osdu-mvp-'${UNIQUE}'-application-clientid)' \
  AZURE_AD_OTHER_APP_RESOURCE_OID='$(osdu-mvp-'${UNIQUE}'-application-oid)' \
  AZURE_AD_USER_EMAIL='$(ad-user-email)' \
  AZURE_AD_USER_OID='$(ad-user-oid)' \
  AZURE_LEGAL_TOPICNAME="legaltags" \
  DEPLOY_ENV="empty" \
  EXPIRED_TOKEN=$INVALID_TOKEN \
  HOST_URL="https://${DNS_HOST}/" \
  NO_DATA_ACCESS_TESTER='$(osdu-mvp-'${UNIQUE}'-noaccess-clientid)' \
Daniel Scholl's avatar
Daniel Scholl committed
76
  NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET='$(osdu-mvp-'${UNIQUE}'-noaccess-secret)' \
77
78
79
  PUBSUB_TOKEN="az" \
  SERVICE_CONNECTION_NAME=$SERVICE_CONNECTION_NAME \
  GOOGLE_CLOUD_PROJECT="opendes" \
80
  ENTITLEMENT_URL="https://${DNS_HOST}/entitlements/v1/" \
81
  ENTITLEMENT_V2_URL="https://${DNS_HOST}/api/entitlements/v2/" \
82
83
84
  LEGAL_URL="https://${DNS_HOST}/api/legal/v1/" \
  STORAGE_URL="https://${DNS_HOST}/api/storage/v2/" \
  SEARCH_URL="https://${DNS_HOST}/api/search/v2/" \
85
86
  FILE_URL="https://${DNS_HOST}/api/file/v2" \
  DELIVERY_URL="https://${DNS_HOST}/api/delivery/v2/" \
87
  UNIT_URL="https://${DNS_HOST}/api/unit/v2/" \
Nicholas Karsky's avatar
Nicholas Karsky committed
88
  CRS_CATALOG_URL="https://${DNS_HOST}/api/crs/catalog/v2/" \
Sumra Zafar's avatar
Sumra Zafar committed
89
  CRS_CONVERSION_URL="https://${DNS_HOST}/api/crs/converter/v2/" \
Vivek Ojha's avatar
Vivek Ojha committed
90
  DATASET_URL="https://${DNS_HOST}/api/dataset/v1" \
harshit aggarwal's avatar
harshit aggarwal committed
91
  REGISTER_BASE_URL="https://${DNS_HOST}/" \
harshit aggarwal's avatar
harshit aggarwal committed
92
93
94
95
96
  ACL_OWNERS="data.test1" \
  ACL_VIEWERS="data.test1" \
  DATA_PARTITION_ID="opendes" \
  TENANT_NAME="opendes" \
  VENDOR="azure" \
Komal Makkar's avatar
Komal Makkar committed
97
  OSDU_TENANT="opendes" \
Komal Makkar's avatar
Komal Makkar committed
98
  NOTIFICATION_REGISTER_BASE_URL="https://${DNS_HOST}" \
99
100
  NOTIFICATION_BASE_URL="https://${DNS_HOST}/api/notification/v1/" \
  REGISTER_CUSTOM_PUSH_URL_HMAC="https://${DNS_HOST}/api/register/v1/test/challenge/1" \
101
  AGENT_IMAGE="ubuntu-latest" \
102
  PROVIDER_NAME="$PROVIDER_NAME" \
103
  ENABLE_KEYVAULT_CERT="false" \
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
  -ojson
```


__Setup and Configure the ADO Library `Azure - OSDU Secrets`__

This variable group is a linked variable group that links to the Common Key Vault `osducommon<random>` and retrieves secret common settings.

- ad-guest-email
- ad-guest-oid
- ad-user-email
- ad-user-oid
- istio-username
- istio-password
- osdu-mvp-{unique}-application-clientid
- osdu-mvp-{unique}-application-secret
Daniel Scholl's avatar
Daniel Scholl committed
120
- osdu-mvp-{unique}-application-oid
121
122
- osdu-infra-{unique}-noaccess-clientid
- osdu-infra-{unique}-noaccess-oid
Daniel Scholl's avatar
Daniel Scholl committed
123
- osdu-infra-{unique}-noaccess-secret
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139


__Setup and Configure the ADO Library `Azure Target Env - demo`__

This variable group will be used to hold the specific environment values necessary for integration testing of the services being deployed. Variable values here that are other variables are being pulled from the Library group accessing the environment Key Vault.

| Variable | Value |
|----------|-------|
| AZURE_AD_APP_RESOURCE_ID                      | `$(aad-client-id)`                |
| AZURE_DEPLOY_SUBSCRIPTION                     | `$(subscription-id)`              |
| AZURE_LEGAL_SERVICEBUS                        | `$(opendes-sb-connection)`        |
| AZURE_TENANT_ID                               | `$(tenant-id)`                    |
| AZURE_TESTER_SERVICEPRINCIPAL_SECRET          | `$(app-dev-sp-password)`          |
| CONTAINER_REGISTRY_NAME                       | `$(container_registry)`           |
| DNS_HOST                                      | <your_FQDN>                       |
| DOMAIN                                        | `contoso.com`                     |
140
| ROOT_DATA_GROUP_QUOTA                         | `5000`                            |
141
| ELASTIC_ENDPOINT                              | `$(opendes-elastic-endpoint)`     |
Daniel Scholl's avatar
Daniel Scholl committed
142
| ELASTIC_USERNAME                              | `$(opendes-elastic-username)`     |
harshit aggarwal's avatar
harshit aggarwal committed
143
| ELASTIC_PASSWORD                              | `$(opendes-elastic-password)`     |
144
| ENTV2_REDIS_TTL_SECONDS                       | `1`                               |
145
| ENVIRONMENT_NAME                              | <your_environment_name_or_identifier>     |
Daniel Scholl's avatar
Daniel Scholl committed
146
| IDENTITY_CLIENT_ID                            | `$(osdu-identity-id)`             |
147
148
| INTEGRATION_TESTER                            | `$(app-dev-sp-username)`          |
| MY_TENANT                                     | `opendes`                         |
149
| LEGAL_TAG                                     | `opendes-public-usa-dataset-7643990`        |
150
| REDIS_PORT                                    | `6380`                            |
151
152
| STORAGE_ACCOUNT                               | `$(opendes-storage)`              |
| STORAGE_ACCOUNT_KEY                           | `$(opendes-storage-key)`          |
153
154
155
| AZURE_EVENT_SUBSCRIBER_SECRET                 | Subscriber Secret used while performing handshake, the value to be used can be referenced from [here](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/application.properties#L64)                      |
| AZURE_EVENT_SUBSCRIPTION_ID                   | Subscription ID created by Base64 encoding a string formed by concatenating (no spaces) GET /test/challenge/{} endpoint in register service and topic name i.e. recordstopic<br/>  For eg. BASE64(recordstopic + https://{DNS}/api/register/v1/test/challenge/1           |
| AZURE_EVENT_TOPIC_NAME                        | `recordstopic`          |
harshit aggarwal's avatar
harshit aggarwal committed
156
| AZURE_DNS_NAME                                | <your_FQDN>                       |
157
| AZURE_MAPPINGS_STORAGE_CONTAINER              | `osdu-wks-mappings`               |
158
159
| AZURE_COSMOS_KEY                              | `$(opendes-cosmos-primary-key)`   This variable will not be required after the data partition changes|
| AZURE_COSMOS_URL                              | `$(opendes-cosmos-endpoint)`      This variable will not be required after the data partition changes|
Sumra Zafar's avatar
Sumra Zafar committed
160
| SIS_DATA                                      | `$(Pipeline.Workspace)/s/apachesis_setup/SIS_DATA`      This variable should point to your SIS_DATA setup|
161
162
163

```bash
DATA_PARTITION_NAME=opendes
164
LEGAL_TAG=opendes-public-usa-dataset-7643990
165
DNS_HOST="<your_ingress_hostname>"  # ie: osdu.contoso.com
166
ENVIRONMENT_NAME=$UNIQUE
167
168
REDIS_PORT="6380"

169
170
171
172
173
174
175
176
177
178

az pipelines variable-group create \
  --name "Azure Target Env - ${UNIQUE}" \
  --authorize true \
  --variables \
  AZURE_AD_APP_RESOURCE_ID='$(aad-client-id)' \
  AZURE_DEPLOY_SUBSCRIPTION='$(subscription-id)' \
  AZURE_LEGAL_SERVICEBUS='$('${DATA_PARTITION_NAME}'-sb-connection)' \
  AZURE_TENANT_ID='$(tenant-id)' \
  AZURE_TESTER_SERVICEPRINCIPAL_SECRET='$(app-dev-sp-password)' \
179
  CONTAINER_REGISTRY_NAME='$(container-registry)' \
180
181
  DNS_HOST="$DNS_HOST" \
  DOMAIN="contoso.com" \
182
  ROOT_DATA_GROUP_QUOTA="5000" \
183
  ELASTIC_ENDPOINT='$('${DATA_PARTITION_NAME}'-elastic-endpoint)' \
Daniel Scholl's avatar
Daniel Scholl committed
184
185
  ELASTIC_USERNAME='$('${DATA_PARTITION_NAME}'-elastic-username)' \
  ELASTIC_PASSWORD='$('${DATA_PARTITION_NAME}'-elastic-password)' \
186
  ENTV2_REDIS_TTL_SECONDS="1" \
187
  ENVIRONMENT_NAME="$ENVIRONMENT_NAME" \
188
189
190
  IDENTITY_CLIENT_ID='$(identity_id)' \
  INTEGRATION_TESTER='$(app-dev-sp-username)' \
  MY_TENANT="$DATA_PARTITION_NAME" \
191
  LEGAL_TAG="$LEGAL_TAG" \
192
  REDIS_PORT="$REDIS_PORT" \
193
194
  STORAGE_ACCOUNT='$('${DATA_PARTITION_NAME}'-storage)' \
  STORAGE_ACCOUNT_KEY='$('${DATA_PARTITION_NAME}'-storage-key)' \
harshit aggarwal's avatar
harshit aggarwal committed
195
196
197
  AZURE_EVENT_SUBSCRIBER_SECRET="secret" \
  AZURE_EVENT_SUBSCRIPTION_ID="subscriptionId" \
  AZURE_EVENT_TOPIC_NAME="topic name" \
harshit aggarwal's avatar
harshit aggarwal committed
198
  AZURE_DNS_NAME="<your_fqdn>" \
harshit aggarwal's avatar
harshit aggarwal committed
199
200
201
  AZURE_MAPPINGS_STORAGE_CONTAINER="osdu-wks-mappings" \
  AZURE_COSMOS_KEY='$(opendes-cosmos-primary-key)' \
  AZURE_COSMOS_URL='$(opendes-cosmos-endpoint)' \
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
  -ojson
```

__Setup and Configure the ADO Library `Azure Target Env Secrets - demo`__

This variable group is a linked variable group that links to the Environment Key Vault and retrieves secret common settings.

- aad-client-id
- app-dev-sp-id
- app-dev-sp-password
- app-dev-sp-tenant-id
- app-dev-sp-username
- appinsights-key
- base-name-cr
- base-name-sr
- container-registry
- {partition-name}-cosmos-connection
- {partition-name}-cosmos-endpoint
- {partition-name}-cosmos-primary-key
- {partition-name}-elastic-endpoint
- {partition-name}-elastic-password
- {partition-name}-elastic-username
Daniel Scholl's avatar
Daniel Scholl committed
224
225
226
227
- {partition-name}-storage
- {partition-name}-storage-key
- {partition-name}-sb-connection
- {partition-name}-sb-namespace
228
229
230
231
- osdu-identity-id
- subscription-id
- tenant-id

232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
__Setup and Configure the ADO Library `Azure Target Env Data Partition opendes - demo`__

This needs to be configured only if airflow multi partition support is enabled

This variable group will be used to hold the specific environment values necessary for configuring airflow in data partition.

| Variable | Value |
|----------|-------|
| AAD_CLIENT_ID                         | `$(aad-client-id)`                |
| AIRFLOW_DNS_HOST                      | <your_airflow_FQDN>               |
| AZURE_CLIENT_ID                       | `$(app-dev-sp-username)`        |
| AZURE_CLIENT_SECRET                   | `$(app-dev-sp-password)`                    |
| AZURE_TENANT_ID                       | `$(app-dev-sp-tenant-id)`          |
| DP_ENABLE_KEYVAULT_CERT               | `false`              Set this variable to `true` if you want to use your own certificate from Keyvault certificate - istio-ssl-certificate. <br> To configure certificate in keyvault follow the instructions [here](/docs/byoc-dp-airflow-istio-certificate.md)                                         |
| OSDU_SVC_ENDPOINT                     | <your_FQDN>                       |

```bash
DATA_PARTITION_NAME=opendes
DNS_HOST="<your_ingress_hostname>"  # ie: osdu.contoso.com
AIRFLOW_DNS_HOST="<your airflow dns host>"


az pipelines variable-group create \
255
  --name "Azure Target Env Data Partition opendes - ${UNIQUE}" \
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
  --authorize true \
  --variables \
  AAD_CLIENT_ID='$(aad-client-id)' \
  AIRFLOW_DNS_HOST="$AIRFLOW_DNS_HOST" \
  AZURE_CLIENT_ID='$(app-dev-sp-username)' \
  AZURE_TENANT_ID='$(app-dev-sp-tenant-id)' \
  AZURE_CLIENT_SECRET='$(app-dev-sp-password)' \
  DP_ENABLE_KEYVAULT_CERT="false" \
  OSDU_SVC_ENDPOINT="$DNS_HOST" \
  -ojson
```

__Setup and Configure the ADO Library `Azure Target Env Data Partition opendes Secrets - demo`__

This needs to be configured only if airflow multi partition support is enabled

This variable group will be used to hold the specific environment values which are part of keyvault secrets necessary for configuring airflow in data partition.

This variable group should be linked to the keyvault created in data partition resource group

276
277
278
279
280
- aad-client-id
- app-dev-sp-password
- app-dev-sp-tenant-id
- app-dev-sp-username
- appinsights-key
281
282
283
284
285
286
287
- data-partition-tenant-id
- data-partition-subscription-id
- base-name-dp
- management-identity-id
- data-partition-name


288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335

__Setup and Configure the ADO Library `Azure Service Release - partition`__

This variable group is the service specific variables necessary for testing and deploying the `partition` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop/provider/partition-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DPARTITION_BASE_URL=$(HOST_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_OTHER_APP_RESOURCE_ID=$(AZURE_AD_OTHER_APP_RESOURCE_ID) -DENVIRONMENT=HOSTED` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/partition-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_PARTITION_SERVICE_NAME)` |


```bash
az pipelines variable-group create \
  --name "Azure Service Release - partition" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/partition-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DPARTITION_BASE_URL=$(HOST_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_OTHER_APP_RESOURCE_ID=$(AZURE_AD_OTHER_APP_RESOURCE_ID) -DENVIRONMENT=HOSTED' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/partition-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_PARTITION_SERVICE_NAME)' \
  -ojson
```

__Setup and Configure the ADO Library `Azure Service Release - entitlements-azure`__

This variable group is the service specific variables necessary for testing and deploying the `entitlements-azure` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DENTITLEMENT_URL=$(ENTITLEMENT_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DENTITLEMENT_MEMBER_NAME_VALID=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DAZURE_AD_OTHER_APP_RESOURCE_ID=$(AZURE_AD_OTHER_APP_RESOURCE_ID) -DAZURE_AD_OTHER_APP_RESOURCE_OID=$(AZURE_AD_OTHER_APP_RESOURCE_OID) -DDOMAIN=$(DOMAIN) -DEXPIRED_TOKEN=$(EXPIRED_TOKEN) -DENTITLEMENT_GROUP_NAME_VALID=integ.test.data.creator -DENTITLEMENT_MEMBER_NAME_INVALID=InvalidTestAdmin -DAZURE_AD_USER_EMAIL=$(ad-user-email) -DAZURE_AD_USER_OID=$(ad-user-oid) -DAZURE_AD_GUEST_EMAIL=$(ad-guest-email) -DAZURE_AD_GUEST_OID=$(ad-guest-oid)` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/integration-tests` |
| SERVICE_RESOURCE_NAME | `$(AZURE_ENTITLEMENTS_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - entitlements-azure" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DENTITLEMENT_URL=$(ENTITLEMENT_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DENTITLEMENT_MEMBER_NAME_VALID=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DAZURE_AD_OTHER_APP_RESOURCE_ID=$(AZURE_AD_OTHER_APP_RESOURCE_ID) -DAZURE_AD_OTHER_APP_RESOURCE_OID=$(AZURE_AD_OTHER_APP_RESOURCE_OID) -DDOMAIN=$(DOMAIN) -DEXPIRED_TOKEN=$(EXPIRED_TOKEN) -DENTITLEMENT_GROUP_NAME_VALID=integ.test.data.creator -DENTITLEMENT_MEMBER_NAME_INVALID=InvalidTestAdmin -DAZURE_AD_USER_EMAIL=$(ad-user-email) -DAZURE_AD_USER_OID=$(ad-user-oid) -DAZURE_AD_GUEST_EMAIL=$(ad-guest-email) -DAZURE_AD_GUEST_OID=$(ad-guest-oid)' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/integration-tests" \
  SERVICE_RESOURCE_NAME='$(AZURE_ENTITLEMENTS_SERVICE_NAME)' \
  -ojson
```

336
__Setup and Configure the ADO Library `Azure Service Release - entitlements`__
337

338
This variable group is the service specific variables necessary for testing and deploying the `entitlements` service.
339
340
341
342

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop/provider/entitlements-v2-azure` |
343
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DENTITLEMENT_V2_URL=$(ENTITLEMENT_V2_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID)"`  |
344
345
346
347
348
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/entitlements-v2-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_ENTITLEMENTS_V2_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
349
  --name "Azure Service Release - entitlements" \
350
351
352
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/entitlements-v2-azure" \
353
  MAVEN_INTEGRATION_TEST_OPTIONS='-DargLine="-DENTITLEMENT_V2_URL=$(ENTITLEMENT_V2_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID)"' \
354
355
356
357
358
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/entitlements-v2-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_ENTITLEMENTS_V2_SERVICE_NAME)' \
  -ojson
```

359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
__Setup and Configure the ADO Library `Azure Service Release - legal`__

This variable group is the service specific variables necessary for testing and deploying the `legal` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/legal-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DHOST_URL=$(LEGAL_URL) -DENTITLEMENT_URL=$(ENTITLEMENT_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DAZURE_LEGAL_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_LEGAL_STORAGE_KEY=$(STORAGE_ACCOUNT_KEY) -DAZURE_LEGAL_SERVICEBUS=$(AZURE_LEGAL_SERVICEBUS) -DAZURE_LEGAL_TOPICNAME=$(AZURE_LEGAL_TOPICNAME)` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/legal-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_LEGAL_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - legal" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/legal-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DHOST_URL=$(LEGAL_URL) -DENTITLEMENT_URL=$(ENTITLEMENT_URL) -DMY_TENANT=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DAZURE_LEGAL_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_LEGAL_STORAGE_KEY=$(STORAGE_ACCOUNT_KEY) -DAZURE_LEGAL_SERVICEBUS=$(AZURE_LEGAL_SERVICEBUS) -DAZURE_LEGAL_TOPICNAME=$(AZURE_LEGAL_TOPICNAME)' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/legal-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_LEGAL_SERVICE_NAME)' \
  -ojson
```


__Setup and Configure the ADO Library `Azure Service Release - storage`__

This variable group is the service specific variables necessary for testing and deploying the `storage` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/storage-azure` |
Daniel Scholl's avatar
Daniel Scholl committed
390
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=empty` |
391
392
393
394
395
396
397
398
399
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/storage-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_STORAGE_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - storage" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/storage-azure" \
Daniel Scholl's avatar
Daniel Scholl committed
400
  MAVEN_INTEGRATION_TEST_OPTIONS='-DSTORAGE_URL=$(STORAGE_URL) -DLEGAL_URL=$(LEGAL_URL) -DTENANT_NAME=$(MY_TENANT) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DDOMAIN=$(DOMAIN) -DPUBSUB_TOKEN=$(PUBSUB_TOKEN) -DDEPLOY_ENV=empty' \
401
402
403
404
405
406
407
408
409
410
411
412
413
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/storage-test-azure" \
  SERVICE_RESOURCE_NAME='$(AZURE_STORAGE_SERVICE_NAME)' \
  -ojson
```


__Setup and Configure the ADO Library `Azure Service Release - indexer-service`__

This variable group is the service specific variables necessary for testing and deploying the `indexer-service` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/indexer-azure` |
Daniel Scholl's avatar
Daniel Scholl committed
414
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DHOST=$(HOST_URL) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US` |
415
416
417
418
419
420
421
422
423
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/indexer-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_INDEXER_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - indexer-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/indexer-azure" \
Daniel Scholl's avatar
Daniel Scholl committed
424
  MAVEN_INTEGRATION_TEST_OPTIONS='-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -Daad_client_id=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DENVIRONMENT=CLOUD -DLEGAL_TAG=opendes-public-usa-dataset-7643990 -DOTHER_RELEVANT_DATA_COUNTRIES=US' \
425
426
427
428
429
430
431
432
433
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/indexer-test-azure" \
  SERVICE_RESOURCE_NAME='$(AZURE_INDEXER_SERVICE_NAME)' \
  -ojson
```



__Setup and Configure the ADO Library `Azure Service Release - search-service`__

434
This variable group is the service specific variables necessary for testing and deploying the `search` service.
435
436
437
438

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/search-azure` |
Daniel Scholl's avatar
Daniel Scholl committed
439
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)` |
440
441
442
443
444
445
446
447
448
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/integration-tests/search-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_SEARCH_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - search-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/search-azure" \
Daniel Scholl's avatar
Daniel Scholl committed
449
  MAVEN_INTEGRATION_TEST_OPTIONS='-DSEARCH_HOST=$(SEARCH_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID)  -DSTORAGE_HOST=$(STORAGE_URL) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PORT=$(ELASTIC_PORT) -DELASTIC_USER_NAME=$(ELASTIC_USERNAME) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=othertenant2 -DENTITLEMENTS_DOMAIN=$(DOMAIN)' \
450
451
452
453
454
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/integration-tests/search-test-azure" \
  SERVICE_RESOURCE_NAME='$(AZURE_SEARCH_SERVICE_NAME)' \
  -ojson
```

455
456
457
458
459
460
461
__Setup and Configure the ADO Library `Azure Service Release - file`__

This variable group is the service specific variables necessary for testing and deploying the `file` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/file-azure` |
462
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DFILE_SERVICE_HOST=$(FILE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTAGING_CONTAINER_NAME=file-staging-area -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0" -DDATA_PARTITION_ID=$(MY_TENANT)` |
463
464
465
466
467
468
469
470
471
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/file-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_FILE_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - file" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/file-azure" \
472
  MAVEN_INTEGRATION_TEST_OPTIONS='-DFILE_SERVICE_HOST=$(FILE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTAGING_CONTAINER_NAME=file-staging-area -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0" -DDATA_PARTITION_ID=$(MY_TENANT)' \
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/file-test-azure" \
  SERVICE_RESOURCE_NAME='$(AZURE_FILE_SERVICE_NAME)' \
  -ojson
```

__Setup and Configure the ADO Library `Azure Service Release - delivery`__

This variable group is the service specific variables necessary for testing and deploying the `delivery` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/delivery-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DDOMAIN=$(DOMAIN) -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DLEGAL_HOST=$(LEGAL_URL) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=common -DOTHER_RELEVANT_DATA_COUNTRIES=US -DLEGAL_TAG=opendes-public-usa-dataset-1 -DSEARCH_HOST=$(SEARCH_URL) -DSTORAGE_HOST=$(STORAGE_URL) -DDELIVERY_HOST=$(DELIVERY_URL)"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/delivery-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_DELIVERY_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - delivery" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/delivery-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DargLine="-DDOMAIN=$(DOMAIN) -DENTITLEMENTS_DOMAIN=$(DOMAIN) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DLEGAL_HOST=$(LEGAL_URL) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(MY_TENANT) -DDEFAULT_DATA_PARTITION_ID_TENANT2=common -DOTHER_RELEVANT_DATA_COUNTRIES=US -DLEGAL_TAG=opendes-public-usa-dataset-1 -DSEARCH_HOST=$(SEARCH_URL) -DSTORAGE_HOST=$(STORAGE_URL) -DDELIVERY_HOST=$(DELIVERY_URL)"' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/delivery-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_DELIVERY_SERVICE_NAME)' \
  -ojson
```

Sumra Zafar's avatar
Sumra Zafar committed
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
__Setup and Configure the ADO Library `Azure Service Release - schema-service`__

This variable group is the service specific variables necessary for testing and deploying the `schema` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/schema-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DPRIVATE_TENANT1=$(TENANT_NAME) -DPRIVATE_TENANT2=tenant2 -DSHARED_TENANT=$(TENANT_NAME) -DVENDOR=$(VENDOR) -DHOST=https://$(DNS_HOST)"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/schema-test-core/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_SCHEMA_SERVICE_NAME)` |
| AZURE_DEPLOYMENTS_SUBDIR | `drop/deployments/scripts/azure` |
| AZURE_DEPLOYMENTS_SCRIPTS_SUBDIR | `drop/deployments/scripts` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - schema-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/schema-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS=`-DargLine="-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DPRIVATE_TENANT1=$(TENANT_NAME) -DPRIVATE_TENANT2=tenant2 -DSHARED_TENANT=$(TENANT_NAME) -DVENDOR=$(VENDOR) -DHOST=https://$(DNS_HOST)"` \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/schema-test-core/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_SCHEMA_SERVICE_NAME)' \
  AZURE_DEPLOYMENTS_SUBDIR="drop/deployments/scripts/azure" \
  AZURE_DEPLOYMENTS_SCRIPTS_SUBDIR="drop/deployments/scripts" \
  -ojson
```

528
529
530
531
532
533
534
535
536
537
538
__Setup and Configure the ADO Library `Azure Service Release - unit-service`__

This variable group is the service specific variables necessary for testing and deploying the `unit` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/unit-azure/unit-aks` |


```bash
az pipelines variable-group create \
539
  --name "Azure Service Release - unit-service" \
540
541
542
543
544
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/unit-azure/unit-aks" \
  -ojson
```
545

Sumra Zafar's avatar
Sumra Zafar committed
546
547
548
549
550
551
__Setup and Configure the ADO Library `Azure Service Release - crs-catalog-service`__

This variable group is the service specific variables necessary for testing and deploying the `crs-catalog` service.

| Variable | Value |
|----------|-------|
552
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop/provider/crs-catalog-azure/crs-catalog-aks` |
Sumra Zafar's avatar
Sumra Zafar committed
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584

No Test Path is needed since the service has python tests

```bash
az pipelines variable-group create \
  --name "Azure Service Release - crs-catalog-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/crs-catalog-azure/crs-catalog-aks" \
  -ojson
```

__Setup and Configure the ADO Library `Azure Service Release - crs-conversion-service`__

This variable group is the service specific variables necessary for testing and deploying the `crs-conversion` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop/provider/crs-converter-azure/crs-converter-aks` |

No Test Path is needed since the service has python tests

```bash
az pipelines variable-group create \
  --name "Azure Service Release - crs-conversion-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/crs-converter-azure/crs-converter-aks" \
  -ojson
```


harshit aggarwal's avatar
harshit aggarwal committed
585
586
587
588
589
590
591
__Setup and Configure the ADO Library `Azure Service Release - register`__

This variable group is the service specific variables necessary for testing and deploying the `register` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/register-azure` |
592
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DREGISTER_BASE_URL=$(REGISTER_BASE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DENVIRONMENT=DEV -DSUBSCRIBER_SECRET=$(AZURE_EVENT_SUBSCRIBER_SECRET) -DTEST_TOPIC_NAME=$(AZURE_EVENT_TOPIC_NAME) -DSUBSCRIPTION_ID=$(AZURE_EVENT_SUBSCRIPTION_ID)"` |
harshit aggarwal's avatar
harshit aggarwal committed
593
594
595
596
597
598
599
600
601
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/register-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_REGISTER_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - register" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/register-azure" \
602
  MAVEN_INTEGRATION_TEST_OPTIONS='-DargLine="-DREGISTER_BASE_URL=$(REGISTER_BASE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DENVIRONMENT=DEV -DSUBSCRIBER_SECRET=$(AZURE_EVENT_SUBSCRIBER_SECRET) -DTEST_TOPIC_NAME=$(AZURE_EVENT_TOPIC_NAME) -DSUBSCRIPTION_ID=$(AZURE_EVENT_SUBSCRIPTION_ID)"' \
harshit aggarwal's avatar
harshit aggarwal committed
603
604
605
606
607
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/register-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_REGISTER_SERVICE_NAME)' \
  -ojson
```

harshit aggarwal's avatar
harshit aggarwal committed
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
__Setup and Configure the ADO Library `Azure Service Release - wks`__

This variable group is the service specific variables necessary for testing and deploying the `wks` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/wks-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DVENDOR=$(VENDOR) -DSTORAGE_URL=$(STORAGE_URL) -DHOST=https://$(DNS_HOST) -DLEGAL_TAG=$(LEGAL_TAG) -DDOMAIN=$(DOMAIN) -DACL_OWNERS=$(ACL_OWNERS) -DACL_VIEWERS=$(ACL_VIEWERS) -DDATA_PARTITION_ID=$(DATA_PARTITION_ID) -DTENANT_NAME=$(TENANT_NAME)"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/wks-test-core/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_WKS_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - wks" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/wks-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DargLine="-DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DAZURE_TESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DVENDOR=$(VENDOR) -DSTORAGE_URL=$(STORAGE_URL) -DHOST=https://$(DNS_HOST) -DLEGAL_TAG=$(LEGAL_TAG) -DDOMAIN=$(DOMAIN) -DACL_OWNERS=$(ACL_OWNERS) -DACL_VIEWERS=$(ACL_VIEWERS) -DDATA_PARTITION_ID=$(DATA_PARTITION_ID) -DTENANT_NAME=$(TENANT_NAME)"' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/wks-test-core/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_WKS_SERVICE_NAME)' \
  -ojson
```


Komal Makkar's avatar
Komal Makkar committed
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
__Setup and Configure the ADO Library `Azure Service Release - notification`__

This variable group is the service specific variables necessary for testing and deploying the `notification` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/notification-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine="-DNOTIFICATION_REGISTER_BASE_URL=$(NOTIFICATION_REGISTER_BASE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DENVIRONMENT=DEV -DHMAC_SECRET=$(AZURE_EVENT_SUBSCRIBER_SECRET) -DTOPIC_ID=$(AZURE_EVENT_TOPIC_NAME) -DNOTIFICATION_BASE_URL=$(NOTIFICATION_BASE_URL) -DREGISTER_CUSTOM_PUSH_URL_HMAC=$(REGISTER_CUSTOM_PUSH_URL_HMAC) -DOSDU_TENANT=$(OSDU_TENANT)"` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/notification-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_NOTIFICATION_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - notification" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/notification-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS=`-DargLine="-DNOTIFICATION_REGISTER_BASE_URL=$(NOTIFICATION_REGISTER_BASE_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DENVIRONMENT=DEV -DHMAC_SECRET=$(AZURE_EVENT_SUBSCRIBER_SECRET) -DTOPIC_ID=$(AZURE_EVENT_TOPIC_NAME) -DNOTIFICATION_BASE_URL=$(NOTIFICATION_BASE_URL) -DREGISTER_CUSTOM_PUSH_URL_HMAC=$(REGISTER_CUSTOM_PUSH_URL_HMAC) -DOSDU_TENANT=$(OSDU_TENANT)"` \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/notification-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_NOTIFICATION_SERVICE_NAME)' \
  -ojson
```

655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
__Setup and Configure the ADO Library `Azure Service Release - ingestion-workflow`__

This variable group is the service specific variables necessary for testing and deploying the `ingestion-workflow` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/workflow-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine=""` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/workflow-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(AZURE_INGESTION_WORKFLOW_SERVICE_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - ingestion-workflow" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/workflow-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS=`-DargLine=""` \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/workflow-test-azure/pom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_INGESTION_WORKFLOW_SERVICE_NAME)' \
  -ojson
```

Vivek Ojha's avatar
Vivek Ojha committed
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
__Setup and Configure the ADO Library `Azure Service Release - dataset`__

This variable group is the service specific variables necessary for testing and deploying the `dataset` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/dataset-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DDATASET_HOST=$(DATASET_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTAGING_CONTAINER_NAME=dataset-staging-area -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0" -DDATA_PARTITION_ID=$(MY_TENANT)` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/dataset-test-azure` |
| SERVICE_RESOURCE_NAME | `$(AZURE_DATASET_NAME)` |

```bash
az pipelines variable-group create \
  --name "Azure Service Release - dataset" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/dataset-azure" \
  MAVEN_INTEGRATION_TEST_OPTIONS='-DDATASET_HOST=$(DATASET_URL) -DAZURE_AD_TENANT_ID=$(AZURE_TENANT_ID) -DINTEGRATION_TESTER=$(INTEGRATION_TESTER) -DTESTER_SERVICEPRINCIPAL_SECRET=$(AZURE_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_AD_APP_RESOURCE_ID=$(AZURE_AD_APP_RESOURCE_ID) -DSTAGING_CONTAINER_NAME=dataset-staging-area -DNO_DATA_ACCESS_TESTER=$(NO_DATA_ACCESS_TESTER) -DNO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET=$(NO_DATA_ACCESS_TESTER_SERVICEPRINCIPAL_SECRET) -DAZURE_STORAGE_ACCOUNT=$(STORAGE_ACCOUNT) -DUSER_ID=osdu-user -DEXIST_FILE_ID=8900a83f-18c6-4b1d-8f38-309a208779cc -DTIME_ZONE="UTC+0" -DDATA_PARTITION_ID=$(MY_TENANT)' \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/dataset-test-azure" \
  SERVICE_RESOURCE_NAME='$(AZURE_DATASET_NAME)' \
  -ojson

700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
__Setup and Configure the ADO Library `Azure Service Release - seismic-store-service`__

This variable group is the service specific variables necessary for testing and deploying the `seismic-store-service` service.

| Variable                         | Value                                                                                   |
|----------------------------------|-----------------------------------------------------------------------------------------|
| e2eAdminEmail                    | <your_sslcert_admin_email>                                                              |
| e2eDataPartition                 | `opendes`                                                                               |
| e2eLegaltag01                    | `opendes-public-usa-dataset-7643990`                                                    |
| e2eLegaltag02                    | `opendes-dps-integration-test-valid2-legal-tag`                                         |
| e2eSubproject                    | `demo`                                                                                  |
| e2eSubprojectLongname            | `looooooooooooooooooooooooooooooooooooooooooooooooooooongnaaaaaaaaaaaaaaaaaaaameeeeeee` |
| e2eTenant                        | `opendes`                                                                               |
| PORT                             | `80`                                                                                    |
| REPLICA_COUNT                    | `1`                                                                                     |
| serviceUrlSuffix                 | `seistore-svc/api/v3`                                                                   |
716
717
718
| hldRegPath                       | `providers/azure/hld-registry`                                                          |
| utest_mount_dir                  | `/service`                                                                              |
| utest_runtime_image              | `seistore-svc-runtime`                                                                  |
719
720
721
722
723
724
725
726
727
728
729
730

```bash
e2eAdminEmail="<your_cert_admin>"     # ie: admin@email.com
e2eDataPartition=opendes
e2eLegaltag01=opendes-public-usa-dataset-7643990
e2eLegaltag02=opendes-dps-integration-test-valid2-legal-tag
e2eSubproject=demo
e2eSubprojectLongname=looooooooooooooooooooooooooooooooooooooooooooooooooooongnaaaaaaaaaaaaaaaaaaaameeeeeee
e2eTenant=opendes
PORT="80"
REPLICA_COUNT="1"
serviceUrlSuffix="seistore-svc/api/v3"
731
732
733
hldRegPath="providers/azure/hld-registry"
utest_mount_dir="/service"
utest_runtime_image=seistore-svc-runtime
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748

az pipelines variable-group create \
  --name "Azure Service Release - seismic-store-service" \
  --authorize true \
  --variables \
  e2eAdminEmail=${e2eAdminEmail} \
  e2eDataPartition=${e2eDataPartition} \
  e2eLegaltag01=${e2eLegaltag01} \
  e2eLegaltag02=${e2eLegaltag02} \
  e2eSubproject=${e2eSubproject} \
  e2eSubprojectLongname=${e2eSubprojectLongname} \
  e2eTenant=${e2eTenant} \
  PORT='${PORT}' \
  REPLICA_COUNT='${REPLICA_COUNT}' \
  serviceUrlSuffix='${serviceUrlSuffix}' \
749
750
751
  hldRegPath='${hldRegPath}' \
  utest_mount_dir='${utest_mount_dir}' \
  utest_runtime_image=${utest_runtime_image} \
752
753
  -ojson
```
Sumra Zafar's avatar
Sumra Zafar committed
754

Sumra Zafar's avatar
Sumra Zafar committed
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
__Setup and Configure the ADO Library `Azure Service Release - ingestion-service`__

This variable group is the service specific variables necessary for testing and deploying the `ingestion` service.

| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH     | `drop/provider/ingest-azure` |

No Test Path is needed since the service has python tests

```bash
az pipelines variable-group create \
  --name "Azure Service Release - ingestion-service" \
  --authorize true \
  --variables \
  MAVEN_DEPLOY_POM_FILE_PATH="drop/provider/ingest-azure" 
  MAVEN_INTEGRATION_TEST_OPTIONS=`-DargLine=""` \
  MAVEN_INTEGRATION_TEST_POM_FILE_PATH="drop/deploy/testing/ingest-test-azurepom.xml" \
  SERVICE_RESOURCE_NAME='$(AZURE_INGESTION_SERVICE_NAME)' \
  -ojson
```


778
779
780
781
782
783
784
__Create the Chart Pipelines__

Create the pipelines and run things in this exact order.

1. Add a Pipeline for __chart-osdu-common__ to deploy common components.

    _Repo:_ `infra-azure-provisioning`
Daniel Scholl's avatar
Daniel Scholl committed
785
    _Path:_ `/devops/pipelines/chart-osdu-common.yml`
786
787
    _Validate:_ https://<your_dns_name> is alive.

Daniel Scholl's avatar
Daniel Scholl committed
788
789
790
791
792
793
```bash
az pipelines create \
  --name 'chart-osdu-common'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
Daniel Scholl's avatar
Daniel Scholl committed
794
  --yaml-path /devops/pipelines/chart-osdu-common.yml  \
Daniel Scholl's avatar
Daniel Scholl committed
795
796
  -ojson
```
797
798
799
800

2. Add a Pipeline for __chart-osdu-istio__  to deploy Istio components.

    _Repo:_ `infra-azure-provisioning`
Daniel Scholl's avatar
Daniel Scholl committed
801
    _Path:_ `/devops/pipelines/chart-osdu-istio.yml`
802
803
    _Validate:_ Pods are running in Istio Namespace.

Daniel Scholl's avatar
Daniel Scholl committed
804
805
806
807
808
809
810
811
812
813
```bash
az pipelines create \
  --name 'chart-osdu-istio'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/pipelines/chart-osdu-istio.yml  \
  -ojson
```

814
815
816
817

3. Add a Pipeline for __chart-osdu-istio-auth__  to deploy Istio Authorization Policies.

    _Repo:_ `infra-azure-provisioning`
Daniel Scholl's avatar
Daniel Scholl committed
818
    _Path:_ `/devops/pipelines/chart-osdu-istio-auth.yml`
819
820
    _Validate:_ Authorization Policies exist in osdu namespace.

Daniel Scholl's avatar
Daniel Scholl committed
821
822
823
824
825
826
827
828
829
```bash
az pipelines create \
  --name 'chart-osdu-istio-auth'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/pipelines/chart-osdu-istio-auth.yml  \
  -ojson
```
830

831
4. Add a Pipeline for __chart-osdu-airflow__  to deploy airflow.
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846

    _Repo:_ `infra-azure-provisioning`
    _Path:_ `/devops/pipelines/chart-airflow.yml`
    _Validate:_ Airflow Pods are running except for airflow-setup-default-user which is a job pod.

```bash
az pipelines create \
  --name 'chart-airflow'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/pipelines/chart-airflow.yml  \
  -ojson
```

847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
5. Add a Pipeline for __chart-osdu-airflow-opendes__  to deploy airflow in data partition if it is enabled.

    _Repo:_ `infra-azure-provisioning`
    _Path:_ `/charts/airflow/pipeline-dp.yml`
    _Validate:_ Airflow Pods are running except for airflow-setup-default-user which is a job pod.

```bash
az pipelines create \
  --name 'chart-airflow-opendes'  \
  --repository infra-azure-provisioning  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /charts/airflow/pipeline-dp.yml  \
  -ojson
```

863
864
865
866
867
868
869
870
871
872
__Create the Service Pipelines__

Create the pipelines and run things in this exact order.

1. Add a Pipeline for __service-partition__  to deploy the Partition Service.

    _Repo:_ `partition`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/partition/v1/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
873
874
875
876
877
878
879
880
881
882
```bash
az pipelines create \
  --name 'service-partition'  \
  --repository partition  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

883
**Stop here**. Before you continue, you must register your partition with the Data Partition API by following the instructions [here](../tools/rest/README.md) to configure your IDE to make authenticated requests to your OSDU instance and send the API request located [here](../tools/rest/partition.http) (createPartition).
884

885
886
887
888
889
890
891
2. Add a Pipeline for __service-entitlements-azure__  to deploy the Entitlements Service.
    > This pipeline may have to be run twice for integration tests to pass due to a preload data issue.

    _Repo:_ `entitlements-azure`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/entitlements/v1/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
892
893
894
895
896
897
898
899
900
901
```bash
az pipelines create \
  --name 'service-entitlements-azure'  \
  --repository entitlements-azure  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

902
903
904
905
906
907
908

3. Add a Pipeline for __service-legal__  to deploy the Legal Service.

    _Repo:_ `legal`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/legal/v1/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
909
910
911
912
913
914
915
916
917
```bash
az pipelines create \
  --name 'service-legal'  \
  --repository legal  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
918
919
920
921
922
923
924

4. Add a Pipeline for __service-storage__  to deploy the Storage Service.

    _Repo:_ `storage`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/storage/v2/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
925
926
927
928
929
930
931
932
933
934
```bash
az pipelines create \
  --name 'service-storage'  \
  --repository storage  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

935
936
937
938
939
940
941

5. Add a Pipeline for __service-indexer-queue__  to deploy the Indexer Queue Function.

    _Repo:_ `indexer-queue`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ ScaledObject exist in osdu namespace.

Daniel Scholl's avatar
Daniel Scholl committed
942
943
944
945
946
947
948
949
950
```bash
az pipelines create \
  --name 'service-indexer-queue'  \
  --repository indexer-queue  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
951
952
953
954
955
956
957

6. Add a Pipeline for __service-indexer__  to deploy the Indexer Service.

    _Repo:_ `indexer-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/indexer/v2/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
958
959
960
961
962
963
964
965
966
```bash
az pipelines create \
  --name 'service-indexer'  \
  --repository indexer-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
967
968
969
970
971
972
973

7. Add a Pipeline for __service-search__  to deploy the Search Service.

    _Repo:_ `search-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/search/v2/swagger-ui.html is alive.

Daniel Scholl's avatar
Daniel Scholl committed
974
975
976
977
978
979
980
981
982
```bash
az pipelines create \
  --name 'service-search'  \
  --repository search-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
983

984
8. Add a Pipeline for __file__  to deploy the File Service.
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999

    _Repo:_ `file`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/file/v2/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-file'  \
  --repository file  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

1000
9. Add a Pipeline for __delivery__  to deploy the Delivery Service.
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013

    _Repo:_ `delivery`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/delivery/v2/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-delivery'  \
  --repository delivery  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
1014
1015
```

Sumra Zafar's avatar
Sumra Zafar committed
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
10. Add a Pipeline for __schema__  to deploy the Schema Service.

    _Repo:_ `schema-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/schema-service/v1/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-schema'  \
  --repository schema-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

1032
11. Add a Pipeline for __unit__  to deploy the Unit Service.
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046

    _Repo:_ `unit-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/unit/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-unit'  \
  --repository unit-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
Komal Makkar's avatar
Komal Makkar committed
1047

1048
12. Add a Pipeline for __crs-catalog-service__  to deploy the Crs Catalog Service.
Komal Makkar's avatar
Komal Makkar committed
1049

Sumra Zafar's avatar
Sumra Zafar committed
1050
1051
1052
    _Repo:_ `crs-catalog-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/crs/catalog/swagger-ui.html
harshit aggarwal's avatar
harshit aggarwal committed
1053

Sumra Zafar's avatar
Sumra Zafar committed
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
```bash
az pipelines create \
  --name 'service-crs-catalog'  \
  --repository crs-catalog-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```

1064
13. Add a Pipeline for __crs-conversion-service__  to deploy the Crs Conversion Service.
Sumra Zafar's avatar
Sumra Zafar committed
1065
1066

    _Repo:_ `crs-conversion-service`
harshit aggarwal's avatar
harshit aggarwal committed
1067
    _Path:_ `/devops/azure/pipeline.yml`
Sumra Zafar's avatar
Sumra Zafar committed
1068
    _Validate:_ https://<your_dns_name>/api/crs/converter/swagger-ui.html
harshit aggarwal's avatar
harshit aggarwal committed
1069
1070
1071

```bash
az pipelines create \
Sumra Zafar's avatar
Sumra Zafar committed
1072
1073
  --name 'service-crs-conversion'  \
  --repository crs-conversion-service  \
harshit aggarwal's avatar
harshit aggarwal committed
1074
1075
1076
1077
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
harshit aggarwal's avatar
harshit aggarwal committed
1078
1079
1080
```


1081
14. Add a Pipeline for __register__  to deploy the Register Service.
Sumra Zafar's avatar
Sumra Zafar committed
1082
1083

    _Repo:_ `register`
harshit aggarwal's avatar
harshit aggarwal committed
1084
    _Path:_ `/devops/azure/pipeline.yml`
Sumra Zafar's avatar
Sumra Zafar committed
1085
    _Validate:_ https://<your_dns_name>/api/register/v1/swagger-ui.html is alive.
harshit aggarwal's avatar
harshit aggarwal committed
1086
1087
1088

```bash
az pipelines create \
Sumra Zafar's avatar
Sumra Zafar committed
1089
1090
  --name 'service-register'  \
  --repository register  \
harshit aggarwal's avatar
harshit aggarwal committed
1091
1092
1093
1094
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
Komal Makkar's avatar
Komal Makkar committed
1095
1096
```

1097
15. Add a Pipeline for __notification__  to deploy the Notification Service.
Komal Makkar's avatar
Komal Makkar committed
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111

    _Repo:_ `notification-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/notification/v1/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-notification'  \
  --repository notification  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
harshit aggarwal's avatar
harshit aggarwal committed
1112

1113
16. Add a Pipeline for __wks__  to deploy the Wks Service.
harshit aggarwal's avatar
harshit aggarwal committed
1114

Sumra Zafar's avatar
Sumra Zafar committed
1115
    _Repo:_ `wks`
harshit aggarwal's avatar
harshit aggarwal committed
1116
    _Path:_ `/devops/azure/pipeline.yml`
Sumra Zafar's avatar
Sumra Zafar committed
1117
    _Validate:_ ScaledObject exist in osdu namespace.
harshit aggarwal's avatar
harshit aggarwal committed
1118
1119
1120

```bash
az pipelines create \
Sumra Zafar's avatar
Sumra Zafar committed
1121
1122
  --name 'service-wks'  \
  --repository wks  \
harshit aggarwal's avatar
harshit aggarwal committed
1123
1124
1125
1126
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
Nicholas Karsky's avatar
Nicholas Karsky committed
1127
```
Sumra Zafar's avatar
Sumra Zafar committed
1128

1129
17. Add a Pipeline for __ingestion-workflow__  to deploy the Schema Service.
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143

    _Repo:_ `ingestion-workflow`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/workflow/v1/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'service-ingestion-workflow'  \
  --repository ingestion-workflow  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
1144
18. Add a Pipeline for __seismic-store-service__  to deploy the Seismic Store Service.
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158

    _Repo:_ `seismic-store-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>//seistore-svc/api/v3/svcstatus is alive.

```bash
az pipelines create \
  --name 'service-seismic-store'  \
  --repository seismic-store-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
1159

1160
19. Add a Pipeline for __service-entitlements__  to deploy the Entitlements Service.
1161
1162
1163
1164
    > This pipeline may have to be run twice for integration tests to pass due to a preload data issue.

    _Repo:_ `entitlements`
    _Path:_ `/devops/azure/pipeline.yml`
1165
    _Validate:_ https://<your_dns_name>/api/entitlements/v2/swagger-ui.html is alive.
1166
1167
1168

```bash
az pipelines create \
1169
  --name 'service-entitlements'  \
1170
1171
1172
1173
1174
1175
  --repository entitlements  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
20. Add a Pipeline for __wellbore-domain-services__  to deploy the Wellbore Domain Services.

    _Repo:_ `wellbore-domain-services`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/os-wellbore-ddms/docs is alive.

```bash
az pipelines create \
  --name 'wellbore-domain-services'  \
  --repository wellbore-domain-services  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
Sumra Zafar's avatar
Sumra Zafar committed
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
```

21. Add a Pipeline for __ingestion-service__  to deploy the Ingestion Service.

    _Repo:_ `ingestion-service`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/ingestion/docs is alive.

```bash
az pipelines create \
  --name 'ingestion-service'  \
  --repository ingestion-service  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
1206
```
Vivek Ojha's avatar
Vivek Ojha committed
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
22. Add a Pipeline for __dataset__  to deploy the Dataset Service.

   _Repo:_ `dataset`
   _Path:_ `/devops/azure/pipeline.yml`
   _Validate:_ https://<your_dns_name>/api/dataset/v1/swagger-ui.html is alive.

```bash
az pipelines create \
  --name 'dataset'  \
  --repository dataset  \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
```
1222

Vivek Ojha's avatar
Vivek Ojha committed
1223
23. Add a Pipeline for __policy-service__  to deploy the Policy Service.
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236

    _Repo:_ `policy`
    _Path:_ `/devops/azure/pipeline.yml`
    _Validate:_ https://<your_dns_name>/api/ingestion/docs is alive.

```bash
az pipelines create \
  --name 'policy-service'  \
  --repository policy \
  --branch master  \
  --repository-type tfsgit  \
  --yaml-path /devops/azure/pipeline.yml  \
  -ojson
1237
```