README.md 18.6 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# os-entitlements-azure

os-entitlements-azure is a [Spring Boot](https://spring.io/projects/spring-boot) service that hosts CRUD APIs that enable management of user entitlements.

## Running Locally

### Requirements

In order to run this service locally, you will need the following:

- [Maven 3.6.0+](https://maven.apache.org/download.cgi)
- [AdoptOpenJDK8](https://adoptopenjdk.net/)
- Infrastructure dependencies, deployable through the relevant [infrastructure template](https://dev.azure.com/slb-des-ext-collaboration/open-data-ecosystem/_git/infrastructure-templates?path=%2Finfra&version=GBmaster&_a=contents)
- While not a strict dependency, example commands in this document use [bash](https://www.gnu.org/software/bash/)

### General Tips

**Environment Variable Management**
The following tools make environment variable configuration simpler
 - [direnv](https://direnv.net/) - for a shell/terminal environment
 - [EnvFile](https://plugins.jetbrains.com/plugin/7861-envfile) - for [Intellij IDEA](https://www.jetbrains.com/idea/)

**Lombok**
This project uses [Lombok](https://projectlombok.org/) for code generation. You may need to configure your IDE to take advantage of this tool.
 - [Intellij configuration](https://projectlombok.org/setup/intellij)
 - [VSCode configuration](https://projectlombok.org/setup/vscode)


Daniel Scholl's avatar
Daniel Scholl committed
29
### Understanding Environment Variables
30
31
32
33
34
35
36
37
38
39
40
41

In order to run the service locally, you will need to have the following environment variables defined.

**Note** The following command can be useful to pull secrets from keyvault:
```bash
az keyvault secret show --vault-name $KEY_VAULT_NAME --name $KEY_VAULT_SECRET_NAME --query value -otsv
```

**Required to run service**

| name | value | description | sensitive? | source |
| ---  | ---   | ---         | ---        | ---    |
42
| `server.servlet.contextPath` | `/entitlements/v1/` | Servlet context path | no | - |
Mingyang Zhu's avatar
Mingyang Zhu committed
43
| `service_domain_name` | ex `contoso.com` | The name of the domain for which the service will run | no | output of infrastructure deployment |
44
45
| `partition_service_endpoint` |  ex `https://foo-partition.azurewebsites.net` | Partition Service API endpoint | no | output of infrastructure deployment |
| `azure.activedirectory.app-resource-id` | `********` | AAD client application ID | yes | output of infrastructure deployment |
46
47
48
49
50
51
52
53
54
| `aad_client_id` | `********` | AAD client application ID | yes | output of infrastructure deployment |
| `azure.activedirectory.AppIdUri` | `api://${azure.activedirectory.client-id}` | URI for AAD Application | no | -- |
| `cosmosdb_database` | ex `foo-db` | The name of the CosmosDB database | no | output of infrastructure deployment |
| `KEYVAULT_URI` | ex `https://foo-keyvault.vault.azure.net/` | URI of KeyVault that holds application secrets | no | output of infrastructure deployment |
| `appinsights_key` | `********` | API Key for App Insights | yes | output of infrastructure deployment |
| `spring.application.name` | `entitlements-azure` | Name of application. Needed by App Insights | no | -- |
| `AZURE_CLIENT_ID` | `********` | Identity to run the service locally. This enables access to Azure resources. You only need this if running locally | yes | keyvault secret: `$KEYVAULT_URI/secrets/app-dev-sp-username` |
| `AZURE_TENANT_ID` | `********` | AD tenant to authenticate users from | yes | -- |
| `AZURE_CLIENT_SECRET` | `********` | Secret for `$AZURE_CLIENT_ID` | yes | keyvault secret: `$KEYVAULT_URI/secrets/app-dev-sp-password` |
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

In Order to run service with AAD authentication add below environment variables.This is the recommended approach to run entitlement service in local.
As AAD will authenticate the token and extract the payload to fetch user information.

| name | value | description | sensitive? | source |
| ---  | ---   | ---         | ---        | ---    |
| `azure_istioauth_enabled` | `false` | Flag to Disable AAD auth | no | -- |
| `azure_activedirectory_session_stateless` | `true` | Flag run in stateless mode (needed by AAD dependency) | no | -- |
| `azure_activedirectory_client_id` | `********` | AAD client application ID | yes | output of infrastructure deployment |

In Order to run service with Istio authentication add below environment variables.This is needed only to test istio filter scenarios,
with these settings service expects "x-payload" header which contains Base64 encoded format of Payload.In this approach service will not do Authentication.

 name | value | description | sensitive? | source |
| ---  | ---   | ---         | ---        | ---    |
70
| `azure_istioauth_enabled` | `true` | Flag to Disable AAD auth | no | -- |
71

72

73
74
75
76
77
78
**Required to run integration tests**

| name | value | description | sensitive? | source |
| ---  | ---   | ---         | ---        | ---    |
| `ENTITLEMENT_URL` | ex `http://localhost:8080/` | The host where the service is running | no | -- |
| `MY_TENANT` | ex `opendes` | OSDU tenant used for testing | no | -- |
79
| `AZURE_AD_TENANT_ID` | `********` | AD tenant to authenticate users from | yes | -- |
80
| `INTEGRATION_TESTER` | `********` | System identity to assume for API calls. Note: this user must have entitlements configured already | no | -- |
81
| `ENTITLEMENT_MEMBER_NAME_VALID` | `********` | Secret from `$INTEGRATION_TESTER` for userInfo cosmoscollection partitionId | yes | Create in userInfo cosmosCollection |
82
83
| `AZURE_TESTER_SERVICEPRINCIPAL_SECRET` | `********` | Secret for `$INTEGRATION_TESTER` | yes | -- |
| `AZURE_AD_APP_RESOURCE_ID` | `********` | AAD client application ID | yes | output of infrastructure deployment |
84
85
86
| `AZURE_AD_OTHER_APP_RESOURCE_ID` | ex. `********` |  Valid secondary application for Testing | yes | -- |
| `AZURE_AD_OTHER_APP_RESOURCE_OID` | ex. `********` |  Valid secondary application for Testing | yes | -- |
| `DOMAIN` | ex `contoso.com` | Must match the value of `service_domain_name` above | no | -- |
87
88
| `EXPIRED_TOKEN` | `********` | An expired JWT token | yes | Create one, then wait until it expires |
| `ENTITLEMENT_GROUP_NAME_VALID` | ex. `data.test1` | A group name generated by running integration tests. | no | -- |
89
| `ENTITLEMENT_MEMBER_NAME_INVALID` | ex. `InvalidTestAdmin` | Used for negative testing | no | -- |
Erik Leckner's avatar
Erik Leckner committed
90
| `AZURE_AD_USER_EMAIL` | ex. `********` | Valid member user email in the AD Tenant for Testing | yes | -- |
91
92
93
94
| `AZURE_AD_USER_OID` | ex. `********` | Valid member user objectIdin the AD Tenant for Testing | yes | -- |
| `AZURE_AD_GUEST_EMAIL` | ex. `********` | Valid guest user email in the AD Tenant for Testing | yes | -- |
| `AZURE_AD_GUEST_OID` | ex. `********` | Valid guest user objectId in the AD Tenant for Testing | yes | -- |

Erik Leckner's avatar
Erik Leckner committed
95
96
97
98
| `AZURE_INVALID_EMAIL` | ex. `invalid.test@email.com` |  Invalid Email  for Testing | no | -- |
| `AZURE_INVALID_APP_ID` | ex. `03015fad-093c-424a-a7c4-42ed9993f9e3` |  Invalid Appilication Identity for Testing | no | -- |
| `AZURE_INVALID_ID` | ex. `03012fadBADX424a-a7c4-42ed9993f9e3` |  Invalid Identity for Testing | no | -- |

99
100
101
102
103
104
105
106
107
108
109
110
111

### Configure Maven

Check that maven is installed:
```bash
$ mvn --version
Apache Maven 3.6.0
Maven home: /usr/share/maven
Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /usr/lib/jvm/jdk8u212-b04/jre
...
```


Daniel Scholl's avatar
Daniel Scholl committed
112
113

### Build, Run and Test the application Locally
114
115
116
117
118
119
120
121
122

After configuring your environment as specified above, you can follow these steps to build and run the application

```bash
# execute build + unit tests
$ mvn clean package
...
[INFO] BUILD SUCCESS

Daniel Scholl's avatar
Daniel Scholl committed
123
# run service locally **REQUIRES SPECIFIC ENVIRONMENT VARIABLES SET**
124
$ java -jar $(find ./target/ -name '*.jar')
Daniel Scholl's avatar
Daniel Scholl committed
125
126
127

# Test the application  **REQUIRES SPECIFIC ENVIRONMENT VARIABLES SET**
$ mvn clean test -f integration-tests/pom.xml
128
129
```

Daniel Scholl's avatar
Daniel Scholl committed
130
_After the service has started it should be accessible via a web browser by visiting [http://localhost:8080/swagger-ui.html](http://localhost:8080/swagger-ui.html). If the request does not fail, you can then run the integration tests._
131
132
133
134
135
136
137
138
139



## Debugging

Jet Brains - the authors of Intellij IDEA, have written an [excellent guide](https://www.jetbrains.com/help/idea/debugging-your-first-java-application.html) on how to debug java programs.

## Configuring User Entitlements

Erik Leckner's avatar
Erik Leckner committed
140
141
Here is how you can configure user entitlements via the Azure specific API.

142
###Create a new user or service principal.
Erik Leckner's avatar
Erik Leckner committed
143

144
The request body contains the user or service principal to create in JSON format. At a minimum, you must specify the required properties for the user or service principal.
Erik Leckner's avatar
Erik Leckner committed
145
146
147
148
149
The required  properties for a user or service principal is the uid and one tenant with one group. The uid is either a user email or a service principal UUID.
You can optionally specify any additional tenants and groups.

####Permissions

150
The following permission is required to call this API.
Erik Leckner's avatar
Erik Leckner committed
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171

service.entitlements.admin

##### POST /profile
| header | value | required |
| ---  | ---  | ---  |
| Authorization | Bearer {token} | Yes |
| Content-Type | application/json | Yes |
| Request body  |  In the request body, supply a JSON representation of user object. | Yes |

The following table lists the properties that are required when you create a user. .

| Property	| Type	| Description | Required |
| ---  | ---  | ---  | ---  |
| uid | user email or service principal UUID. | The user email or service principal UUID. | Yes |
| id | OID | The OID for the user or service principal. | No value required. |
| tenants| list of TenantInfo | The tenants for the user or service principal. | Yes. |

##### Response
If successful, this method returns 201 response code and user object in the response body.

Johnny Guo's avatar
Johnny Guo committed
172
173
If the user profile already exists, this method returns a 409 response code and an error message in the response body.

Erik Leckner's avatar
Erik Leckner committed
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
##### Example: Create a user

##### Request
Here is an example of the request.

###### POST /profile
###### Content-type: application/json

```json
{
    "id": "",
    "uid": "erik.leckner@wipro.com",
    "tenants": [
        {
            "name": "$SOME_OSDU_TENANT",
            "groups": [
                "service.storage.admin",
                "service.legal.admin",
                "data.datalake.admin",
                "data.datalake.viewer",
                "data.default.viewer"
            ]
        },
        {
            "name": "$ANOTHER_OSDU_TENANT",
            "groups": [
                 "service.storage.admin"
             ]
        }
    ]
}
```

In the request body, supply a JSON representation of user object.

209
###Update a user or service principal.
Erik Leckner's avatar
Erik Leckner committed
210

211
The request body contains the user or service principal to update in JSON format. At a minimum, you must specify the required properties for the user or service principal.
Erik Leckner's avatar
Erik Leckner committed
212
213
214
215
216
The required  properties for a user or service principal is the uid and one tenant with one group. The uid is either a user email or a service principal UUID.
You can optionally specify any additional tenants and groups.

####Permissions

217
The following permission is required to call this API.
Erik Leckner's avatar
Erik Leckner committed
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275

service.entitlements.admin

##### PUT /profile
| header | value | required |
| ---  | ---  | ---  |
| Authorization | Bearer {token} | Yes |
| Content-Type | application/json | Yes |
| Request body  |  In the request body, supply a JSON representation of user object. | Yes |

The following table lists the properties that are required when you create a user. .

| Property	| Type	| Description | Required |
| ---  | ---  | ---  | ---  |
| uid | user email or service principal UUID. | The user email or service principal UUID. | Yes |
| id | OID | The OID for the user or service principal. | No value required. |
| tenants| list of TenantInfo | The tenants for the user or service principal. | Yes. |

##### Response
If successful, this method returns 200 response code and user object in the response body.

##### Example: Update a user

##### Request
Here is an example of the request.

###### PUT /profile
###### Content-type: application/json

```json
{
    "id": "",
    "uid": "erik.leckner@wipro.com",
    "tenants": [
        {
            "name": "$SOME_OSDU_TENANT",
            "groups": [
                "service.storage.admin",
                "service.legal.admin",
                "data.datalake.admin",
                "data.datalake.viewer"
            ]
        },
        {
            "name": "$ANOTHER_OSDU_TENANT",
            "groups": [
                 "service.storage.admin"
             ]
        }
    ]
}
```

In the request body, supply a JSON representation of user object.

## Configuring User Entitlements (Deprecated)

Here is how you can configure user entitlements manually.
276
277
278
279
280
281
282
283

 - Identify the correct CosmosDB account. This can be found in the output of the infrastructure template. Alternatively, you should be able to identify it as the singular CosmosDB account that is provisioned in the resource group that hosts the this service
 - Use the `Data Explorer` tool in the CosmosDB UI and navigate to the `UserInfo` container
 - Identify if there already exists a `User` record in the table by applying a filter equal to `WHERE c.id = '$IDENTITY_ID'`. `$IDENTITY_ID` will be one of: (1) Object ID of a user (i.e., `974cb327-1406-76b9-91de-cbd6eb7ec949`) or (2) Application ID of a system assigned identity (i.e., `930b212b-e21f-478f-b993-af9b41abe836`)
 - If the user exists, verify that the permissions are correct. If the user needs to be added to a group, you can edit the document directly and click `Save`
 - If the user does not exist, you can add a document that has the following schema. The exact groups you wish to provision to the user will most likely be different, so be sure to add/remove the appropriate roles. The below listing represents a user with full access to all services.
```json
{
Erik Leckner's avatar
Erik Leckner committed
284
285
    "id": "$OBJECT_ID",
    "uid": "$IDENTITY_ID",
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
    "tenants": [
        {
            "name": "$SOME_OSDU_TENANT",
            "groups": [
                "service.storage.admin",
                "service.legal.admin",
                "data.datalake.admin",
                "data.datalake.viewer",
                "data.default.viewer",
                "data.default.owner",
                "service.search.admin",
                "service.search.user",
                "data.default.viewers",
                "data.default.owners",
                "service.entitlements.admin"
            ]
        },
        {
            "name": "$ANOTHER_OSDU_TENANT",
            "groups": [
              ...
            ]
        }
    ]
}
```

Daniel Scholl's avatar
Daniel Scholl committed
313
## Deploying the Service
314

Daniel Scholl's avatar
Azure  
Daniel Scholl committed
315
316
317
Service deployments into Azure are standardized to make the process the same for all services if using ADO and are closely related to the infrastructure deployed. The steps to deploy into Azure can be [found here](https://github.com/azure/osdu-infrastructure)

The default ADO pipeline is /devops/azure-pipelines.yml
318

Daniel Scholl's avatar
Daniel Scholl committed
319
### Manual Deployment Steps
320

Daniel Scholl's avatar
Daniel Scholl committed
321
__Environment Settings__
322
323
324
325

The following environment variables are necessary to properly deploy a service to an Azure OSDU Environment.

```bash
Daniel Scholl's avatar
Daniel Scholl committed
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
# Group Level Variables
export AZURE_TENANT_ID=""
export AZURE_SUBSCRIPTION_ID=""
export AZURE_SUBSCRIPTION_NAME=""
export AZURE_PRINCIPAL_ID=""
export AZURE_PRINCIPAL_SECRET=""
export AZURE_APP_ID=""
export AZURE_NO_ACCESS_ID=""
export AZURE_NO_ACCESS_SECRET=""
export AZURE_OTHER_APP_ID=""
export AZURE_BASENAME_21=""
export AZURE_BASENAME=""
export AZURE_BASE=""
export AZURE_INVALID_JWT=""

# Pipeline Level Variable
export AZURE_SERVICE="entitlements"
Daniel Scholl's avatar
Azure  
Daniel Scholl committed
343
344
export AZURE_BUILD_SUBDIR="."
export AZURE_TEST_SUBDIR="integration-tests"
Daniel Scholl's avatar
Daniel Scholl committed
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
export AZURE_OSDU_TENANT="opendes"
export AZURE_COMPANY_DOMAIN="contoso.com"
export AZURE_VALID_GROUPNAME="integ.test.data.creator"
export AZURE_INVALID_GROUPNAME="InvalidTestAdmin"

# Required for Azure Deployment
export AZURE_CLIENT_ID="${AZURE_PRINCIPAL_ID}"
export AZURE_CLIENT_SECRET="${AZURE_PRINCIPAL_SECRET}"
export AZURE_RESOURCE_GROUP="${AZURE_BASENAME}-osdu-r2-app-rg"
export AZURE_APPSERVICE_PLAN="${AZURE_BASENAME}-osdu-r2-sp"
export AZURE_APPSERVICE_NAME="${AZURE_BASENAME_21}-au-${AZURE_SERVICE}"

# Required for Testing
export ENTITLEMENT_URL="https://${AZURE_BASENAME_21}-au-entitlements.azurewebsites.net/"
export AZURE_AD_TENANT_ID="${AZURE_TENANT_ID}"
export INTEGRATION_TESTER="${AZURE_PRINCIPAL_ID}"
export ENTITLEMENT_MEMBER_NAME_VALID="${AZURE_PRINCIPAL_ID}"
export AZURE_TESTER_SERVICEPRINCIPAL_SECRET="${AZURE_PRINCIPAL_SECRET}"
export AZURE_AD_APP_RESOURCE_ID="${AZURE_APP_ID}"
export AZURE_AD_OTHER_APP_RESOURCE_ID="${AZURE_OTHER_APP_ID}"
export EXPIRED_TOKEN="${AZURE_INVALID_JWT}"
export DOMAIN="${AZURE_COMPANY_DOMAIN}"
export MY_TENANT="${AZURE_OSDU_TENANT}"
export ENTITLEMENT_GROUP_NAME_VALID="${AZURE_VALID_GROUPNAME}"
export ENTITLEMENT_MEMBER_NAME_INVALID="${AZURE_INVALID_GROUPNAME}"
Erik Leckner's avatar
Erik Leckner committed
370
371
372
373
374
375
376
377
378
export AZURE_AD_USER_EMAIL="${AZURE_AD_USER_EMAIL}"
export AZURE_AD_USER_OID="${AZURE_AD_USER_OID}"
export AZURE_AD_GUEST_EMAIL="${AZURE_AD_GUEST_EMAIL}"
export AZURE_AD_GUEST_OID="${AZURE_AD_GUEST_OID}"
export AZURE_AD_OTHER_APP_RESOURCE_ID="${AZURE_AD_OTHER_APP_RESOURCE_ID}"
export AZURE_AD_OTHER_APP_RESOURCE_OID="${AZURE_AD_OTHER_APP_RESOURCE_OID}"
export AZURE_INVALID_EMAIL="invalid.test@email.com"
export AZURE_INVALID_APP_ID="03015fad-093c-424a-a7c4-42ed9993f9e3"
export AZURE_INVALID_ID="03012fadBADX424a-a7c4-42ed9993f9e3"
379
380
```

Daniel Scholl's avatar
Daniel Scholl committed
381
__Azure Service Deployment__
382
383


Daniel Scholl's avatar
Daniel Scholl committed
384
1. Deploy the service using the Maven Plugin  _(azure_deploy)_
385
386

```bash
Daniel Scholl's avatar
Azure  
Daniel Scholl committed
387
388
389
cd $AZURE_BUILD_SUBDIR
mvn azure-webapp:deploy \
  -DAZURE_DEPLOY_TENANT=$AZURE_TENANT_ID \
390
  -Dazure.appservice.subscription=$AZURE_SUBSCRIPTION_ID \
Daniel Scholl's avatar
Azure  
Daniel Scholl committed
391
392
  -DAZURE_DEPLOY_CLIENT_ID=$AZURE_CLIENT_ID \
  -DAZURE_DEPLOY_CLIENT_SECRET=$AZURE_CLIENT_SECRET \
393
394
395
396
397
  -Dazure.appservice.resourcegroup=$AZURE_RESOURCE_GROUP \
  -Dazure.appservice.plan=$AZURE_APPSERVICE_PLAN \
  -Dazure.appservice.appname=$AZURE_APPSERVICE_NAME
```

Daniel Scholl's avatar
Daniel Scholl committed
398
399
2. Configure the Web App to start the SpringBoot Application _(azure_config)_

400
401
402
403
404

```bash
az login --service-principal -u $AZURE_CLIENT_ID -p $AZURE_CLIENT_SECRET --tenant $AZURE_TENANT_ID

# Set the JAR FILE as required
Daniel Scholl's avatar
Daniel Scholl committed
405
406
TARGET=$(find ./target/ -name '*.jar')
JAR_FILE=${TARGET##*/}
407
408
409
410
411
412
413
414
415

JAVA_COMMAND="java -jar /home/site/wwwroot/${JAR_FILE}"
JSON_TEMPLATE='{"appCommandLine":"%s"}'
JSON_FILE="config.json"
echo $(printf "$JSON_TEMPLATE" "$JAVA_COMMAND") > $JSON_FILE

az webapp config set --resource-group $AZURE_RESOURCE_GROUP --name $AZURE_APPSERVICE_NAME --generic-configurations @$JSON_FILE
```

Daniel Scholl's avatar
Daniel Scholl committed
416
417
418
419
3. Execute the Integration Tests against the Service Deployment _(azure_test)_


```bash
Daniel Scholl's avatar
Azure  
Daniel Scholl committed
420
mvn clean test -f $AZURE_TEST_SUBDIR/pom.xml
Daniel Scholl's avatar
Daniel Scholl committed
421
422
```

423
424
425
426
427
428
429
430
431
432
433
434
435
436
437

## License
Copyright © Microsoft Corporation

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.