diff --git a/provider/notification-gcp/README.md b/provider/notification-gcp/README.md new file mode 100644 index 0000000000000000000000000000000000000000..16b5cf2d1e10d78ae5e4b3b9bddbcd395f9985d5 --- /dev/null +++ b/provider/notification-gcp/README.md @@ -0,0 +1,177 @@ +#Notification Service +notification-gcp is a [Spring Boot](https://spring.io/projects/spring-boot) service that allow for interested consumers to subscribe to data and metadata changes using a publish/subscriber pattern. + +## Getting Started +These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system. + +### Requirements +* Java 8 +* [Maven 3.6.0+](https://maven.apache.org/download.cgi) +* GCloud command line tool +* GCloud access to opendes project + +### General Tips + +**Environment Variable Management** +The following tools make environment variable configuration simpler + - [direnv](https://direnv.net/) - for a shell/terminal environment + - [EnvFile](https://plugins.jetbrains.com/plugin/7861-envfile) - for [Intellij IDEA](https://www.jetbrains.com/idea/) + +**Lombok** +This project uses [Lombok](https://projectlombok.org/) for code generation. You may need to configure your IDE to take advantage of this tool. + - [Intellij configuration](https://projectlombok.org/setup/intellij) + - [VSCode configuration](https://projectlombok.org/setup/vscode) + +### Installation +In order to run the service locally or remotely, you will need to have the following environment variables defined. + +| name | value | description | sensitive? | source | +| --- | --- | --- | --- | --- | +| `APP_ENTITLEMENTS` | ex `https://entitlements.com/entitlements/v1` | Entitlements API endpoint | no | output of infrastructure deployment | +| `APP_REGISTER` | ex `https://register.com/api/register/v1` | Storage API endpoint | no | output of infrastructure deployment | +| `APP_PROJECT` | ex `opendes` | Google Cloud Project Id | no | output of infrastructure deployment | +| `APP_AUDIENCES` | ex `*****.apps.googleusercontent.com` | Client ID for getting access to cloud resources | yes | https://console.cloud.google.com/apis/credentials | + +**System Environment required to run service** + +| name | value | description | sensitive? | source | +| --- | --- | --- | --- | --- | +| `SPRING_PROFILES_ACTIVE` | `local` | spring active profile | no | + +### Run Locally +Check that maven is installed: +```bash +$ mvn --version +Apache Maven 3.6.0 +Maven home: /usr/share/maven +Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /usr/lib/jvm/jdk8u212-b04/jre +... +``` + +You will need to configure access to the remote maven repository that holds the OSDU dependencies. This file should live within `~/.m2/settings.xml`: + +```bash +$ cat ~/.m2/settings.xml +<?xml version="1.0" encoding="UTF-8"?> +<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> + <servers> + <server> + <id>os-core</id> + <username>slb-des-ext-collaboration</username> + <!-- Treat this auth token like a password. Do not share it with anyone, including Microsoft support. --> + <password>${VSTS_FEED_TOKEN}</password> + </server> + </servers> +</settings> +``` + +* Update the Google cloud SDK to the latest version: + +```bash +gcloud components update +``` +* Set Google Project Id: + +```bash +gcloud config set project <YOUR-PROJECT-ID> +``` + +* Perform a basic authentication in the selected project: + +```bash +gcloud auth application-default login +``` + +* Navigate to notification service's root folder and run: + +```bash +mvn jetty:run +## Testing +* Navigate to notification service's root folder and run: + +```bash +mvn clean install +``` + +* If you wish to see the coverage report then go to testing/target/site/jacoco-aggregate and open index.html + +* If you wish to build the project without running tests + +```bash +mvn clean install -DskipTests +``` + +After configuring your environment as specified above, you can follow these steps to build and run the application. These steps should be invoked from the *repository root.* + +```bash +cd provider/notification-gcp/ && mvn spring-boot:run -Dspring-boot.run.profiles=local +``` + +## Testing +Navigate to notification service's root folder and run all the tests: + +```bash +# build + test + install core service code +$ (cd notification-core/ && mvn clean install) +``` + +## Test the application + +After the service has started it should be accessible via a web browser by visiting [http://localhost:8080/api/notification/v1/swagger-ui.html](http://localhost:8080/swagger-ui.html). If the request does not fail, you can then run the integration tests. + +###Dependencies needed to run the integration tests +* Java 8 +* Maven +* Values for the following environment variables in Config.java + +| name | value | description | sensitive? | source | +| --- | --- | --- | --- | --- | +| `DE_OPS_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already, also **Private key id** of this account must be set in Register service variable SUBSCRIBER_PRIVATE_KEY_ID | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | +| `DE_ADMIN_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | +| `DE_EDITOR_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | +| `DE_NO_ACCESS_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | +| `ENVIRONMENT` | `dev` OR `local` OR `dev_gke`| Local for running locally with services url's predefined as http://localhost , Dev & Dev_Gke is configurable environment | no | - | +| `HMAC_SECRET` | ex`7a786376626e` | String in hex , must match pattern ^[a-zA-Z0-9]{8,30}+$ & be in register variable SUBSCRIBER_SECRET | yes | - | +| `REGISTER_BASE_URL` | `http://localhost:8081/api/register/v1` | Register service url | no | - | +| `NOTIFICATION_BASE_URL` | `http://localhost:8080/` | Notification service url | no | - | +| `INTEGRATION_TEST_AUDIENCE` | `********` | Client application ID | yes | https://console.cloud.google.com/apis/credentials | +| `CLIENT_TENANT` | ex `opendes` | Client tenant | no | - | +| `OSDU_TENANT` | ex `osdu` | Osdu tenant | no | - | +| `TOPIC_ID` | ex `records-changed` | PubSub topic id | no | https://console.cloud.google.com/cloudpubsub/topic | +| `REGISTER_CUSTOM_PUSH_URL_HMAC` | ex `http://localhost:8081/api/register/v1/test/challenge/hmac-integration-test` | Register testing push url | no | - | + + **Entitlements configuration for integration accounts** + + | DE_OPS_TESTER | DE_ADMIN_TESTER | DE_EDITOR_TESTER | DE_NO_ACCESS_TESTER | + | --- | --- | --- | --- | + |notification.pubsub<br/>service.entitlements.user<br/>users<br/>users.datalake.ops</br>| service.entitlements.user<br/>users<br/>users.datalake.admins</br> | service.entitlements.user<br/>users<br/>users.datalake.editors</br> | service.entitlements.user<br/>users<br/>| + +Above variables should be configured in the release pipeline to run integration tests. You should also replace them with proper values if you wish to run tests locally. + +###Commands to run tests +* Integration tests are refactored into two pieces: Core and Provider. Core contains business logic for tests and is a dependency for executing the tests from provider module. To build the core module, simply navigate to `notification-test-core` directory and run `mvn clean install`. This will build the core module +* Next, to execute the integration tests, navigate to the provider module and execute `mvn test` +```bash +# (cd testing/notification-test-core/ && mvn clean install) +# Note: this assumes that the environment variables for integration tests as outlined +# above are already exported in your environment. +$ (cd testing/notification-test-gcp/ && mvn clean test) +``` +## License +Copyright © Google LLC + +Copyright © EPAM Systems + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. \ No newline at end of file diff --git a/testing/README.md b/testing/README.md index f2f4870f44b49e3f79f29e5072d1f892cfda0e30..20c3a69c83cb6ad6f341768a4a9b27578255c9cf 100644 --- a/testing/README.md +++ b/testing/README.md @@ -42,4 +42,4 @@ Notification integration tests are refactored so that the business logic for int * Next, to execute the integration tests, navigate to the provider module and execute `mvn test` ### GCP -Instructions for running the GCP integration tests can be found [here](./notification-test-gcp/README.md). \ No newline at end of file +Instructions for running the GCP integration tests can be found [here](../provider/notification-gcp/README.md). \ No newline at end of file diff --git a/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointGSATests.java b/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointGSATests.java index a15f93348bf98faceeda1c3444e82dd04339b960..e4969dfd26b42d4eaabaead0c15eea4b492c8872 100644 --- a/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointGSATests.java +++ b/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointGSATests.java @@ -74,7 +74,12 @@ public abstract class PubsubEndpointGSATests extends BaseTestTemplate { gsaSecret.setSecretType("GSA"); ((GsaSecret) gsaSecret).setValue(gsaSecretValue); subscription.setSecret(gsaSecret); - subscription = this.subscriptionService.create(subscription); + try { + subscription = this.subscriptionService.create(subscription); + }catch (SubscriptionException e){ + System.out.println("Subscription exception inner response : " + e.getHttpResponse()); + throw e; + } Config.Instance().NotificationId = subscription.getNotificationId(); this.subscriptionId = subscription.getId(); diff --git a/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointHMACTests.java b/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointHMACTests.java index fcb164172b5b64d77d2b81e2e2e0c18e0f6d0b42..a0962f805211090dc90e4028a572030971d17d20 100644 --- a/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointHMACTests.java +++ b/testing/notification-test-core/src/main/java/org/opengroup/osdu/notification/api/PubsubEndpointHMACTests.java @@ -91,11 +91,15 @@ public abstract class PubsubEndpointHMACTests extends BaseTestTemplate { secret.setValue(Config.Instance().hmacSecretValue); subscription.setSecret(secret); - Subscription subscriptionCreated = subscriptionService.create(subscription); - notificationId = subscriptionCreated.getNotificationId(); - subscriptionId = subscriptionCreated.getId(); - - Config.Instance().NotificationId = notificationId; + try { + Subscription subscriptionCreated = subscriptionService.create(subscription); + notificationId = subscriptionCreated.getNotificationId(); + subscriptionId = subscriptionCreated.getId(); + Config.Instance().NotificationId = notificationId; + }catch (SubscriptionException e){ + System.out.println("Subscription exception inner response : " + e.getHttpResponse()); + throw e; + } } @Override diff --git a/testing/notification-test-gcp/README.md b/testing/notification-test-gcp/README.md deleted file mode 100644 index 90d4c022afcf3e377cd8ee91edfece97d9b3266e..0000000000000000000000000000000000000000 --- a/testing/notification-test-gcp/README.md +++ /dev/null @@ -1,49 +0,0 @@ -###Dependencies needed to run the integration tests -* JDK11 -* Maven -* Values for the following environment variables in Config.java - -| name | value | description | sensitive? | source | -| --- | --- | --- | --- | --- | -| `DE_OPS_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | -| `DE_ADMIN_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | -| `DE_EDITOR_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | -| `DE_NO_ACCESS_TESTER` | `*****` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | https://console.cloud.google.com/iam-admin/serviceaccounts | -| `ENVIRONMENT` | `dev` OR `local` | Local for running locally with services url's predefined as http://localhost , Dev is configurable environment | no | - | -| `HMAC_SECRET` | `*****` | sensitive secret to run HMAC tests | yes | - | -| `REGISTER_BASE_URL` | `http://localhost:8081/api/register/v1` | Register service url | no | - | -| `NOTIFICATION_BASE_URL` | `http://localhost:8080/` | Notification service url | no | - | -| `INTEGRATION_TEST_AUDIENCE` | `********` | Client application ID | yes | https://console.cloud.google.com/apis/credentials | -| `CLIENT_TENANT` | ex `opendes` | Client tenant | no | - | -| `OSDU_TENANT` | ex `osdu` | Osdu tenant | no | - | -| `TOPIC_ID` | ex `records-changed` | PubSub topic id | no | https://console.cloud.google.com/cloudpubsub/topic | - - **Entitlements configuration for integration accounts** - - | DE_OPS_TESTER | DE_ADMIN_TESTER | DE_EDITOR_TESTER | DE_NO_ACCESS_TESTER | - | --- | --- | --- | --- | - |notification.pubsub<br/>service.entitlements.user<br/>users<br/>| service.entitlements.user<br/>users<br/> | service.entitlements.user<br/>users<br/> | service.entitlements.user<br/>users<br/>| - -Above variables should be configured in the release pipeline to run integration tests. You should also replace them with proper values if you wish to run tests locally. - -###Commands to run tests -* Integration tests are refactored into two pieces: Core and Provider. Core contains business logic for tests and is a dependency for executing the tests from provider module. To build the core module, simply navigate to `notification-test-core` directory and run `mvn clean install`. This will build the core module -* Next, to execute the integration tests, navigate to the provider module and execute `mvn test` - - -## License -Copyright © Google LLC - -Copyright © EPAM Systems - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. \ No newline at end of file