Merge branch 'aws-m10-release' into 'master'

Added AWS README and HTTP Security Fix

See merge request !100

provider/crs-catalog-aws/README.md

+# CRS Catalog Service
+The main goals of the Spatial Reference Catalog (aka crs-catalog) service are to offer:
+1. Coordinate Reference Systems (CRSs) to enable end-users to make a CRS selection.
+2. Search for CRSs given a number of constraints.
+3. Download of the entire catalog for local caching and when the cache has to be refreshed.
+4. Once a CRS is found, produce a persistable reference to be stored with the data, which fully
+describes the CRS; this persistable reference string becomes catalog independent. This means
+any consumer will be able to understand the CRS definition even if a different catalog is used in the
+future context.
+
+## Running Locally
+
+These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
+
+### Prerequisites
+Pre-requisites
+
+* JDK 8 (https://docs.aws.amazon.com/corretto/latest/corretto-8-ug/downloads-list.html)
+* Maven 3.8.3 or later
+* Lombok 1.16 or later
+* OSDU Instance deployed on AWS
+
+### Service Configuration
+In order to run the service locally or remotely, you will need to have the following environment variables defined.
+
+| name | example value | required | description | sensitive? |
+| ---  | ---   | ---         | ---        | ---    |
+| `APPLICATION_PORT` | `8080` | yes | The port the service will be hosted on. | no |
+| `AWS_REGION` | `us-east-1` | yes | The region where resources needed by the service are deployed | no |
+| `AWS_ACCESS_KEY_ID` | `ASIAXXXXXXXXXXXXXX` | yes | The AWS Access Key for a user with access to Backend Resources required by the service | yes |
+| `AWS_SECRET_ACCESS_KEY` | `super-secret-key==` | yes | The AWS Secret Key for a user with access to Backend Resources required by the service | yes |
+| `AWS_SESSION_TOKEN` | `session-token-xxxxxxxxxx` | no | AWS Session token needed if using an SSO user session to authenticate | yes |
+| `ENVIRONMENT` | `osdu-prefix` | yes | The Resource Prefix defined during deployment | no |
+| `LOG_LEVEL` | `DEBUG` | yes | The Log Level severity to use (https://www.tutorialspoint.com/log4j/log4j_logging_levels.htm) | no |
+| `SSM_ENABLED` | `true` | yes | Set to 'true' to use SSM to resolve config properties, otherwise use env vars | no |
+| `SSL_ENABLED` | `false` | no | Set to 'false' to disable SSL for local development | no |
+| `ENTITLEMENTS_BASE_URL` | `http://localhost:8081` or `https://some-hosted-url` | yes | Specify the base url for an entitlements service instance. Can be run locally or remote | no |
+| `CRS_CATALOG_FILENAME` | `data/crs_catalog_v2.json` | no | Provides a reference to the CRS catalog | no |
+
+### Run Locally
+Check that maven is installed:
+
+example:
+```bash
+$ mvn --version
+Apache Maven 3.8.3 (ff8e977a158738155dc465c6a97ffaf31982d739)
+Maven home: /usr/local/Cellar/maven/3.8.3/libexec
+Java version: 1.8.0_312, vendor: Amazon.com Inc., runtime: /Library/Java/JavaVirtualMachines/amazon-corretto-8.jdk/Contents/Home/jre
+...
+```
+
+You may need to configure access to the remote maven repository that holds the OSDU dependencies. Copy one of the below files' content to your .m2 folder
+* For development against the OSDU GitLab environment, leverage: `<REPO_ROOT>~/.mvn/community-maven.settings.xml`
+* For development in an AWS Environment, leverage: `<REPO_ROOT>/provider/crs-catalog-aws/maven/settings.xml`
+
+* Navigate to the service's root folder and run:
+
+```bash
+mvn clean package -pl crs-catalog-core,provider/crs-catalog-aws
+```
+
+* If you wish to build the project without running tests
+
+```bash
+mvn clean package -pl crs-catalog-core,provider/crs-catalog-aws -DskipTests
+```
+
+After configuring your environment as specified above, you can follow these steps to run the application. These steps should be invoked from the *repository root.*
+<br/>
+<br/>
+NOTE: If not on osx/linux: Replace `*` with version numbers as defined in the provider/crs-catalog-aws/pom.xml file
+
+```bash
+java -jar provider/crs-catalog-aws/target/crs-catalog-aws-*.*.*-SNAPSHOT-spring-boot.jar
+```
+
+## Testing
+
+### Running Integration Tests
+This section describes how to run OSDU Integration tests (testing/crs-catalog-test-aws).
+
+You will need to have the following environment variables defined.
+
+| name | example value | description | sensitive?
+ | ---  | ---   | ---         | ---        |
+| `AWS_ACCESS_KEY_ID` | `ASIAXXXXXXXXXXXXXX` | The AWS Access Key for a user with access to Backend Resources required by the service | yes |
+| `AWS_SECRET_ACCESS_KEY` | `super-secret-key==` | The AWS Secret Key for a user with access to Backend Resources required by the service | yes |
+| `AWS_SESSION_TOKEN` | `session-token-xxxxxxxxx` | AWS Session token needed if using an SSO user session to authenticate | yes |
+| `AWS_COGNITO_USER_POOL_ID` | `us-east-1_xxxxxxxx` | User Pool Id for the reference cognito | no |
+| `AWS_COGNITO_CLIENT_ID` | `xxxxxxxxxxxx` | Client ID for the Auth Flow integrated with the Cognito User Pool | no |
+| `ADMIN_USER` | `int-test-user@testing.com` | Int Test Username | no |
+| `USER_NO_ACCESS` | `no-access-user@testing.com` | Int Test No Access Username | no |
+| `ADMIN_PASSWORD` | `some-secure-password` | Int Test User/NoAccessUser Password | yes |
+| `CRS_CATALOG_HOST` | `localhost:8080` | The url where the CRS Catalog API is hosted | no |
+
+
+**Creating a new user to use for integration tests**
+ ```
+ aws cognito-idp admin-create-user --user-pool-id ${AWS_COGNITO_USER_POOL_ID} --username ${AWS_COGNITO_AUTH_PARAMS_USER} --user-attributes Name=email,Value=${AWS_COGNITO_AUTH_PARAMS_USER} Name=email_verified,Value=True --message-action SUPPRESS
+
+ aws cognito-idp initiate-auth --auth-flow ${AWS_COGNITO_AUTH_FLOW} --client-id ${AWS_COGNITO_CLIENT_ID} --auth-parameters USERNAME=${AWS_COGNITO_AUTH_PARAMS_USER},PASSWORD=${AWS_COGNITO_AUTH_PARAMS_PASSWORD}
+ ```
+
+**Entitlements group configuration for integration accounts**
+<br/>
+In order to add user entitlements, run entitlements bootstrap scripts in the entitlements project
+
+Execute following command to build code and run all the integration tests:
+
+### Run Tests simulating Pipeline
+
+* Prior to running tests, scripts must be executed locally to generate pipeline env vars
+
+```bash
+testing/catalog_test_aws/build-aws/prepare-dist.sh
+
+#Set Neccessary ENV Vars here as defined in run-tests.sh
+
+dist/testing/integration/bin/catalog_test_aws/build-aws/run-tests.sh 
+```
+
+## License
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

provider/crs-catalog-aws/src/main/java/org/opengroup/osdu/crs/security/AuthSecurityConfig.java

@@ -79,7 +79,6 @@ public class AuthSecurityConfig extends WebSecurityConfigurerAdapter implements
                 .and()
                 .authorizeRequests()
                 .antMatchers(AUTH_WHITELIST).permitAll()
-                .anyRequest().authenticated()
                 .and()
                 .addFilterBefore(authFilter, UsernamePasswordAuthenticationFilter.class);
     }

testing/catalog_test_aws/run-integration-tests.sh

@@ -14,26 +14,45 @@
 
 #!/usr/bin/env bash
 
-# Install venv for python3
-which apt-get && sudo apt-get install -y python3 python3-pip python3-venv || echo "Not Ubuntu, skipping"
-which yum && sudo yum install -y python3 python3-pip python3-venv || echo "Not RHEL, skipping"
-
-python3 -m venv env
-# sed -i 's/$1/${1:-}/' env/bin/activate # Fix deactivation bug '$1 unbound variable'
-source env/bin/activate
-python3 -m pip install --upgrade pip
-python3 -m pip install -r requirements.txt
-
-# Run tests
-echo ***RUNNING CRS Catalog API V2 TESTS***
-python3 run_test_api_v2.py
-V2_TEST_STATUS=$?
-echo ***FINISHED CRS Catalog API V2 TESTS***
-
+if [[ "$OSTYPE" == "msys" ]]; then
+  python -m pip install --upgrade pip
+  python -m pip install --user virtualenv
+  python -m venv env
+  source env/Scripts/activate
+  python -m pip install --upgrade pip
+  python -m pip install -r requirements.txt
+
+  # Run tests
+  echo ***RUNNING CRS Catalog API V2 TESTS***
+  python run_test_api_v2.py
+  V2_TEST_STATUS=$?
+  echo ***FINISHED CRS Catalog API V2 TESTS***
+
+  # python -m pip freeze > requirements.txt
+  python -m pip uninstall -r requirements.txt -y
+
+else
+  # Install venv for python3
+  which apt-get && sudo apt-get install -y python3 python3-pip python3-venv || echo "Not Ubuntu, skipping"
+  which yum && sudo yum install -y python3 python3-pip python3-venv || echo "Not RHEL, skipping"
+
+  python3 -m venv env
+  # sed -i 's/$1/${1:-}/' env/bin/activate # Fix deactivation bug '$1 unbound variable'
+  source env/bin/activate
+  python3 -m pip install --upgrade pip
+  python3 -m pip install -r requirements.txt
+
+  # Run tests
+  echo ***RUNNING CRS Catalog API V2 TESTS***
+  python3 run_test_api_v2.py
+  V2_TEST_STATUS=$?
+  echo ***FINISHED CRS Catalog API V2 TESTS***
+
+  # python3 -m pip freeze > requirements.txt
+  python3 -m pip uninstall -r requirements.txt -y
 
+fi
 
-# python3 -m pip freeze > requirements.txt
-python3 -m pip uninstall -r requirements.txt -y
 deactivate
 rm -rf env/