Skip to content
Snippets Groups Projects
Commit 287c19f9 authored by Rustam Lotsmanenko (EPAM)'s avatar Rustam Lotsmanenko (EPAM)
Browse files

Stop using base64 encoded service accounts for tests(GONRG-7865)

parent e2ff80bf
No related branches found
No related tags found
1 merge request!936Stop using base64 encoded service accounts for tests(GONRG-7865)
Hide whitespace changes
Inline Side-by-side
NOTICE
+ 2
− 2
View file @ 287c19f9
......@@ -68,8 +68,8 @@ The following software have components provided under the terms of this license:
- Byte Buddy (without dependencies) (from https://repo1.maven.org/maven2/net/bytebuddy/byte-buddy)
- Byte Buddy Java agent (from https://repo1.maven.org/maven2/net/bytebuddy/byte-buddy-agent)
- ClassMate (from http://github.com/cowtowncoder/java-classmate)
- Cloud Key Management Service (KMS) API (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-cloudkms)
- Cloud Storage JSON API (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-storage)
- Cloud Key Management Service (KMS) API v1-rev20240903-2.0.0 (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-cloudkms)
- Cloud Storage JSON API v1-rev20240916-2.0.0 (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-storage)
- Collections (from https://repo1.maven.org/maven2/commons-collections/commons-collections)
- Commons Digester (from http://commons.apache.org/digester/)
- Converter: Jackson (from https://github.com/square/retrofit, https://repo1.maven.org/maven2/com/squareup/retrofit2/converter-jackson)
......
provider/storage-gc/docs/gc/README.md
+ 13
− 13
View file @ 287c19f9
......@@ -154,19 +154,19 @@ TBD
You will need to have the following environment variables defined.
| name | value | description | sensitive? | source |
|---------------------------|------------------------------------------------|----------------------------------------------------------------------------------------------------------------|------------|--------------------------------------------------------------|
| `DEPLOY_ENV` | `empty` | Required but not used, should be set up with string "empty" | no | - |
| `GROUP_ID` | ex`opendes-gc.projects.com` | OSDU R2 to run tests under | no | - |
| `INTEGRATION_TESTER` | `********` | Service account base64 encoded string for API calls. Note: this user must have entitlements configured already | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| `LEGAL_URL` | ex`http://localhsot:8080/api/legal/v1/` | Legal API endpoint | no | - |
| `NO_DATA_ACCESS_TESTER` | `********` | Service account base64 encoded string without data access | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| `PUBSUB_TOKEN` | `****` | ? | no | - |
| `STORAGE_URL` | ex`http://localhost:8080/api/storage/v2/` | Endpoint of storage service | no | - |
| `TENANT_NAME` | ex `opendes` | OSDU tenant used for testing | no | -- |
| `OPA_INTEGRATION_ENABLED` | `true` OR `false` | Should be update if integration with OPA\Policy enabled\disabled | no | -- |
| `ENTITLEMENTS_URL` | ex`http://localhost:8080/api/entitlements/v2/` | Endpoint of entitlements service | no | - |
| `DATA_ROOT_TESTER` | `********` | Service account base64 encoded string with data root access | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| name | value | description | sensitive? | source |
|---------------------------|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|------------|--------------------------------------------------------------|
| `DEPLOY_ENV` | `empty` | Required but not used, should be set up with string "empty" | no | - |
| `GROUP_ID` | ex`opendes-gc.projects.com` | OSDU R2 to run tests under | no | - |
| `INTEGRATION_TESTER` | `ewogICJ0....` or `tmp/service-acc.json` | Service account base64 encoded string or path to a file for API calls. Note: this user must have entitlements configured already | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| `LEGAL_URL` | ex`http://localhsot:8080/api/legal/v1/` | Legal API endpoint | no | - |
| `NO_DATA_ACCESS_TESTER` | `ewogICJ0....` or `tmp/service-acc.json` | Service account base64 encoded string or path to a file without data access | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| `PUBSUB_TOKEN` | `****` | ? | no | - |
| `STORAGE_URL` | ex`http://localhost:8080/api/storage/v2/` | Endpoint of storage service | no | - |
| `TENANT_NAME` | ex `opendes` | OSDU tenant used for testing | no | -- |
| `OPA_INTEGRATION_ENABLED` | `true` OR `false` | Should be update if integration with OPA\Policy enabled\disabled | no | -- |
| `ENTITLEMENTS_URL` | ex`http://localhost:8080/api/entitlements/v2/` | Endpoint of entitlements service | no | - |
| `DATA_ROOT_TESTER` | `********` | Service account base64 encoded string with data root access | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
**Entitlements configuration for integration accounts**
......
provider/storage-gc/pom.xml
+ 1
− 1
View file @ 287c19f9
......@@ -59,7 +59,7 @@
<dependency>
<groupId>org.opengroup.osdu</groupId>
<artifactId>core-lib-gc</artifactId>
<version>0.27.0-rc7</version>
<version>0.27.0-rc8</version>
</dependency>
<!-- Mappers -->
<dependency>
......
testing/storage-test-gc/src/test/java/org/opengroup/osdu/storage/util/DecodedContentExtractor.java 0 → 100644
+ 127
− 0
View file @ 287c19f9
/*
* Copyright 2020-2024 Google LLC
* Copyright 2020-2024 EPAM Systems, Inc
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.opengroup.osdu.storage.util;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Base64;
import java.util.function.Predicate;
import lombok.RequiredArgsConstructor;
import lombok.extern.java.Log;
@Log
@RequiredArgsConstructor
public class DecodedContentExtractor {
private final String inputFilenameOrContent;
private final Predicate<String> contentAcceptanceTester;
private boolean validOutputContentFound;
private String outputContent;
public String getContent() {
validOutputContentFound = false;
outputContent = null;
log.info("Treat value as a content");
if (inputFilenameOrContent.trim().isEmpty()) {
log.info("provided value is empty. Output as is.");
return setValidOutputContent(inputFilenameOrContent);
}
if (!treatValueAsAContent(inputFilenameOrContent)) {
log.info("Value is not a valid content. Treat value as a filename");
if (!treatValueAsAFileName(inputFilenameOrContent)){
log.info("Value is not a filename with a valid content");
}
}
return getValidOutputContentIfFound();
}
private boolean treatValueAsAContent(String input) {
if (contentAcceptanceTester.test(input)) {
log.info("the value is a valid content. Output as is.");
setValidOutputContent(input);
return true;
}
String output;
try {
output = new String(Base64.getDecoder().decode(input));
log.info("the value is probably Base64 encoded. Just decoded");
if (contentAcceptanceTester.test(output)) {
log.info("the decoded value is a valid content. Output decoded value.");
setValidOutputContent(output);
} else {
log.info("the decoded value is not a valid content.");
}
} catch (IllegalArgumentException e) {
log.info("the value is not Base64 encoded. ");
}
return validOutputContentFound;
}
private boolean treatValueAsAFileName(String filename) {
if (treatFileContent(filename)) return true;
try {
filename = new String(Base64.getDecoder().decode(filename));
log.info("the filename is probably Base64 encoded. Just decoded");
if (treatFileContent(filename)) return true;
} catch (IllegalArgumentException e) {
log.info("the filename is not Base64 encoded. ");
}
return validOutputContentFound;
}
private boolean treatFileContent(String filename) {
try {
Path path = Paths.get(filename);
if (Files.exists(path)) {
log.info("the filename is of existing file. Read file.");
try {
String fileContent = new String(Files.readAllBytes(path));
if (treatValueAsAContent(fileContent)) {
return true;
}
} catch (IOException | SecurityException | OutOfMemoryError ex) {
log.info(() -> ("unable to read the file: " + ex.getClass().getSimpleName()));
}
}
} catch (InvalidPathException ex) {
log.info("the filename is not valid or the file doesn't exist.");
}
return false;
}
private String setValidOutputContent(String outputContent) {
this.outputContent = outputContent;
this.validOutputContentFound = true;
return getValidOutputContentIfFound();
}
public String getValidOutputContentIfFound() {
return validOutputContentFound ? outputContent : null;
}
}
testing/storage-test-gc/src/test/java/org/opengroup/osdu/storage/util/GoogleServiceAccount.java
+ 11
− 17
View file @ 287c19f9
......@@ -14,14 +14,19 @@
package org.opengroup.osdu.storage.util;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import io.jsonwebtoken.JwtBuilder;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;
import java.util.function.Predicate;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
......@@ -31,25 +36,14 @@ import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import io.jsonwebtoken.JwtBuilder;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;
class GoogleServiceAccount {
private final static Predicate<String> contentAcceptanceTester = s -> s.trim().startsWith("{");
private static final String DEFAULT_TARGET_AUDIENCE = "osdu";
public GoogleServiceAccount(String serviceAccountEncoded) throws IOException {
this(Base64.getDecoder().decode(serviceAccountEncoded));
}
public GoogleServiceAccount(byte[] serviceAccountJson) throws IOException {
try (InputStream inputStream = new ByteArrayInputStream(serviceAccountJson)) {
public GoogleServiceAccount(String serviceAccountValue) throws IOException {
serviceAccountValue = new DecodedContentExtractor(serviceAccountValue, contentAcceptanceTester).getContent();
try (InputStream inputStream = new ByteArrayInputStream(serviceAccountValue.getBytes())) {
this.serviceAccount = ServiceAccountCredentials.fromStream(inputStream);
}
}
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment