Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • osdu/platform/system/indexer-service
  • schundu/indexer-service
2 results
Show changes
Commits on Source (8)
......@@ -369,7 +369,7 @@ The following software have components provided under the terms of this license:
- Byte Buddy (without dependencies) (from https://repo1.maven.org/maven2/net/bytebuddy/byte-buddy)
- Byte Buddy Java agent (from https://repo1.maven.org/maven2/net/bytebuddy/byte-buddy-agent)
- ClassMate (from http://github.com/cowtowncoder/java-classmate)
- Cloud Key Management Service (KMS) API v1-rev20230407-2.0.0 (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-cloudkms)
- Cloud Key Management Service (KMS) API (from https://repo1.maven.org/maven2/com/google/apis/google-api-services-cloudkms)
- CloudWatch Metrics for AWS Java SDK (from https://aws.amazon.com/sdkforjava)
- Cobertura (from http://cobertura.sourceforge.net)
- Cobertura Limited Runtime (from http://cobertura.sourceforge.net)
......@@ -418,10 +418,10 @@ The following software have components provided under the terms of this license:
- JBoss Marshalling River (from https://repo1.maven.org/maven2/org/jboss/marshalling/jboss-marshalling-river)
- JCIP Annotations under Apache License (from http://stephenc.github.com/jcip-annotations)
- JDOM
- JJWT :: Legacy Transitive Dependency Jar (from https://repo1.maven.org/maven2/io/jsonwebtoken/jjwt)
- JMES Path Query library (from https://aws.amazon.com/sdkforjava)
- JOpt Simple (from http://jopt-simple.github.io/jopt-simple, http://pholser.github.io/jopt-simple)
- JSON Small and Fast Parser (from https://repo1.maven.org/maven2/net/minidev/json-smart, https://urielch.github.io/)
- JSON Web Token support for the JVM (from https://github.com/jwtk/jjwt)
- JSON library from Android SDK (from http://developer.android.com/sdk)
- JSON.simple (from http://code.google.com/p/json-simple/)
- JSONassert (from http://github.com/skyscreamer/yoga, https://github.com/skyscreamer/JSONassert)
......@@ -464,7 +464,6 @@ The following software have components provided under the terms of this license:
- Kotlin Stdlib Common (from https://kotlinlang.org/)
- Kotlin Stdlib Jdk7 (from <https://kotlinlang.org/>, https://kotlinlang.org/)
- Kotlin Stdlib Jdk8 (from <https://kotlinlang.org/>, https://kotlinlang.org/)
- LatencyUtils (from http://latencyutils.github.io/LatencyUtils/)
- Lucene Common Analyzers (from <https://repo1.maven.org/maven2/org/apache/lucene/lucene-analyzers-common>, https://repo1.maven.org/maven2/org/apache/lucene/lucene-analyzers-common)
- Lucene Core (from https://repo1.maven.org/maven2/org/apache/lucene/lucene-core)
- Lucene Grouping (from https://lucene.apache.org/, https://repo1.maven.org/maven2/org/apache/lucene/lucene-grouping)
......@@ -548,7 +547,6 @@ The following software have components provided under the terms of this license:
- Redisson (from http://redisson.org)
- Retrofit (from https://github.com/square/retrofit, https://repo1.maven.org/maven2/com/squareup/retrofit2/retrofit)
- RxJava (from https://github.com/ReactiveX/RxJava)
- Servlet API (from https://repo1.maven.org/maven2/org/mortbay/jetty/servlet-api)
- Servlet Specification 2.5 API (from http://jetty.mortbay.org, https://repo1.maven.org/maven2/org/mortbay/jetty/servlet-api-2.5)
- SnakeYAML (from http://code.google.com/p/snakeyaml/, http://www.snakeyaml.org, https://bitbucket.org/snakeyaml/snakeyaml)
- Spatial4J (from https://projects.eclipse.org/projects/locationtech.spatial4j)
......@@ -604,12 +602,12 @@ The following software have components provided under the terms of this license:
- aggs-matrix-stats (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- compiler (from http://github.com/spullara/mustache.java)
- documentdb-bulkexecutor (from http://azure.microsoft.com/en-us/services/documentdb/)
- elasticsearch (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git, https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch)
- elasticsearch-cli (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch-core (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch (from https://github.com/elastic/elasticsearch, https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch)
- elasticsearch-cli (from https://github.com/elastic/elasticsearch)
- elasticsearch-core (from https://github.com/elastic/elasticsearch)
- elasticsearch-geo (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch-secure-sm (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch-x-content (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch-secure-sm (from https://github.com/elastic/elasticsearch)
- elasticsearch-x-content (from https://github.com/elastic/elasticsearch)
- error-prone annotations (from https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations)
- io.grpc:grpc-alts (from https://github.com/grpc/grpc-java)
- io.grpc:grpc-api (from https://github.com/grpc/grpc-java)
......@@ -629,7 +627,7 @@ The following software have components provided under the terms of this license:
- jakarta.inject (from https://repo1.maven.org/maven2/org/glassfish/hk2/external/jakarta.inject)
- java-cloudant (from https://cloudant.com)
- javatuples (from http://www.javatuples.org)
- javax.annotation-api (from http://jcp.org/en/jsr/detail?id=250)
- javax.annotation API (from http://jcp.org/en/jsr/detail?id=250)
- javax.inject (from http://code.google.com/p/atinject/, https://repo1.maven.org/maven2/org/glassfish/hk2/external/javax.inject)
- jersey-core-common (from https://repo1.maven.org/maven2/org/glassfish/jersey/core/jersey-common)
- jersey-core-server (from https://repo1.maven.org/maven2/org/glassfish/jersey/core/jersey-server)
......@@ -690,6 +688,7 @@ The following software have components provided under the terms of this license:
- JSch (from http://www.jcraft.com/jsch/)
- Jodd BeanUtil (from http://jodd.org)
- Jodd Core (from http://jodd.org)
- LatencyUtils (from http://latencyutils.github.io/LatencyUtils/)
- Lucene Common Analyzers (from <https://repo1.maven.org/maven2/org/apache/lucene/lucene-analyzers-common>, https://repo1.maven.org/maven2/org/apache/lucene/lucene-analyzers-common)
- Lucene Grouping (from https://lucene.apache.org/, https://repo1.maven.org/maven2/org/apache/lucene/lucene-grouping)
- Lucene Highlighter (from https://lucene.apache.org/, https://repo1.maven.org/maven2/org/apache/lucene/lucene-highlighter)
......@@ -721,6 +720,7 @@ The following software have components provided under the terms of this license:
- Apache Lucene (module: suggest) (from https://lucene.apache.org/, https://repo1.maven.org/maven2/org/apache/lucene/lucene-suggest)
- AspectJ Weaver (from http://www.aspectj.org, https://www.eclipse.org/aspectj/)
- Class Model for Hk2 (from https://repo1.maven.org/maven2/org/glassfish/hk2/class-model)
- FindBugs-jsr305 (from http://findbugs.sourceforge.net/)
- GAX (Google Api eXtensions) for Java (Core) (from https://github.com/googleapis, https://github.com/googleapis/gax-java, https://repo1.maven.org/maven2/com/google/api/gax)
- GAX (Google Api eXtensions) for Java (gRPC) (from <https://repo1.maven.org/maven2/com/google/api/gax-grpc>, https://repo1.maven.org/maven2/com/google/api/gax-grpc)
- Google APIs Client Library for Java (from https://repo1.maven.org/maven2/com/google/api-client/google-api-client)
......@@ -732,6 +732,7 @@ The following software have components provided under the terms of this license:
- Hamcrest (from http://hamcrest.org/JavaHamcrest/)
- Hamcrest Core (from http://hamcrest.org/, http://hamcrest.org/JavaHamcrest/, https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core)
- JSch (from http://www.jcraft.com/jsch/)
- Jackson module: Afterburner (from http://wiki.fasterxml.com/JacksonHome, https://github.com/FasterXML/jackson-modules-base)
- Jakarta Activation API (from https://github.com/eclipse-ee4j/jaf, https://github.com/jakartaee/jaf-api, https://repo1.maven.org/maven2/jakarta/activation/jakarta.activation-api)
- Jakarta Annotations API (from https://projects.eclipse.org/projects/ee4j.ca)
- Jakarta RESTful WS API (from https://github.com/eclipse-ee4j/jaxrs-api)
......@@ -850,7 +851,7 @@ The following software have components provided under the terms of this license:
- Java Architecture for XML Binding (from http://jaxb.java.net/, https://repo1.maven.org/maven2/javax/xml/bind/jaxb-api)
- JavaBeans Activation Framework (from <http://java.sun.com/javase/technologies/desktop/javabeans/jaf/index.jsp>, http://java.sun.com/javase/technologies/desktop/javabeans/jaf/index.jsp, https://repo1.maven.org/maven2/com/sun/activation/javax.activation)
- Servlet Specification 2.5 API (from http://jetty.mortbay.org, https://repo1.maven.org/maven2/org/mortbay/jetty/servlet-api-2.5)
- javax.annotation-api (from http://jcp.org/en/jsr/detail?id=250)
- javax.annotation API (from http://jcp.org/en/jsr/detail?id=250)
========================================================================
CDDL-1.1
......@@ -864,7 +865,7 @@ The following software have components provided under the terms of this license:
- JavaBeans Activation Framework (from <http://java.sun.com/javase/technologies/desktop/javabeans/jaf/index.jsp>, http://java.sun.com/javase/technologies/desktop/javabeans/jaf/index.jsp, https://repo1.maven.org/maven2/com/sun/activation/javax.activation)
- JavaMail API (from https://repo1.maven.org/maven2/com/sun/mail/javax.mail)
- Run Level Service (from https://repo1.maven.org/maven2/org/glassfish/hk2/hk2-runlevel)
- javax.annotation-api (from http://jcp.org/en/jsr/detail?id=250)
- javax.annotation API (from http://jcp.org/en/jsr/detail?id=250)
- tomcat-embed-core (from http://tomcat.apache.org/)
========================================================================
......@@ -875,6 +876,13 @@ The following software have components provided under the terms of this license:
- JUnit (from http://junit.org)
- System Rules (from http://stefanbirkner.github.com/system-rules/, http://stefanbirkner.github.io/system-rules/)
========================================================================
EDL-1.0
========================================================================
The following software have components provided under the terms of this license:
- Jakarta Activation API (from https://github.com/eclipse-ee4j/jaf, https://github.com/jakartaee/jaf-api, https://repo1.maven.org/maven2/jakarta/activation/jakarta.activation-api)
========================================================================
EPL-1.0
========================================================================
......@@ -887,11 +895,9 @@ The following software have components provided under the terms of this license:
- JUnit Jupiter Params (from http://junit.org/junit5/, https://junit.org/junit5/)
- JUnit Platform Commons (from http://junit.org/junit5/, https://junit.org/junit5/)
- JUnit Platform Engine API (from http://junit.org/junit5/, https://junit.org/junit5/)
- Jakarta Activation API (from https://github.com/eclipse-ee4j/jaf, https://github.com/jakartaee/jaf-api, https://repo1.maven.org/maven2/jakarta/activation/jakarta.activation-api)
- Jakarta Annotations API (from https://projects.eclipse.org/projects/ee4j.ca)
- Jakarta Bean Validation API (from https://beanvalidation.org)
- Jakarta XML Binding API (from https://repo1.maven.org/maven2/jakarta/xml/bind/jakarta.xml.bind-api, https://repo1.maven.org/maven2/org/jboss/spec/javax/xml/bind/jboss-jaxb-api_2.3_spec)
- Jetty :: Utilities (from http://jetty.mortbay.org, http://www.eclipse.org/jetty, https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-util, https://repo1.maven.org/maven2/org/mortbay/jetty/jetty-util)
- Logback Contrib :: JSON :: Classic (from https://repo1.maven.org/maven2/ch/qos/logback/contrib/logback-json-classic)
- Logback Contrib :: JSON :: Core (from https://repo1.maven.org/maven2/ch/qos/logback/contrib/logback-json-core)
- Logback Contrib :: Jackson (from https://repo1.maven.org/maven2/ch/qos/logback/contrib/logback-jackson)
......@@ -1003,7 +1009,7 @@ The following software have components provided under the terms of this license:
- ServiceLocator Default Implementation (from https://repo1.maven.org/maven2/org/glassfish/hk2/hk2-locator)
- aopalliance-repackaged (from https://repo1.maven.org/maven2/org/glassfish/hk2/external/aopalliance-repackaged)
- jakarta.inject (from https://repo1.maven.org/maven2/org/glassfish/hk2/external/jakarta.inject)
- javax.annotation-api (from http://jcp.org/en/jsr/detail?id=250)
- javax.annotation API (from http://jcp.org/en/jsr/detail?id=250)
- jersey-container-servlet (from https://repo1.maven.org/maven2/org/glassfish/jersey/containers/jersey-container-servlet)
- jersey-container-servlet-core (from https://repo1.maven.org/maven2/org/glassfish/jersey/containers/jersey-container-servlet-core)
- jersey-core-client (from https://repo1.maven.org/maven2/org/glassfish/jersey/core/jersey-client)
......@@ -1230,7 +1236,7 @@ SunPro
The following software have components provided under the terms of this license:
- Lucene Core (from https://repo1.maven.org/maven2/org/apache/lucene/lucene-core)
- elasticsearch-core (from https://github.com/elastic/elasticsearch, https://github.com/elastic/elasticsearch.git)
- elasticsearch-core (from https://github.com/elastic/elasticsearch)
========================================================================
W3C
......@@ -1280,16 +1286,8 @@ cc-pd
========================================================================
The following software have components provided under the terms of this license:
- Guava: Google Core Libraries for Java (from http://code.google.com/p/guava-libraries, https://github.com/google/guava, https://repo1.maven.org/maven2/com/google/guava/guava)
- Netty/Codec (from https://repo1.maven.org/maven2/io/netty/netty-codec)
========================================================================
mit-old-style-no-advert
========================================================================
The following software have components provided under the terms of this license:
- Brave (from https://repo1.maven.org/maven2/io/zipkin/brave/brave)
========================================================================
proprietary-license
========================================================================
......@@ -1306,7 +1304,6 @@ The following software have components provided under the terms of this license:
- HdrHistogram (from http://hdrhistogram.github.io/HdrHistogram/)
- JBoss Logging 3 (from http://www.jboss.org)
- JSON in Java (from https://github.com/douglascrockford/JSON-java)
- LatencyUtils (from http://latencyutils.github.io/LatencyUtils/)
- Microsoft Azure client library for Blob Storage (from https://github.com/Azure/azure-sdk-for-java)
- jersey-core-common (from https://repo1.maven.org/maven2/org/glassfish/jersey/core/jersey-common)
......@@ -1315,7 +1312,6 @@ unknown
========================================================================
The following software have components provided under the terms of this license:
- Jakarta Activation API (from https://github.com/eclipse-ee4j/jaf, https://github.com/jakartaee/jaf-api, https://repo1.maven.org/maven2/jakarta/activation/jakarta.activation-api)
- System Rules (from http://stefanbirkner.github.com/system-rules/, http://stefanbirkner.github.io/system-rules/)
- xml-apis (from https://repo1.maven.org/maven2/xml-apis/xml-apis)
......
......@@ -242,6 +242,12 @@
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.7</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
<dependency>
......
......@@ -411,6 +411,12 @@
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>${cobertura-maven-plugin.version}</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
<dependency>
......
......@@ -14,16 +14,19 @@
package org.opengroup.osdu.util;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import lombok.ToString;
import lombok.extern.java.Log;
import lombok.extern.slf4j.Slf4j;
import org.opengroup.osdu.common.TestsBase;
import javax.ws.rs.core.Response;
import java.net.SocketTimeoutException;
import java.util.Map;
@Log
import java.util.UUID;
@Slf4j
@ToString
public class AzureHTTPClient extends HTTPClient {
......@@ -41,42 +44,42 @@ public class AzureHTTPClient extends HTTPClient {
return token;
}
public ClientResponse send(String httpMethod, String url, String payLoad, Map<String, String> headers, String token) {
ClientResponse response;
ClientResponse response = null;
System.out.println("in Azure send method");
String correlationId = java.util.UUID.randomUUID().toString();
log.info(String.format("Request correlation id: %s", correlationId));
headers.put(HEADER_CORRELATION_ID, correlationId);
Client client = getClient();
client.setReadTimeout(300000);
client.setConnectTimeout(300000);
log.info(String.format("httpMethod: %s", httpMethod));
log.info(String.format("payLoad: %s", payLoad));
log.info(String.format("headers: %s", headers));
log.info(String.format("URL: %s", url));
WebResource webResource = client.resource(url);
log.info("waiting on response in azure send");
int retryCount = 2;
try{
response = this.getClientResponse(httpMethod, payLoad, webResource, headers, token);
while (retryCount > 0) {
int count = 1;
int MaxRetry = 3;
while (count < MaxRetry) {
try {
headers.put("correlation-id", headers.getOrDefault("correlation-id", UUID.randomUUID().toString()));
//removing Auth header before logging
headers.remove("Authorization");
log.info(String.format("Request method: %s\nHeaders: %s\nRequest Body: %s", httpMethod, headers, TestsBase.indentatedResponseBody(payLoad)));
log.info(String.format("Attempt: #%s/%s, CorrelationId: %s", count, MaxRetry, headers.get("correlation-id")));
response = this.getClientResponse(httpMethod, payLoad, webResource, headers, token);
if (response.getStatusInfo().getFamily().equals(Response.Status.Family.valueOf("SERVER_ERROR"))) {
log.info(String.format("got resoponse : %s", response.getStatusInfo()));
count++;
Thread.sleep(5000);
log.info(String.format("Retrying --- "));
response = this.getClientResponse(httpMethod, payLoad, webResource, headers, token);
} else
continue;
} else {
break;
retryCount--;
}
System.out.println("sending response from azure send method");
return response;
} catch (Exception e) {
if (e.getCause() instanceof SocketTimeoutException) {
System.out.println("Retrying in case of socket timeout exception");
return this.getClientResponse(httpMethod, payLoad, webResource, headers, token);
}
} catch (Exception ex) {
log.error("Exception While Making Request: ", ex);
count++;
if (count == MaxRetry) {
throw new AssertionError("Error: Send request error", ex);
}
} finally {
if (response != null) {
log.info(String.format("This is the response received : %s\nHeaders: %s\nStatus code: %s", response, response.getHeaders(), response.getStatus()));
}
}
e.printStackTrace();
throw new AssertionError("Error: Send request error", e);
}
return response;
}
}
# tag indexer-extended enabled by default
cucumber.options=--tags '~@* and @indexer-extended'
cucumber.options=--tags '~@indexer-extended'
......@@ -127,6 +127,7 @@ public class RecordSteps extends TestsBase {
String payLoad = new Gson().toJson(records);
log.log(Level.INFO, "Start ingesting records={0}", payLoad);
ClientResponse clientResponse = httpClient.send(HttpMethod.PUT, getStorageBaseURL() + "records", payLoad, headers, httpClient.getAccessToken());
log.info(String.format("Response body: %s\n Correlation id: %s\nResponse Status code: %s", indentatedResponseBody(clientResponse.getEntity(String.class)), clientResponse.getHeaders().get("correlation-id"), clientResponse.getStatus()));
assertEquals(201, clientResponse.getStatus());
} catch (Exception ex) {
throw new AssertionError(ex.getMessage());
......
......@@ -2,6 +2,9 @@ package org.opengroup.osdu.common;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import org.opengroup.osdu.core.common.model.legal.Legal;
import org.opengroup.osdu.models.Setup;
import org.opengroup.osdu.models.TestIndex;
......@@ -120,9 +123,16 @@ public abstract class TestsBase {
T response = new Gson().fromJson(responseEntity, typeParameterClass);
response.setHeaders(clientResponse.getHeaders());
response.setResponseCode(clientResponse.getStatus());
log.info(String.format("Response body: %s\nCorrelation id: %s\nResponse Status code: %s", indentatedResponseBody(responseEntity), response.getHeaders().get("correlation-id"), response.getResponseCode()));
return response;
}
public static String indentatedResponseBody(String responseBody) {
JsonParser jsonParser = new JsonParser();
JsonElement jsonElement = jsonParser.parse(responseBody);
String indentedResponseEntity =new GsonBuilder().setPrettyPrinting().create().toJson(jsonElement);
return indentedResponseEntity;
}
private void logCorrelationIdWithFunctionName(MultivaluedMap<String, String> headers) {
log.info(String.format("Scenario Name: %s, Correlation-Id: %s", scenario.getId(), headers.get("correlation-id")));
......
......@@ -8,6 +8,7 @@ import lombok.Data;
import javax.ws.rs.HttpMethod;
import org.opengroup.osdu.common.TestsBase;
import org.opengroup.osdu.core.common.model.entitlements.Acl;
import org.opengroup.osdu.core.common.model.legal.Legal;
import org.opengroup.osdu.util.ElasticUtils;
......@@ -60,8 +61,9 @@ public class TestIndex {
public void setupSchema() {
ClientResponse clientResponse = this.httpClient.send(HttpMethod.POST, getStorageBaseURL() + "schemas", this.getStorageSchemaFromJson(), headers, httpClient.getAccessToken());
if (clientResponse.getType() != null)
LOGGER.info(String.format("Response status: %s, type: %s", clientResponse.getStatus(), clientResponse.getType().toString()));
if (clientResponse.getType() != null) {
LOGGER.info(String.format("Response status: %s, type: %s\nResponse body: %s", clientResponse.getStatus(), clientResponse.getType().toString(), TestsBase.indentatedResponseBody(clientResponse.getEntity(String.class))));
}
}
public void deleteSchema(String kind) {
......@@ -142,4 +144,4 @@ public class TestIndex {
return legal;
}
}
\ No newline at end of file
}