Commit e2e14001 authored by Rustam Lotsmanenko (EPAM)'s avatar Rustam Lotsmanenko (EPAM) Committed by Rostislav Dublin (EPAM)
Browse files

Nested query search

parent 5544f214
......@@ -10,7 +10,11 @@
+ [Grouping](#grouping)
+ [Reserved characters](#reserved-characters)
+ [Wildcards](#wildcards)
* [Query by "nested" arrays objects](#nested-queries)
* [Sort](#sort-queries)
+ [Sort by "nested" arrays objects](#nested-sort)
* [Aggregation](#aggregation)
+ [Aggregation by "nested" arrays objects](#nested-aggregation)
* [Range Queries](#range-queries)
* [Geo-Spatial Queries](#geo-spatial-queries)
+ [Geo Distance](#geo-distance)
......@@ -70,7 +74,7 @@ Data Ecosystem search provides a JSON-style domain-specific language that you ca
POST /api/search/v2/query
{
"kind": "common:welldb:wellbore:1.0.0",
"query": "data.Status:Active",
"query": "data.Status:Active AND nested(data.VerticalMeasurements)",
"offset": 0,
"limit": 30,
"sort": {
......@@ -140,7 +144,7 @@ __Note:__ : It can take a delay of atleast 30 seconds once records are successfu
| Parameter | Description |
| :--- | :--- |
| kind | The kind of the record to query e.g. 'common:welldb:wellbore:1.0.0'. kind is a __required__ field and can be formatted as OSDU-Data-Partition-Id:data-source-id:entity-type:schema-version |
| query | The query string in Lucene query string syntax. |
| query | Query string based on Lucene query string syntax, supplemented with a specific format for describing queries to fields of object arrays indexed with the "nested" hint. |
| offset | The starting offset from which to return results. |
| limit | The maximum number of results to return from the given offset. If no limit is provided, then it will return __10__ items. Max number of items which can be fetched by the query is __100__. (If you wish to fetch large set of items, please use [query_with_cursor](#query-with-cursor) API). |
| sort | Allows you to add one or more sorts on specific fields. The length of fields and the length of order must match. Order value must be either ASC or DESC (case insensitive). For more details, ability and limitation about this feature, please refer to [Sort](#sort_querires)
......@@ -318,8 +322,102 @@ If you need to use date in your query, it has to be in one of the following form
For more info please refer [Date format](http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateOptionalTimeParser--)
## Query by "nested" arrays objects <a name="nested-queries"></a>
Starting from version 0.9.0 we can set "nested" hints in data schemes object array nodes.
It leads to accurate indexing of those arrays objects in the underlying Elasticsearch engine,
which then indexes such hinted arrays' objects as separate documents,
giving an option to query parent documents by children arrays objects fields data.
Those (say "nested arrays") queries require more complicated syntax in native ES Query DSL
which we wish to avoid in the OSDU Search query DSL, so we have developed our own simplified dialect
to describe such requests in the Search service in the form of the ```nested()``` function:
- for one level "nested array":
```json
nested(<path-to-root-nested-array-node>, <root-nested-array-object-fields-query>)
```
- for nested (multi-level) "nested array" queries
```json
nested(<path-to-root-nested-array-node>, nested(<path-to-subrootA-nested-array-node>, <subrootA-nested-array-object-fields-query>))
```
Multi-level nested queries are not limited in their depth. You nest them as required by the certain schema.
Several examples of the root and multi-level nested queries examples you can see in the below paragraphs.
The syntax of those queries is the same we learned from the above sections.
The only distinction is that their conditions are scoped by the own fields of objects of the array,
pointed in the first argument of the current nested(path,(conditions)) function.
### Single-level one condition nested query
It queries for wellboremarkerset WPCs having any Marker with MarkerMeasuredDepth field value greater than 10000
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"query":"nested(data.Markers, (MarkerMeasuredDepth:(>10000)))"
}
```
### Single-level several conditions nested query
It queries for wellboremarkerset WPCs having any Marker with both conditions match:
(MarkerMeasuredDepth > 10000 AND PositiveVerticalDelta < 13000)
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"query":"nested(data.Markers, (MarkerMeasuredDepth:(>10000) AND PositiveVerticalDelta:(<13000)))"
}
```
### Combination of single-level nested queries
It queries for wellboremarkerset WPCs having any Marker with any of conditions match:
(MarkerMeasuredDepth>10000 OR SurfaceDipAzimuth:<30000)
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"query":"nested(data.Markers, (MarkerMeasuredDepth:(>10000))) OR nested(data.Markers, (SurfaceDipAzimuth:(<30000)))"
}
```
### Multi-level nested queries
Assume we have a schema (not yet exists in OSDU schemas) describing document with "nested" array,
containing another "nested" array as a value of one of its fields.
For example, let's fantasize that the data.Markers Marker object has a nested 'Revisions' array of Revision objects
having two own fields: "RevisionDate" and "RevisionEngeneer". An indexed document might then look like this:
```json
"data": {
...
"Markers": [
{
...
"MarkerMeasuredDepth": 12345.6,
"PositiveVerticalDelta": 12345.6,
"Revisions": [
"RevisionDate": "2020-02-13T09:13:15.55+0000",
"RevisionEngineer": "John Smith"
]
}
]
}
```
We then might wish to search for wellboremarkerset WPCs having any Marker revised on a certain date by a certain engineer:
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"query":"nested(data.Markers, nested(data.Markers.Revisions, (RevisionDate:\"2020-02-13T09:13:15.55+0000\" AND RevisionEngineer:\"John Smith\")))"
}
```
### Nested and "non-nested" queries parts combinations
We can combine both types of queries in one request, eg:
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"query":"data.Name:\"Example Name\" AND nested(data.Markers, (MarkerMeasuredDepth:(>10000)))"
}
```
## Sort <a name="sort-queries"></a>
The sort feature supports int, float, double, long and datetime, but it does not support array object, nested object or string field as of now, and for the records contain such types won't return in the response.
The sort feature supports int, float, double, long and datetime.
It does not support array object or string field as of now, and for the records contain such types won't return to the response.
Starting from version 0.9.0 we can set "nested" hints in data schemes object array nodes.
And use such way indexed data for sorting. See in below "Sort by nested arrays objects".
The records either does not have the sorted fields or have empty value will be listed last in the result.
......@@ -344,7 +442,74 @@ The above request payload asks search service to sort on "data.Id" in an ascendi
**NOTE:** Search service does not validate the provided sort field, whether it exists or is of the supported data types. Different kinds may have attributes with the same names, but are different data types. Therefore, it is the user's responsibility to be aware and validate this in one's own workflow.
The sort query could be very expensive, especially if the given kind is too broad (e.g. "kind": "*:*:*:*"). The current time-out threshold is 60 seconds; a 504 error ("Request timed out after waiting for 1m") will be returned if the request times out. The suggestion is to make the kind parameter as narrow as possible while using the sort feature.
### Sort by "nested" arrays objects <a name="nested-sort"></a>
We generally have several objects in each "nested" array.
The question is, how to choose the properties of which of them will be used for sorting of the owning documents.
For this, there is the third parameter "mode" of the sorting function: ```nested(path, field, mode)```.
The modes are min, max, avg. eg., with 'min' we sort by the less field value of objects of each array.
In the following example we apply two levels of sorting by different fields of the "nested" Markers array objects.
For the first level we use 'min' mode and then ASC sorting order, for the second level - 'max' mode and then DESC sorting order.
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"sort": {
"field": ["nested(data.Markers, MarkerMeasuredDepth, min)", "nested(data.Markers, SurfaceDipAzimuth, max)"],
"order": ["ASC", "DESC"]
}
}
```
## Aggregation <a name="aggregation"></a>
Aggregation feature leads to "aggregations" node appearing in the result output.
It contains an array of objects containing two fields: "key" and "count",
reflecting the set of all distinct values of the field chosen for aggregation with the corresponding occurrences number of each value found.
Let's aggregate wellboremarkerset WPCs by their kind
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"aggregateBy": "kind"
}
```
As expected, all wellboremarkerset WOCs belong to the same kind, so, we can see one "key" in the "aggregations" result array:
```json
{
"results": [
{
...
"aggregations": [
{
"key": "osdu:wks:work-product-component--WellboreMarkerSet:1.0.0",
"count": 26
}
],
"totalCount": 26
}
}
}
```
### Aggregation by "nested" arrays objects <a name="nested-aggregation"></a>
Let's aggregate wellboremarkerset WPCs by their Markers "nested" array MarkerMeasuredDepth field
```json
{
"kind" : "osdu:wks:work-product-component--wellboremarkerset:1.0.0",
"aggregateBy": "nested(data.Markers, MarkerMeasuredDepth)"
}
```
Each wellboremarkerset WPC in our test env index has many Markers with most the same test values.
This is why we see multiple "keys" with "count" > 1 in the "aggregations" result array:
{
"aggregations": [
{"key": "3043.0", "count": 48 },
...
{"key": "3544.0","count": 48 },
...
]
}
## Range Queries <a name="range-queries"></a>
Ranges can be specified for date, numeric or string fields. Inclusive ranges are specified with square brackets `[min TO max]` and exclusive ranges with curly brackets `{min TO max}`. Here are some of the examples:
......
......@@ -20,6 +20,7 @@ import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.geo.builders.CircleBuilder;
import org.elasticsearch.common.geo.builders.CoordinatesBuilder;
......@@ -32,6 +33,8 @@ import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.Aggregation;
import org.elasticsearch.search.aggregations.bucket.nested.ParsedNested;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
......@@ -48,14 +51,16 @@ import org.opengroup.osdu.search.policy.service.IPolicyService;
import org.opengroup.osdu.search.policy.service.PartitionPolicyStatusService;
import org.opengroup.osdu.search.provider.aws.service.FieldMappingTypeService;
import org.opengroup.osdu.search.provider.interfaces.IProviderHeaderService;
import org.opengroup.osdu.search.util.AggregationParserUtil;
import org.opengroup.osdu.search.util.CrossTenantUtils;
import org.opengroup.osdu.search.util.IQueryParserUtil;
import org.opengroup.osdu.search.util.ISortParserUtil;
import org.springframework.beans.factory.annotation.Autowired;
import javax.inject.Inject;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.*;
import java.util.stream.Collectors;
import static org.elasticsearch.index.query.QueryBuilders.*;
......@@ -77,8 +82,11 @@ abstract class QueryBase {
private IPolicyService iPolicyService;
@Inject
private PartitionPolicyStatusService statusService;
@Autowired
private IQueryParserUtil queryParserUtil;
@Autowired
private ISortParserUtil sortParserUtil;
static final String AGGREGATION_NAME = "agg";
private static final String GEO_SHAPE_INDEXED_TYPE = "geo_shape";
// if returnedField contains property matching from excludes than query result will NOT include that property
......@@ -97,8 +105,8 @@ abstract class QueryBase {
QueryBuilder spatialQueryBuilder = null;
QueryBuilder queryBuilder = null;
if (StringUtils.isNotEmpty(simpleQuery)) {
textQueryBuilder = getSimpleQuery(simpleQuery);
if (!Strings.isNullOrEmpty(simpleQuery)) {
textQueryBuilder = queryParserUtil.buildQueryBuilderFromQueryString(simpleQuery);
}
// use only one of the spatial request
......@@ -242,7 +250,13 @@ abstract class QueryBase {
List<AggregationResponse> results = null;
if (searchResponse.getAggregations() != null) {
Terms kindAgg = searchResponse.getAggregations().get(AGGREGATION_NAME);
Terms kindAgg = null;
ParsedNested nested = searchResponse.getAggregations().get(AggregationParserUtil.NESTED_AGGREGATION_NAME);
if(nested != null){
kindAgg = (Terms) getTermsAggregationFromNested(nested);
}else {
kindAgg = searchResponse.getAggregations().get(AggregationParserUtil.TERM_AGGREGATION_NAME);
}
if (kindAgg.getBuckets() != null) {
results = new ArrayList<>();
for (Terms.Bucket bucket : kindAgg.getBuckets()) {
......@@ -254,6 +268,15 @@ abstract class QueryBase {
return results;
}
private Aggregation getTermsAggregationFromNested(ParsedNested parsedNested){
ParsedNested nested = parsedNested.getAggregations().get(AggregationParserUtil.NESTED_AGGREGATION_NAME);
if(nested != null){
return getTermsAggregationFromNested(nested);
}else {
return parsedNested.getAggregations().get(AggregationParserUtil.TERM_AGGREGATION_NAME);
}
}
SearchSourceBuilder createSearchSourceBuilder(Query request) throws IOException {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
......@@ -277,10 +300,10 @@ abstract class QueryBase {
// It will ignore the characters longer than the threshold when sorting.
if (request.getSort() != null) {
for (int idx = 0; idx < request.getSort().getField().size(); idx++) {
sourceBuilder.sort(new FieldSortBuilder(request.getSort().getFieldByIndex(idx))
.order(SortOrder.fromString(request.getSort().getOrderByIndex(idx).name()))
.missing("_last")
.unmappedType("keyword"));
sourceBuilder.sort(sortParserUtil.parseSort(
request.getSort().getFieldByIndex(idx),
request.getSort().getOrderByIndex(idx).name())
);
}
}
......@@ -328,6 +351,8 @@ abstract class QueryBase {
default:
throw new AppException(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Search error", "Error processing search request", e);
}
} catch (AppException e){
throw e;
} catch (IOException e) {
if (e.getMessage().startsWith("listener timeout after waiting for")) {
throw new AppException(HttpServletResponse.SC_GATEWAY_TIMEOUT, "Search error", String.format("Request timed out after waiting for %sm", REQUEST_TIMEOUT.getMinutes()), e);
......
......@@ -28,6 +28,7 @@ import org.opengroup.osdu.search.config.SearchConfigurationProperties;
import org.opengroup.osdu.search.logging.AuditLogger;
import org.opengroup.osdu.search.provider.interfaces.IQueryService;
import org.opengroup.osdu.search.util.ElasticClientHandler;
import org.opengroup.osdu.search.util.IAggregationParserUtil;
import org.opengroup.osdu.search.util.QueryResponseUtil;
import org.springframework.stereotype.Service;
......@@ -49,6 +50,8 @@ public class QueryServiceAwsImpl extends QueryBase implements IQueryService {
private SearchConfigurationProperties configurationProperties;
@Inject
private QueryResponseUtil queryResponseUtil;
@Inject
private IAggregationParserUtil aggregationParserUtil;
@Override
public QueryResponse queryIndex(QueryRequest searchRequest) throws IOException {
......@@ -97,10 +100,7 @@ public class QueryServiceAwsImpl extends QueryBase implements IQueryService {
sourceBuilder.from(searchRequest.getFrom());
if (StringUtils.isNotEmpty(searchRequest.getAggregateBy())) {
TermsAggregationBuilder termsAggregationBuilder = new TermsAggregationBuilder(AGGREGATION_NAME);
termsAggregationBuilder.field(searchRequest.getAggregateBy());
termsAggregationBuilder.size(configurationProperties.getAggregationSize());
sourceBuilder.aggregation(termsAggregationBuilder);
sourceBuilder.aggregation(aggregationParserUtil.parseAggregation(searchRequest.getAggregateBy()));
}
elasticSearchRequest.source(sourceBuilder);
......
......@@ -14,14 +14,13 @@
package org.opengroup.osdu.search.provider.azure.provider.impl;
import com.google.common.base.Strings;
import com.google.common.collect.Sets;
import org.apache.commons.lang3.StringUtils;
import org.elasticsearch.ElasticsearchStatusException;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.geo.builders.CircleBuilder;
import org.elasticsearch.common.geo.builders.CoordinatesBuilder;
......@@ -34,6 +33,8 @@ import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.Aggregation;
import org.elasticsearch.search.aggregations.bucket.nested.ParsedNested;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
......@@ -50,7 +51,10 @@ import org.opengroup.osdu.search.policy.service.IPolicyService;
import org.opengroup.osdu.search.policy.service.PartitionPolicyStatusService;
import org.opengroup.osdu.search.provider.azure.service.FieldMappingTypeService;
import org.opengroup.osdu.search.provider.interfaces.IProviderHeaderService;
import org.opengroup.osdu.search.util.AggregationParserUtil;
import org.opengroup.osdu.search.util.CrossTenantUtils;
import org.opengroup.osdu.search.util.IQueryParserUtil;
import org.opengroup.osdu.search.util.ISortParserUtil;
import org.springframework.beans.factory.annotation.Autowired;
import javax.inject.Inject;
......@@ -77,8 +81,11 @@ abstract class QueryBase {
private IPolicyService iPolicyService;
@Inject
private PartitionPolicyStatusService statusService;
@Autowired
private IQueryParserUtil queryParserUtil;
@Autowired
private ISortParserUtil sortParserUtil;
static final String AGGREGATION_NAME = "agg";
private static final String GEO_SHAPE_INDEXED_TYPE = "geo_shape";
// if returnedField contains property matching from excludes than query result will NOT include that property
......@@ -98,7 +105,7 @@ abstract class QueryBase {
QueryBuilder queryBuilder = null;
if (!Strings.isNullOrEmpty(simpleQuery)) {
textQueryBuilder = getSimpleQuery(simpleQuery);
textQueryBuilder = queryParserUtil.buildQueryBuilderFromQueryString(simpleQuery);
}
// use only one of the spatial request
......@@ -242,7 +249,13 @@ abstract class QueryBase {
List<AggregationResponse> results = null;
if (searchResponse.getAggregations() != null) {
Terms kindAgg = searchResponse.getAggregations().get(AGGREGATION_NAME);
Terms kindAgg = null;
ParsedNested nested = searchResponse.getAggregations().get(AggregationParserUtil.NESTED_AGGREGATION_NAME);
if(nested != null){
kindAgg = (Terms) getTermsAggregationFromNested(nested);
}else {
kindAgg = searchResponse.getAggregations().get(AggregationParserUtil.TERM_AGGREGATION_NAME);
}
if (kindAgg.getBuckets() != null) {
results = new ArrayList<>();
for (Terms.Bucket bucket : kindAgg.getBuckets()) {
......@@ -254,6 +267,15 @@ abstract class QueryBase {
return results;
}
private Aggregation getTermsAggregationFromNested(ParsedNested parsedNested){
ParsedNested nested = parsedNested.getAggregations().get(AggregationParserUtil.NESTED_AGGREGATION_NAME);
if(nested != null){
return getTermsAggregationFromNested(nested);
}else {
return parsedNested.getAggregations().get(AggregationParserUtil.TERM_AGGREGATION_NAME);
}
}
SearchSourceBuilder createSearchSourceBuilder(Query request) throws IOException {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
......@@ -277,10 +299,10 @@ abstract class QueryBase {
// It will ignore the characters longer than the threshold when sorting.
if (request.getSort() != null) {
for (int idx = 0; idx < request.getSort().getField().size(); idx++) {
sourceBuilder.sort(new FieldSortBuilder(request.getSort().getFieldByIndex(idx))
.order(SortOrder.fromString(request.getSort().getOrderByIndex(idx).name()))
.missing("_last")
.unmappedType("keyword"));
sourceBuilder.sort(sortParserUtil.parseSort(
request.getSort().getFieldByIndex(idx),
request.getSort().getOrderByIndex(idx).name())
);
}
}
......@@ -328,6 +350,8 @@ abstract class QueryBase {
default:
throw new AppException(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Search error", "Error processing search request", e);
}
} catch (AppException e) {
throw e;
} catch (IOException e) {
if (e.getMessage().startsWith("listener timeout after waiting for")) {
throw new AppException(HttpServletResponse.SC_GATEWAY_TIMEOUT, "Search error", String.format("Request timed out after waiting for %sm", REQUEST_TIMEOUT.getMinutes()), e);
......
......@@ -27,6 +27,7 @@ import org.opengroup.osdu.search.config.SearchConfigurationProperties;
import org.opengroup.osdu.search.logging.AuditLogger;
import org.opengroup.osdu.search.provider.interfaces.IQueryService;
import org.opengroup.osdu.search.util.ElasticClientHandler;
import org.opengroup.osdu.search.util.IAggregationParserUtil;
import org.opengroup.osdu.search.util.QueryResponseUtil;
import org.springframework.stereotype.Service;
......@@ -46,6 +47,8 @@ public class QueryServiceImpl extends QueryBase implements IQueryService {
private SearchConfigurationProperties configurationProperties;
@Inject
private QueryResponseUtil queryResponseUtil;
@Inject
private IAggregationParserUtil aggregationParserUtil;
@Override
public QueryResponse queryIndex(QueryRequest searchRequest) throws IOException {
......@@ -91,10 +94,7 @@ public class QueryServiceImpl extends QueryBase implements IQueryService {
// aggregation
if (!Strings.isNullOrEmpty(searchRequest.getAggregateBy())) {
TermsAggregationBuilder termsAggregationBuilder = new TermsAggregationBuilder(AGGREGATION_NAME);
termsAggregationBuilder.field(searchRequest.getAggregateBy());
termsAggregationBuilder.size(configurationProperties.getAggregationSize());
sourceBuilder.aggregation(termsAggregationBuilder);
sourceBuilder.aggregation(aggregationParserUtil.parseAggregation(searchRequest.getAggregateBy()));
}
elasticSearchRequest.source(sourceBuilder);
......
......@@ -36,6 +36,7 @@ import org.locationtech.jts.geom.Coordinate;
import org.mockito.ArgumentCaptor;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Spy;
import org.mockito.junit.MockitoJUnitRunner;
import org.opengroup.osdu.core.common.logging.JaxRsDpsLog;
import org.opengroup.osdu.core.common.model.http.AppError;
......@@ -45,14 +46,21 @@ import org.opengroup.osdu.core.common.model.search.Point;
import org.opengroup.osdu.core.common.model.search.QueryRequest;
import org.opengroup.osdu.core.common.model.search.QueryResponse;
import org.opengroup.osdu.core.common.model.search.SpatialFilter;
import org.opengroup.osdu.search.config.SearchConfigurationProperties;
import org.opengroup.osdu.search.logging.AuditLogger;
import org.opengroup.osdu.search.provider.azure.service.FieldMappingTypeService;
import org.opengroup.osdu.search.provider.interfaces.IProviderHeaderService;
import org.opengroup.osdu.search.util.AggregationParserUtil;
import org.opengroup.osdu.search.util.CrossTenantUtils;
import org.opengroup.osdu.search.util.ElasticClientHandler;
import java.io.IOException;
import java.util.*;
import org.opengroup.osdu.search.util.IAggregationParserUtil;
import org.opengroup.osdu.search.util.IQueryParserUtil;
import org.opengroup.osdu.search.util.ISortParserUtil;
import org.opengroup.osdu.search.util.QueryParserUtil;
import org.opengroup.osdu.search.util.SortParserUtil;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
......@@ -121,6 +129,18 @@ public class QueryServiceImplTest {
@Mock
private FieldMappingTypeService fieldMappingTypeService;
@Spy
private SearchConfigurationProperties properties = new SearchConfigurationProperties();
@Spy
private IQueryParserUtil parserService = new QueryParserUtil();
@Spy
private ISortParserUtil sortParserUtil = new SortParserUtil();
@Spy
private IAggregationParserUtil aggregationParserUtil = new AggregationParserUtil(properties);
@InjectMocks
private QueryServiceImpl sut;
......
......@@ -14,14 +14,13 @@
package org.opengroup.osdu.search.provider.byoc.provider.impl;
import com.google.common.base.Strings;
import com.google.common.collect.Sets;
import org.apache.commons.lang3.StringUtils;
import org.elasticsearch.ElasticsearchStatusException;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.text.Text;
import org.elasticsearch.common.unit.DistanceUnit;
......@@ -30,6 +29,8 @@ import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.Aggregation;
import org.elasticsearch.search.aggregations.bucket.nested.ParsedNested;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
......@@ -44,6 +45,9 @@ import org.opengroup.osdu.core.common.logging.JaxRsDpsLog;
import org.opengroup.osdu.search.policy.service.IPolicyService;
import org.opengroup.osdu.search.policy.service.PartitionPolicyStatusService;
import org.opengroup.osdu.search.provider.interfaces.IProviderHeaderService;
import org.opengroup.osdu.search.util.AggregationParserUtil;
import org.opengroup.osdu.search.util.IQueryParserUtil;
import org.opengroup.osdu.search.util.ISortParserUtil;
import org.springframework.beans.factory.annotation.Autowired;
import javax.inject.Inject;
......@@ -68,8 +72,11 @@ abstract class QueryBase {
private IPolicyService iPolicyService;
@Inject
private PartitionPolicyStatusService statusService;
@Autowired
private IQueryParserUtil queryParserUtil;
@Autowired