Skip to content
Snippets Groups Projects
Commit 7129dc15 authored by Sviatoslav Nekhaienko's avatar Sviatoslav Nekhaienko
Browse files

Merge branch 'master' of...

Merge branch 'master' of https://community.opengroup.org/osdu/platform/system/indexer-service into fix-integer-parsing

 Conflicts:
	testing/indexer-test-core/src/main/resources/features/indexrecord/indexRecord-schema-service.feature
parents 7aca7574 47e8a38c
No related branches found
No related tags found
1 merge request!77fix unexpected conversion from int to double
Showing
with 770 additions and 15 deletions
......@@ -180,6 +180,7 @@ The following software have components provided under the terms of this license:
- Apache Commons BeanUtils (from http://commons.apache.org/proper/commons-beanutils/)
- Apache Commons CLI (from http://commons.apache.org/proper/commons-cli/)
- Apache Commons Codec (from http://commons.apache.org/proper/commons-codec/)
- Apache Commons Collections (from http://commons.apache.org/proper/commons-collections/)
- Apache Commons Lang (from http://commons.apache.org/proper/commons-lang/)
- Apache Commons Logging (from http://commons.apache.org/proper/commons-logging/)
- Apache Commons Validator (from http://commons.apache.org/proper/commons-validator/)
......@@ -311,6 +312,7 @@ The following software have components provided under the terms of this license:
- Java Native Access (from https://github.com/java-native-access/jna)
- Java Native Access Platform (from https://github.com/java-native-access/jna)
- Java Servlet API (from http://servlet-spec.java.net)
- Java UUID Generator (from http://wiki.fasterxml.com/JugHome)
- Javassist (from http://www.javassist.org/)
- Javassist (from http://www.javassist.org/)
- Jetty Server (from )
......@@ -376,12 +378,13 @@ The following software have components provided under the terms of this license:
- Microsoft Application Insights Java SDK Spring Boot starter (from https://github.com/Microsoft/ApplicationInsights-Java)
- Microsoft Application Insights Java SDK Web Module (from https://github.com/Microsoft/ApplicationInsights-Java)
- Microsoft Application Insights Log4j 2 Appender (from https://github.com/Microsoft/ApplicationInsights-Java)
- Microsoft Azure Java Core Library (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure Netty HTTP Client Library (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK for SQL API of Azure Cosmos DB Service (from https://github.com/Azure/azure-sdk-for-java)
- Mockito (from http://mockito.org)
- Mockito (from http://mockito.org)
- Mockito (from http://www.mockito.org)
- Mockito (from http://mockito.org)
- Mockito (from http://www.mockito.org)
- Mojo's Maven plugin for Cobertura (from http://mojo.codehaus.org/cobertura-maven-plugin/)
- MongoDB Driver (from http://www.mongodb.org)
- MongoDB Java Driver Core (from http://www.mongodb.org)
......@@ -858,6 +861,13 @@ The following software have components provided under the terms of this license:
- Project Lombok (from https://projectlombok.org)
- javax.ws.rs-api (from http://jax-rs-spec.java.net)
========================================================================
JSON
========================================================================
The following software have components provided under the terms of this license:
- JSON in Java (from https://github.com/douglascrockford/JSON-java)
========================================================================
LGPL-2.1-only
========================================================================
......@@ -909,6 +919,7 @@ The following software have components provided under the terms of this license:
- Animal Sniffer Annotations (from )
- Azure AD Spring Security Integration Spring Boot Starter (from https://github.com/Microsoft/azure-spring-boot)
- Azure Java Client Authentication Library for AutoRest (from https://github.com/Azure/autorest-clientruntime-for-java)
- Azure Java Client Runtime for ARM (from https://github.com/Azure/autorest-clientruntime-for-java)
- Azure Java Client Runtime for AutoRest (from https://github.com/Azure/autorest-clientruntime-for-java)
- Azure Metrics Spring Boot Starter (from https://github.com/Microsoft/azure-spring-boot)
- Azure Spring Boot AutoConfigure (from https://github.com/Microsoft/azure-spring-boot)
......@@ -932,10 +943,13 @@ The following software have components provided under the terms of this license:
- Microsoft Azure Java Core Library (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure Netty HTTP Client Library (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK annotations (from https://github.com/Microsoft/java-api-annotations)
- Microsoft Azure SDK for EventGrid Management (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK for SQL API of Azure Cosmos DB Service (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK for Service Bus (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK for eventgrid (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure client library for Blob Storage (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure client library for Identity (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure client library for KeyVault Keys (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure client library for KeyVault Secrets (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure common module for Storage (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure internal Avro module for Storage (from https://github.com/Azure/azure-sdk-for-java)
......@@ -952,6 +966,8 @@ The following software have components provided under the terms of this license:
- Spongy Castle (from http://rtyley.github.io/spongycastle/)
- Spring Data for Azure Cosmos DB SQL API (from https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/cosmos/azure-spring-data-cosmos)
- adal4j (from https://github.com/AzureAD/azure-activedirectory-library-for-java)
- azure-documentdb (from https://azure.microsoft.com/en-us/services/cosmos-db/)
- documentdb-bulkexecutor (from http://azure.microsoft.com/en-us/services/documentdb/)
- micrometer-core (from https://github.com/micrometer-metrics/micrometer)
- msal4j (from https://github.com/AzureAD/microsoft-authentication-library-for-java)
- msal4j-persistence-extension (from https://github.com/AzureAD/microsoft-authentication-extensions-for-java)
......@@ -1048,6 +1064,7 @@ The following software have components provided under the terms of this license:
- Joda-Time (from http://www.joda.org/joda-time/)
- LatencyUtils (from http://latencyutils.github.io/LatencyUtils/)
- Microsoft Application Insights Java SDK Core (from https://github.com/Microsoft/ApplicationInsights-Java)
- Microsoft Azure SDK for EventGrid Management (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure SDK for SQL API of Azure Cosmos DB Service (from https://github.com/Azure/azure-sdk-for-java)
- Microsoft Azure client library for Blob Storage (from https://github.com/Azure/azure-sdk-for-java)
- Project Lombok (from https://projectlombok.org)
......@@ -1056,6 +1073,7 @@ The following software have components provided under the terms of this license:
- Spring Security JWT Library (from http://github.com/spring-projects/spring-security-oauth)
- Spring Web (from https://github.com/spring-projects/spring-framework)
- StAX API (from http://stax.codehaus.org/)
- azure-documentdb (from https://azure.microsoft.com/en-us/services/cosmos-db/)
- msal4j (from https://github.com/AzureAD/microsoft-authentication-library-for-java)
- reactive-streams (from http://www.reactive-streams.org/)
- xml-apis (from )
......
......@@ -87,6 +87,8 @@ spec:
value: osdu-db
- name: servicebus_topic_name
value: indexing-progress
- name: reindex_topic_name
value: recordstopic
- name: entitlements_service_endpoint
value: http://entitlements/api/entitlements/v2
- name: entitlements_service_api_key
......
......@@ -8,6 +8,7 @@
- [Copy Index <a name="copy-index"></a>](#copy-index)
- [Get task status <a name="get-task-status"></a>](#get-task-status)
- [Schema Service adoption <a name="schema-service-adoption"></a>](#schema-service-adoption)
- [R3 Schema Support <a name="r3-schema-support"></a>](#r3-schema-support)
##Introduction <a name="introduction"></a>
......@@ -251,7 +252,7 @@ API will respond with status of task.
[Back to table of contents](#TOC)
##Shema Service adoption <a name="schema-service-adoption"></a>
##Schema Service adoption <a name="schema-service-adoption"></a>
Indexer service is in adaptation process to use schemas from the Schema service instead of Storage Service.
The Indexer Service retrieves a schema from the Schema Service if the schema is not found on the Storage Service.
......@@ -260,3 +261,38 @@ Later call to the Storage Service will be deprecated and then removed (after the
[Back to table of contents](#TOC)
###R3 Schema Support <a name="r3-schema-support"></a>
Indexer service support r3 schema. These schemas are created via Schema service.
Here is an example following end-to-end workflow can be exercised (please update the schema based on your environment):
* Ingest r3 schema for `opendes:wks:master-data--Wellbore:1.0.0`. Schema service payload can be found [here](https://community.opengroup.org/osdu/platform/system/indexer-service/-/blob/master/testing/indexer-test-core/src/main/resources/testData/r3-index_record_wks_master.schema.json).
* Ingest r3 master-data Wellbore record. Storage service payload can be found [here](https://community.opengroup.org/osdu/platform/system/indexer-service/-/blob/master/testing/indexer-test-core/src/main/resources/testData/r3-index_record_wks_master.json)
* Records can be searched via Search service. Here is sample payload:
```
POST /api/search/v2/query HTTP/1.1
Content-Type: application/json
data-partition-id: opendes
{
"kind": "opendes:wks:master-data--Wellbore:1.0.0",
"spatialFilter": {
"field": "data.SpatialLocation.Wgs84Coordinates",
"byBoundingBox": {
"topLeft": {
"longitude": -100.0,
"latitude": 52.0
},
"bottomRight": {
"longitude": 100.0,
"latitude": 0.0
}
}
}
}
```
[Back to table of contents](#TOC)
......@@ -16,7 +16,7 @@
<properties>
<commons-beanutils.version>1.9.4</commons-beanutils.version>
<osdu.oscorecommon.version>0.6.5</osdu.oscorecommon.version>
<osdu.oscorecommon.version>0.6.9</osdu.oscorecommon.version>
</properties>
<dependencies>
......
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.Data;
import java.util.HashMap;
import java.util.Map;
@Data
public class Feature extends GeoJsonObject {
@JsonInclude()
private Map<String, Object> properties = new HashMap<>();
@JsonInclude()
private GeoJsonObject geometry;
private String id;
public void setProperty(String key, Object value) {
properties.put(key, value);
}
@SuppressWarnings("unchecked")
public <T> T getProperty(String key) {
return (T) properties.get(key);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import lombok.Data;
import org.opengroup.osdu.indexer.model.geojson.jackson.FeatureCollectionSerializer;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
@Data
@JsonSerialize(using = FeatureCollectionSerializer.class)
public class FeatureCollection extends GeoJsonObject implements Iterable<Feature> {
private List<Feature> features = new ArrayList<Feature>();
public FeatureCollection add(Feature feature) {
features.add(feature);
return this;
}
public void addAll(Collection<Feature> features) {
this.features.addAll(features);
}
@Override
public Iterator<Feature> iterator() {
return features.iterator();
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonInclude.Include;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeInfo.Id;
import lombok.Data;
@Data
@JsonTypeInfo(property = "type", use = Id.NAME)
@JsonSubTypes({ @Type(Feature.class), @Type(Polygon.class), @Type(MultiPolygon.class), @Type(FeatureCollection.class),
@Type(Point.class), @Type(MultiPoint.class), @Type(MultiLineString.class), @Type(LineString.class),
@Type(GeometryCollection.class) })
@JsonInclude(Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public abstract class GeoJsonObject implements Serializable {
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import java.util.ArrayList;
import java.util.List;
public abstract class Geometry<T> extends GeoJsonObject {
protected List<T> coordinates = new ArrayList<T>();
public Geometry() {
}
public Geometry(T... elements) {
for (T coordinate : elements) {
coordinates.add(coordinate);
}
}
public Geometry<T> add(T elements) {
coordinates.add(elements);
return this;
}
public List<T> getCoordinates() {
return coordinates;
}
public void setCoordinates(List<T> coordinates) {
this.coordinates = coordinates;
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.Data;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
@Data
public class GeometryCollection extends GeoJsonObject implements Iterable<GeoJsonObject> {
private List<GeoJsonObject> geometries = new ArrayList<GeoJsonObject>();
@Override
public Iterator<GeoJsonObject> iterator() {
return geometries.iterator();
}
public GeometryCollection add(GeoJsonObject geometry) {
geometries.add(geometry);
return this;
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.NoArgsConstructor;
@NoArgsConstructor
public class LineString extends MultiPoint {
public LineString(Position... points) {
super(points);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.NoArgsConstructor;
import java.util.List;
@NoArgsConstructor
public class MultiLineString extends Geometry<List<Position>> {
public MultiLineString(List<Position> line) {
add(line);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.NoArgsConstructor;
@NoArgsConstructor
public class MultiPoint extends Geometry<Position> {
public MultiPoint(Position... points) {
super(points);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.NoArgsConstructor;
import java.util.List;
@NoArgsConstructor
public class MultiPolygon extends Geometry<List<List<Position>>> {
public MultiPolygon(Polygon polygon) {
add(polygon);
}
public MultiPolygon add(Polygon polygon) {
coordinates.add(polygon.getCoordinates());
return this;
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@NoArgsConstructor
@AllArgsConstructor
public class Point extends GeoJsonObject {
private Position coordinates;
public Point(double longitude, double latitude) {
coordinates = new Position(longitude, latitude);
}
public Point(double longitude, double latitude, double altitude) {
coordinates = new Position(longitude, latitude, altitude);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.NoArgsConstructor;
import java.util.Arrays;
import java.util.List;
@NoArgsConstructor
public class Polygon extends Geometry<List<Position>> {
public Polygon(List<Position> polygon) {
add(polygon);
}
public Polygon(Position... polygon) {
add(Arrays.asList(polygon));
}
public void setExteriorRing(List<Position> points) {
if (coordinates.isEmpty()) {
coordinates.add(0, points);
} else {
coordinates.set(0, points);
}
}
@JsonIgnore
public List<Position> getExteriorRing() {
assertExteriorRing();
return coordinates.get(0);
}
@JsonIgnore
public List<List<Position>> getInteriorRings() {
assertExteriorRing();
return coordinates.subList(1, coordinates.size());
}
public List<Position> getInteriorRing(int index) {
assertExteriorRing();
return coordinates.get(1 + index);
}
public void addInteriorRing(List<Position> points) {
assertExteriorRing();
coordinates.add(points);
}
public void addInteriorRing(Position... points) {
assertExteriorRing();
coordinates.add(Arrays.asList(points));
}
private void assertExteriorRing() {
if (coordinates.isEmpty())
throw new RuntimeException("No exterior ring defined");
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import org.opengroup.osdu.indexer.model.geojson.jackson.PositionDeserializer;
import org.opengroup.osdu.indexer.model.geojson.jackson.PositionSerializer;
import java.io.Serializable;
@Getter
@NoArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
@AllArgsConstructor
@JsonDeserialize(using = PositionDeserializer.class)
@JsonSerialize(using = PositionSerializer.class)
public class Position implements Serializable {
private double longitude;
private double latitude;
@JsonIgnore
private double altitude = Double.NaN;
public Position(double longitude, double latitude) {
this.setLongitude(longitude);
this.setLatitude(latitude);
}
public void setLongitude(double longitude) {
if (Double.isNaN(longitude))
throw new IllegalArgumentException("latitude must be number");
if (longitude > 180 || longitude < -180)
throw new IllegalArgumentException("'longitude' value is out of the range [-180, 180]");
this.longitude = longitude;
}
public void setLatitude(double latitude) {
if (Double.isNaN(latitude))
throw new IllegalArgumentException("latitude must be number");
if (latitude > 90 || latitude < -90)
throw new IllegalArgumentException("latitude value is out of the range [-90, 90]");
this.latitude = latitude;
}
public void setAltitude(double altitude) {
this.altitude = altitude;
}
public boolean hasAltitude() {
return !Double.isNaN(altitude);
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson.jackson;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.jsontype.TypeSerializer;
import lombok.Data;
import org.opengroup.osdu.indexer.model.geojson.*;
import java.io.IOException;
@Data
public class FeatureCollectionSerializer extends JsonSerializer<FeatureCollection> {
@Override
public void serialize(FeatureCollection value, JsonGenerator jsonGenerator, SerializerProvider provider) throws IOException {
jsonGenerator.writeStartObject();
jsonGenerator.writeStringField("type", "geometrycollection");
jsonGenerator.writeArrayFieldStart("geometries");
for (Feature feature : value.getFeatures()) {
jsonGenerator.writeStartObject();
if (feature.getGeometry() instanceof GeometryCollection) {
GeometryCollection geometryCollection = (GeometryCollection) feature.getGeometry();
for (GeoJsonObject shape : geometryCollection.getGeometries()) {
serializeGeoShape(shape, jsonGenerator);
}
} else {
serializeGeoShape(feature.getGeometry(), jsonGenerator);
}
jsonGenerator.writeEndObject();
}
jsonGenerator.writeEndArray();
jsonGenerator.writeEndObject();
}
@Override
public void serializeWithType(FeatureCollection value, JsonGenerator jsonGenerator, SerializerProvider provider, TypeSerializer typeSerializer)
throws IOException, JsonProcessingException {
serialize(value, jsonGenerator, provider);
}
private void serializeGeoShape(GeoJsonObject geoJsonObject, JsonGenerator jsonGenerator) throws IOException {
if (geoJsonObject instanceof Point) {
jsonGenerator.writeStringField("type", "point");
jsonGenerator.writeObjectField("coordinates", ((Point) geoJsonObject).getCoordinates());
} else if (geoJsonObject instanceof LineString) {
jsonGenerator.writeStringField("type", "linestring");
jsonGenerator.writeObjectField("coordinates", ((LineString) geoJsonObject).getCoordinates());
} else if (geoJsonObject instanceof Polygon) {
jsonGenerator.writeStringField("type", "polygon");
jsonGenerator.writeObjectField("coordinates", ((Polygon) geoJsonObject).getCoordinates());
} else if (geoJsonObject instanceof MultiPoint) {
jsonGenerator.writeStringField("type", "multipoint");
jsonGenerator.writeObjectField("coordinates", ((MultiPoint) geoJsonObject).getCoordinates());
} else if (geoJsonObject instanceof MultiLineString) {
jsonGenerator.writeStringField("type", "multilinestring");
jsonGenerator.writeObjectField("coordinates", ((MultiLineString) geoJsonObject).getCoordinates());
} else if (geoJsonObject instanceof MultiPolygon) {
jsonGenerator.writeStringField("type", "multipolygon");
jsonGenerator.writeObjectField("coordinates", ((MultiPolygon) geoJsonObject).getCoordinates());
}
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson.jackson;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import org.opengroup.osdu.indexer.model.geojson.Position;
import java.io.IOException;
public class PositionDeserializer extends JsonDeserializer<Position> {
@Override
public Position deserialize(JsonParser jsonParser, DeserializationContext context) throws IOException {
if (jsonParser.isExpectedStartArrayToken()) {
return deserializeArray(jsonParser, context);
}
throw context.mappingException(Position.class);
}
protected Position deserializeArray(JsonParser jsonParser, DeserializationContext context) throws IOException {
Position node = new Position();
node.setLongitude(extractDouble(jsonParser, context, false));
node.setLatitude(extractDouble(jsonParser, context, false));
node.setAltitude(extractDouble(jsonParser, context, true));
return node;
}
private double extractDouble(JsonParser jsonParser, DeserializationContext context, boolean optional) throws IOException {
JsonToken token = jsonParser.nextToken();
if (token == null) {
if (optional)
return Double.NaN;
else
throw context.mappingException("Unexpected end-of-input when binding data into Position");
} else {
switch (token) {
case END_ARRAY:
if (optional)
return Double.NaN;
else
throw context.mappingException("Unexpected end-of-input when binding data into Position");
case VALUE_NUMBER_FLOAT:
return jsonParser.getDoubleValue();
case VALUE_NUMBER_INT:
return jsonParser.getLongValue();
default:
throw context.mappingException(
"Unexpected token (" + token.name() + ") when binding data into Position");
}
}
}
}
// Copyright © Schlumberger
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package org.opengroup.osdu.indexer.model.geojson.jackson;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import org.opengroup.osdu.indexer.model.geojson.Position;
import java.io.IOException;
public class PositionSerializer extends JsonSerializer<Position> {
@Override
public void serialize(Position value, JsonGenerator jsonGenerator, SerializerProvider provider) throws IOException {
jsonGenerator.writeStartArray();
jsonGenerator.writeNumber(value.getLongitude());
jsonGenerator.writeNumber(value.getLatitude());
if (value.hasAltitude()) {
jsonGenerator.writeNumber(value.getAltitude());
}
jsonGenerator.writeEndArray();
}
}
......@@ -13,6 +13,7 @@
// limitations under the License.
package org.opengroup.osdu.indexer.schema.converter;
import org.apache.http.HttpStatus;
import org.opengroup.osdu.core.common.logging.JaxRsDpsLog;
import org.opengroup.osdu.core.common.model.http.AppException;
......@@ -78,12 +79,16 @@ public class PropertiesProcessor {
String definitionSubRef = ref.substring(DEF_PREFIX.length());
if (schemaConverterConfig.getSkippedDefinitions().contains(definitionSubRef)) {
String definitionIdentity = getDefinitionIdentity(definitionSubRef);
if (schemaConverterConfig.getSkippedDefinitions().contains(definitionIdentity)) {
return Stream.empty();
}
if (Objects.nonNull(schemaConverterConfig.getSpecialDefinitionsMap().get(definitionSubRef))) {
return storageSchemaEntry(schemaConverterConfig.getSpecialDefinitionsMap().get(definitionSubRef), pathPrefix);
if (Objects.nonNull(schemaConverterConfig.getSpecialDefinitionsMap().get(definitionIdentity))) {
return storageSchemaEntry(
schemaConverterConfig.getSpecialDefinitionsMap().get(definitionIdentity) + getDefinitionColonVersion(definitionSubRef),
pathPrefix);
}
Definition definition = definitions.getDefinition(definitionSubRef);
......@@ -101,6 +106,26 @@ public class PropertiesProcessor {
return processProperties(definition.getProperties());
}
private String getDefinitionIdentity(String definitionSubRef) {
String[] components = definitionSubRef.split(":");
if (components.length < 4) {
throw new AppException(HttpStatus.SC_CONFLICT, "Wrong definition format:" + definitionSubRef,
"Wrong definition format:" + definitionSubRef);
}
return components[2];
}
private String getDefinitionColonVersion(String definitionSubRef) {
String[] components = definitionSubRef.split(":");
if (components.length < 4) {
throw new AppException(HttpStatus.SC_CONFLICT, "Wrong definition format:" + definitionSubRef,
"Wrong definition format:" + definitionSubRef);
}
return ":" + components[3];
}
private Stream<Map<String, Object>> processOfItems(List<AllOfItem> allOf, List<AllOfItem> anyOf, List<AllOfItem> oneOf) {
Stream<Map<String, Object>> ofItems = null;
......@@ -119,7 +144,7 @@ public class PropertiesProcessor {
return ofItems;
}
public Stream<Map<String, Object>> processProperties(Map<String, TypeProperty> properties){
public Stream<Map<String, Object>> processProperties(Map<String, TypeProperty> properties) {
return properties.entrySet().stream().flatMap(this::processPropertyEntry);
}
......@@ -175,7 +200,7 @@ public class PropertiesProcessor {
PropertiesProcessor propertiesProcessor = new PropertiesProcessor(definitions, pathPrefixWithDot + entry.getKey()
, log, new SchemaConverterPropertiesConfig());
ofItems = Stream.concat(Optional.ofNullable(ofItems).orElseGet(Stream::empty),
ofItems = Stream.concat(Optional.ofNullable(ofItems).orElseGet(Stream::empty),
entry.getValue().getAnyOf().stream().flatMap(propertiesProcessor::processItem));
}
......@@ -207,7 +232,7 @@ public class PropertiesProcessor {
getFromPattern(definitionProperty.getPattern()),
getFromItemsPattern(() -> definitionProperty.getItems() != null ? definitionProperty.getItems().getPattern() : null),
getFromFormat(definitionProperty::getFormat),
getFromItemsType (() -> definitionProperty.getItems() != null ? definitionProperty.getItems().getType() : null))
getFromItemsType(() -> definitionProperty.getItems() != null ? definitionProperty.getItems().getType() : null))
.filter(x -> x.get() != null)
.findFirst()
.orElse(getFromType(definitionProperty::getType)).get();
......@@ -219,20 +244,20 @@ public class PropertiesProcessor {
private Supplier<String> getFromItemsPattern(Supplier<String> itemsPatternSupplier) {
return () -> {
String itemsPattern = itemsPatternSupplier.get();
return Objects.nonNull(itemsPattern) && itemsPattern.startsWith(LINK_PREFIX) ? LINK_TYPE : null;
};
String itemsPattern = itemsPatternSupplier.get();
return Objects.nonNull(itemsPattern) && itemsPattern.startsWith(LINK_PREFIX) ? LINK_TYPE : null;
};
}
private Supplier<String> getFromType(Supplier<String> typeSupplier) {
return () -> {
return () -> {
String type = typeSupplier.get();
return schemaConverterConfig.getPrimitiveTypesMap().getOrDefault(type, type);
};
}
private Supplier<String> getFromFormat(Supplier<String> formatSupplier){
return () -> {
private Supplier<String> getFromFormat(Supplier<String> formatSupplier) {
return () -> {
String format = formatSupplier.get();
return Objects.nonNull(format) ? schemaConverterConfig.getPrimitiveTypesMap().getOrDefault(format, format) : null;
};
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment