Skip to content
Snippets Groups Projects
Commit 46300723 authored by Smitha Manjunath's avatar Smitha Manjunath
Browse files

Merge remote-tracking branch 'origin/deleteApi' into deleteApi

# Conflicts:
#	docs/tutorial/IndexerService.md
#	indexer-core/src/main/java/org/opengroup/osdu/indexer/api/CleanupIndiciesApi.java
#	indexer-core/src/main/java/org/opengroup/osdu/indexer/logging/AuditEvents.java
parents a85c624f 1c03ff51
No related branches found
No related tags found
2 merge requests!346Merge branch 'aws-integration' into 'master',!273Delete api
Pipeline #104779 failed
......@@ -5,11 +5,9 @@
- [Indexer service](#indexer-service)
- [Introduction](#introduction)
- [Indexer API access](#indexer-api-access)
- [API Reference](#api-reference)
- [Version info endpoint](#version-info-endpoint)
- [Reindex](#reindex)
- [Data Partition provision](#data-partition-provision)
- [Schema change](#schema-change)
- [Version info endpoint](#version-info-endpoint)
- [Reindex](#reindex)
- [Data Partition provision](#data-partition-provision)
- [Schema Service adoption](#schema-service-adoption)
* [R3 Schema Support](#r3-schema-support)
- [Troubleshoot Indexing Issues](#troubleshoot-indexing-issues)
......@@ -53,17 +51,9 @@ If the service is initiating the request, an ID should be generated. If the `cor
[Back to table of contents](#TOC)
## API Reference
## Version info endpoint
### Version info endpoint
Provides build and git related information.
#### Request
```http
GET /api/indexer/v2/info HTTP/1.1
```
For deployment available public `/info` endpoint, which provides build and git related information.
#### Example response:
......@@ -96,16 +86,20 @@ This endpoint takes information from files, generated by `spring-boot-maven-plug
[Back to table of contents](#TOC)
### Reindex <a name="reindex"></a>
## Reindex <a name="reindex"></a>
Reindex API allows users to re-index a `kind` without re-ingesting the records via storage API. Reindexing a kind is an asynchronous operation and when a user calls this API, it will respond with HTTP 200 if it can launch the re-indexing or
appropriate error code if it cannot. The current status of the indexing can be tracked by calling search API and making query with this particular kind. Please be advised, it may take few seconds to few hours to finish the re-indexing as
Reindex API allows users to re-index a `kind` without re-ingesting the records via storage API. Reindexing a kind is an
asynchronous operation and when a user calls this API, it will respond with HTTP 200 if it can launch the re-indexing or
appropriate error code if it cannot. The current status of the indexing can be tracked by calling search API and making
query with this particular kind. Please be advised, it may take few seconds to few hours to finish the re-indexing as
multiple factors contribute to latency, such as number of records in the kind, current load at the indexer service etc.
#### Request
__Note__: If a kind has been previously indexed with particular schema and if you wish to apply the schema changes
during re-indexing, previous kind index has to be deleted via Index Delete API. In absence of this clean-up, reindex API
will use the same schema and overwrite records with the same ids.
```http
POST /api/indexer/v2/reindex HTTP/1.1
```
POST /api/indexer/v2/reindex
{
"kind": "opendes:welldb:wellbore:1.0.0"
}
......@@ -127,22 +121,6 @@ curl --request POST \
</details>
#### Prerequisite
Users must be a member of `users.datalake.admins` or `users.datalake.ops` group.
#### Query parameters
`force_clean` <br />
&emsp;&emsp;(optional, Boolean) If a kind has been previously indexed with a schema and if you wish to apply latest schema changes before re-indexing, than use this query parameter. It will drop the current Index schema, apply latest schema changes & re-index records. If `false`, reindex API
will use the same schema and overwrite records with the same ids. Default value is `false`.
#### Request body
`kind` <br />
&emsp;&emsp;(required, String) Kind to be re-indexed.
[Back to table of contents](#TOC)
## Delete API <a name="delete"></a>
Delete API is used to delete an index for a specific kind.
......@@ -162,12 +140,12 @@ curl --request DELETE \
--header 'data-partition-id: opendes'
```
### Data Partition provision <a name="data-partition-provision"></a>
## Data Partition provision <a name="data-partition-provision"></a>
Configures Search backend for a data partition.
```http
PUT /api/indexer/v2/partitions/provision HTTP/1.1
```
PUT /api/indexer/v2/partitions/provision
```
<details><summary>**Curl**</summary>
......@@ -182,52 +160,10 @@ curl --request PUT \
```
</details>
#### Prerequisite
Users must be a member of `users.datalake.ops` group.
> __NOTE__: API should be run at-least once at the data partition provisioning to configure required resources/settings.
[Back to table of contents](#TOC)
### Schema change <a name="schema-change"></a>
Schema change event listener endpoint.
> __Note:__ This is internal API and shouldn't be exposed publicly.
#### Request
```http
POST /api/indexer/v2/_dps/task-handlers/schema-worker HTTP/1.1
{
"messageId": "676894654",
"publishTime": "2017-03-19T00:00:00",
"attributes": {
"data-partition-id": "opendes",
"correlation-id": "b5a281bd-f59d-4db2-9939-b2d85036fc7e"
},
"data": "[{\"kind\":\"slb:indexer:test-data--SchemaEventIntegration:1.0.0\",\"op\":\"create\"}]"
}
```
#### Request body
`messageId` <br />
&emsp;&emsp;(optional, String) Event message id.
`publishTime` <br />
&emsp;&emsp;(optional, String) Event publish time.
`attributes.data-partition-id` <br />
&emsp;&emsp;(required, String) Data partition id for which this message is targeted.
`attributes.correlation-id` <br />
&emsp;&emsp;(optional, String) Correlation-id to enable tracing.
`data` <br />
&emsp;&emsp;(required, String) Schema change event message json string. Only `create` and `update` events are supported.
## Schema Service adoption <a name="schema-service-adoption"></a>
Indexer service is in adaptation process to use schemas from the Schema service instead of Storage Service. The Indexer
......
......@@ -31,7 +31,6 @@ import org.opengroup.osdu.core.common.model.indexer.RecordInfo;
import org.opengroup.osdu.core.common.model.search.RecordChangedMessages;
import org.opengroup.osdu.core.common.model.search.SearchServiceRole;
import org.opengroup.osdu.core.common.model.storage.validation.ValidKind;
import org.opengroup.osdu.core.common.model.storage.validation.ValidKind;
import org.opengroup.osdu.core.common.search.ElasticIndexNameResolver;
import org.opengroup.osdu.indexer.SwaggerDoc;
import org.opengroup.osdu.indexer.logging.AuditLogger;
......@@ -125,4 +124,4 @@ public class CleanupIndiciesApi {
throw new AppException(HttpStatus.INTERNAL_SERVER_ERROR.value(), "Unknown error", "An unknown error has occurred.", e);
}
}
}
\ No newline at end of file
}
......@@ -27,6 +27,7 @@ public class AuditEvents {
private static final String INDEX_CREATE_RECORDS_SUCCESS = "Successfully created record in index";
private static final String INDEX_CREATE_RECORDS_FAILURE = "Failed creating record in index";
private static final String INDEX_UPDATE_RECORD_ACTION_ID = "IN002";
private static final String INDEX_UPDATE_RECORDS_SUCCESS = "Successfully updated record in index";
private static final String INDEX_UPDATE_RECORDS_FAILURE = "Failed updating record in index";
......@@ -53,8 +54,8 @@ public class AuditEvents {
private static final String RUN_JOB_MESSAGE_SUCCESS = "Index clean-up status job run success";
private static final String INDEX_MAPPING_UPDATE_ACTION_ID = "IN0011";
private static final String INDEX_MAPPING_UPDATE_SUCCESS = "Successfully upserted index mapping";
private static final String INDEX_MAPPING_UPDATE_FAILURE = "Failed upserting index mapping";
private static final String INDEX_MAPPING_UPDATE_SUCCESS = "Successfully updated index mapping";
private static final String INDEX_MAPPING_UPDATE_FAILURE = "Failed updating index mapping";
private static final String CONFIGURE_PARTITION_ACTION_ID = "IN0012";
private static final String CONFIGURE_PARTITION_OPERATION = "Data partition cluster configuration update";
......@@ -63,10 +64,6 @@ public class AuditEvents {
private static final String INDEX_DELETE_SUCCESS = "Successfully deleted index";
private static final String INDEX_DELETE_FAILURE = "Failed deleting index";
private static final String INDEX_DELETE_ACTION_ID = "IN0012";
private static final String INDEX_DELETE_SUCCESS = "Successfully deleted index";
private static final String INDEX_DELETE_FAILURE = "Failed deleting index";
private final String user;
public AuditEvents(String user) {
......@@ -252,7 +249,7 @@ public class AuditEvents {
.build();
}
public AuditPayload getIndexMappingUpsertEvent(List<String> resources, boolean isSuccess) {
public AuditPayload getIndexMappingUpdateEvent(List<String> resources, boolean isSuccess) {
if (isSuccess) {
return AuditPayload.builder()
.action(AuditAction.UPDATE)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment