README.md

Indexer Service

os-indexer-gcp is a Spring Boot service that is responsible for indexing Records that enable the os-search service to execute OSDU R2 domain searches against Elasticsearch.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

Pre-requisites

• GCloud SDK with java (latest version)
• JDK 8
• Lombok 1.16 or later
• Maven

Installation

In order to run the service locally or remotely, you will need to have the following environment variables defined.

name	value	description	sensitive?	source
LOG_PREFIX	service	Logging prefix	no	-
SERVER_SERVLET_CONTEXPATH	/api/indexer/v2	Servlet context path	no	-
AUTHORIZE_API	ex https://entitlements.com/entitlements/v1	Entitlements API endpoint	no	output of infrastructure deployment
ENTITLEMENTS_HOST	ex https://entitlements.com/entitlements/v1	Entitlements API endpoint	no	output of infrastructure deployment
LEGALTAG_API	ex https://legal.com/api/legal/v1	Legal API endpoint	no	output of infrastructure deployment
INDEXER_HOST	ex os-indexer-dot-opendes.appspot.com	Indexer Host	no	output of infrastructure deployment
INDEXER_QUEUE_HOST	ex https://os-indexer-queue-dot-opendes.appspot.com/_dps/task-handlers/enqueue	Indexer-Queue API endpoint	no	output of infrastructure deployment
CRS_API	ex https://crs-converter-gae-dot-opendes.appspot.com/api/crs/v1	CRS API endpoint	no	https://console.cloud.google.com/memorystore/redis/instances
STORAGE_HOSTNAME	ex os-storage-dot-opendes.appspot.com	Storage Host	no	output of infrastructure deployment
STORAGE_SCHEMA_HOST	ex https://os-storage-dot-opendes.appspot.com/api/storage/v2/schemas	Storage API endpoint 'schemas'	no	https://console.cloud.google.com/apis/credentials
STORAGE_QUERY_RECORD_FOR_CONVERSION_HOST	ex https://os-storage-dot-opendes.appspot.com/api/storage/v2/query/records:batch	Storage API endpoint 'records'	no	https://console.cloud.google.com/iam-admin/serviceaccounts
REDIS_SEARCH_HOST	ex 127.0.0.1	Redis host for search	no	https://console.cloud.google.com/memorystore/redis/instances
REDIS_GROUP_HOST	ex 127.0.0.1	Redis host for groups	no	https://console.cloud.google.com/memorystore/redis/instances
REDIS_SEARCH_PORT	ex 6379	Redis host for search	no	https://console.cloud.google.com/memorystore/redis/instances
GOOGLE_CLOUD_PROJECT	ex opendes	Google Cloud Project Id	no	output of infrastructure deployment
GOOGLE_AUDIENCES	ex *****.apps.googleusercontent.com	Client ID for getting access to cloud resources	yes	https://console.cloud.google.com/apis/credentials
GOOGLE_APPLICATION_CREDENTIALS	ex /path/to/directory/service-key.json	Service account credentials, you only need this if running locally	yes	https://console.cloud.google.com/iam-admin/serviceaccounts
security.https.certificate.trust	ex false	Elastic client connection uses TrustSelfSignedStrategy(), if it is 'true'	false	output of infrastructure deployment
indexer.que.service.mail	ex default@iam.gserviceaccount.com	IndexerQue environment service account mail, required if Indexer Que deployed in cloud task mode, to validate token from it	yes	-

Run Locally

Check that maven is installed:

$ mvn --version
Apache Maven 3.6.0
Maven home: /usr/share/maven
Java version: 1.8.0_212, vendor: AdoptOpenJDK, runtime: /usr/lib/jvm/jdk8u212-b04/jre
...

You may need to configure access to the remote maven repository that holds the OSDU dependencies. This file should live within ~/.mvn/community-maven.settings.xml:

$ cat ~/.m2/settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
        <server>
            <id>community-maven-via-private-token</id>
            <!-- Treat this auth token like a password. Do not share it with anyone, including Microsoft support. -->
            <!-- The generated token expires on or before 11/14/2019 -->
             <configuration>
              <httpHeaders>
                  <property>
                      <name>Private-Token</name>
                      <value>${env.COMMUNITY_MAVEN_TOKEN}</value>
                  </property>
              </httpHeaders>
             </configuration>
        </server>
    </servers>
</settings>

• Update the Google cloud SDK to the latest version:

gcloud components update

• Set Google Project Id:

gcloud config set project <YOUR-PROJECT-ID>

• Perform a basic authentication in the selected project: