Skip to content

Elasticsearch licensing

Problem Statement

Currently, Search service is using Elasticsearch 7.8.1. There is a need to upgrade the version to provide stability, features and performance improvement.

Specifically following the release of version 7.10.2 https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-core/7.10.2 , Elastic has since transitioned its licensing from the Apache 2.0 license to the Server Side Public License (SSPL) for any future versions https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-core.

OSDU software needs to be licensed using Apache 2.0. This change has raised concerns, particularly regarding compatibility issues with client bindings and providing future updates to Elasticsearch.

Impact

There are 2 components to the search - Elastic client bindings and server.

  • Client bindings are integral components of applications that facilitate seamless communication with our Elastic Search Service. These bindings have traditionally been Apache 2.0 compatible. The shift to SSPL raises compatibility concerns, potentially preventing the upgrade of client bindings.

  • The Elastic server itself is used as a tool, so we don’t need to worry about Apache compatibility. Server-side upgrades are possible but may encounter a future technical barrier without client-binding upgrades.

Objective

We need to address this licensing challenge and find an alternative that allows for a smooth transition. We are actively exploring options for an elastic alternative that can bridge the gap between client bindings and server upgrades.

An option №1 is https://opensearch.org/docs/latest/clients/java/

Pros:

  • OpenSearch is an ElasticSearch fork, and fully compatible with v 7.10 see https://opensearch.org/faq/#q1.8. Thus refactoring should be more or less straightforward.
  • Easier to preserve existing features.
  • It's possible to change clients in Services and keep ElastSearch as a backend server.

Cons:

  • Following-up releases do not guarantee compatibility with ElasticSearch API: https://opensearch.org/faq/#q1.9 Action items:

  • Potentially could bind CSPs to ElasticSearch server v 7.10 or force them to switch to OpenSearch server.

  • Switch Indexer and Search to use OpenSearch clients.

Option №2 is an Elasticsearch client with an Apache license https://github.com/elastic/elasticsearch-java/

Pros:

  • Possible to keep Elasticsearch as a backend.
  • Later we could migrate to Elasticsearch 8.

Cons:

  • Could require a bit more thorough migration for Search and Indexer, unlike OpenSearch. Since it's a different lib with different interfaces, we may need to rewrite a lot of code. In the meantime, OpenSearch has a fork of High-level-rest-client https://opensearch.org/docs/latest/clients/java-rest-high-level/ which could simplify migration to just swapping imports.
  • Additionally, we should be aware that the Elasticsearch server's licensing could still pose an issue.

Action items:

  • Migrate Indexer and Search to use Elasticsearch Apache client.

Decisions

Option 2 seems to be a better long-term solution with the possibility of keeping Elasticsearch as a backend. A separate migration strategy has been written here indexer-service#111

Edited by Chad Leong