Skip to content

Draft: Upgrade rest-high-level client to 7.17 to support Elasticsearch 8.

Rustam Lotsmanenko (EPAM) requested to merge elastic-8-migration into master

Description:

Backward-compatible migration to Elastic 8 using client version 7.17.12: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-compatibility.html

No changes are required from the client builder, thanks to the provided ELASTIC_CLIENT_APIVERSIONING=true environment variable that determines whether to send the 7.x compatibility header.

With these changes, Search is able to work with both ElasticSearch v7.* and v8.*.

How to test:

Functionality was tested via int tests simultaneously on both envs, with Elasticsearch 8 and Elasticsearch7.

Info endpoint example for two environments that run the same binaries, but different Elasticsearch versions:

curl --location 'https://community.gcp.gnrg-osdu.projects.epam.com/api/search/v2/info' 
{
    "groupId": "org.opengroup.osdu",
    "artifactId": "search-gc",
    "version": "0.24.0-SNAPSHOT",
    "buildTime": "2023-08-31T14:14:08.894Z",
    "branch": "trusted-elastic-8-migration",
    "commitId": "e6e5b51e23a103c7b6a1b71a34b285a5d1362f6e",
    "commitMessage": "fix merge conflicts",
    "connectedOuterServices": [
        {
            "name": "Redis-RedisCacheBuilder$2",
            "version": null
        },
        {
            "name": "ElasticSearch-osdu",
            "version": "8.9.1"
        },
        {
            "name": "ElasticSearch-second",
            "version": "8.9.1"
        }
    ]
}

curl --location 'https://osdu.ref.gcp.gnrg-osdu.projects.epam.com/api/search/v2/info'
{
    "groupId": "org.opengroup.osdu",
    "artifactId": "search-gc",
    "version": "0.24.0-SNAPSHOT",
    "buildTime": "2023-08-31T14:14:08.894Z",
    "branch": "trusted-elastic-8-migration",
    "commitId": "e6e5b51e23a103c7b6a1b71a34b285a5d1362f6e",
    "commitMessage": "fix merge conflicts",
    "connectedOuterServices": [
        {
            "name": "Redis-RedisCacheBuilder$2",
            "version": null
        },
        {
            "name": "ElasticSearch-osdu",
            "version": "7.17.12"
        }
    ]
}

Changes include:

  • Refactor (a non-breaking change that improves code maintainability).
  • Bugfix (a non-breaking change that solves an issue).
  • New feature (a non-breaking change that adds functionality).
  • Breaking change (a change that is not backward-compatible and/or changes current functionality).

Changes in:

  • GCP
  • Azure
  • AWS
  • IBM
  • Common code

Dev Checklist:

  • Added Unit Tests, wherever applicable.
  • Updated the Readme, if applicable.
  • Existing Tests pass
  • Verified functionality locally
  • Self Reviewed my code for formatting and complex business logic.

Other comments:

The Azure module Geo Queries builders were not migrated due to previous changes that set them apart from similar components in other Cloud Service Providers: !474 (merged)

mvn dependency:tree -Dincludes=*elastic* output:

[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-core ---
[INFO] org.opengroup.osdu:search-core:jar:0.24.0-SNAPSHOT
[INFO] +- org.elasticsearch:elasticsearch:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-core:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-secure-sm:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-x-content:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-geo:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-lz4:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-cli:jar:7.17.12:compile
[INFO] |  \- org.elasticsearch:elasticsearch-plugin-classloader:jar:7.17.12:runtime
[INFO] +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.12:compile
[INFO] \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:mapper-extras-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:parent-join-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:rank-eval-client:jar:7.17.12:compile
[INFO]    \- org.elasticsearch.plugin:lang-mustache-client:jar:7.17.12:compile
[INFO]
[INFO] -------------------< org.opengroup.osdu:search-byoc >-------------------
[INFO] Building search-byoc 0.24.0-SNAPSHOT                               [3/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-byoc ---
[INFO] org.opengroup.osdu:search-byoc:jar:0.24.0-SNAPSHOT
[INFO] \- org.opengroup.osdu:search-core:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.elasticsearch:elasticsearch:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-core:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-secure-sm:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-x-content:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-geo:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-lz4:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-cli:jar:7.17.12:compile
[INFO]    |  \- org.elasticsearch:elasticsearch-plugin-classloader:jar:7.17.12:runtime
[INFO]    +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.9:compile
[INFO]    \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:mapper-extras-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:parent-join-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:rank-eval-client:jar:7.17.12:compile
[INFO]       \- org.elasticsearch.plugin:lang-mustache-client:jar:7.17.12:compile
[INFO]
[INFO] --------------------< org.opengroup.osdu:search-gc >--------------------
[INFO] Building search-gc 0.24.0-SNAPSHOT                                 [4/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-gc ---
[INFO] org.opengroup.osdu:search-gc:jar:0.24.0-SNAPSHOT
[INFO] \- org.opengroup.osdu:search-core:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.elasticsearch:elasticsearch:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-core:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-secure-sm:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-x-content:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-geo:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-lz4:jar:7.17.12:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-cli:jar:7.17.12:compile
[INFO]    |  \- org.elasticsearch:elasticsearch-plugin-classloader:jar:7.17.12:runtime
[INFO]    +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.9:compile
[INFO]    \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:mapper-extras-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:parent-join-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.17.12:compile
[INFO]       +- org.elasticsearch.plugin:rank-eval-client:jar:7.17.12:compile
[INFO]       \- org.elasticsearch.plugin:lang-mustache-client:jar:7.17.12:compile
[INFO]
[INFO] -------------------< org.opengroup.osdu:search-aws >--------------------
[INFO] Building search-aws 0.24.0-SNAPSHOT                                [5/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-aws ---
[INFO] org.opengroup.osdu:search-aws:jar:0.24.0-SNAPSHOT
[INFO] +- org.opengroup.osdu:search-core:jar:0.24.0-SNAPSHOT:compile
[INFO] |  +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.9:compile
[INFO] |  \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.17.12:compile
[INFO] |     +- org.elasticsearch.plugin:mapper-extras-client:jar:7.17.12:compile
[INFO] |     +- org.elasticsearch.plugin:parent-join-client:jar:7.17.12:compile
[INFO] |     +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.17.12:compile
[INFO] |     +- org.elasticsearch.plugin:rank-eval-client:jar:7.17.12:compile
[INFO] |     \- org.elasticsearch.plugin:lang-mustache-client:jar:7.17.12:compile
[INFO] \- org.elasticsearch:elasticsearch:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-core:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-secure-sm:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-x-content:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-geo:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-lz4:jar:7.17.12:compile
[INFO]    +- org.elasticsearch:elasticsearch-cli:jar:7.17.12:compile
[INFO]    \- org.elasticsearch:elasticsearch-plugin-classloader:jar:7.17.12:runtime
[INFO]
[INFO] ------------------< org.opengroup.osdu:search-azure >-------------------
[INFO] Building search-azure 0.24.0-SNAPSHOT                              [6/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-azure ---
[INFO] org.opengroup.osdu:search-azure:jar:0.24.0-SNAPSHOT
[INFO] \- org.opengroup.osdu:search-core:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.elasticsearch:elasticsearch:jar:7.8.1:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-core:jar:7.8.1:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-secure-sm:jar:7.8.1:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-x-content:jar:7.8.1:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-geo:jar:7.8.1:compile
[INFO]    |  +- org.elasticsearch:elasticsearch-cli:jar:7.8.1:compile
[INFO]    |  \- org.elasticsearch:jna:jar:5.5.0:compile
[INFO]    +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.8:compile
[INFO]    \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.8.1:compile
[INFO]       +- org.elasticsearch.plugin:mapper-extras-client:jar:7.8.1:compile
[INFO]       +- org.elasticsearch.plugin:parent-join-client:jar:7.8.1:compile
[INFO]       +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.8.1:compile
[INFO]       +- org.elasticsearch.plugin:rank-eval-client:jar:7.8.1:compile
[INFO]       \- org.elasticsearch.plugin:lang-mustache-client:jar:7.8.1:compile
[INFO]
[INFO] -------------------< org.opengroup.osdu:search-ibm >--------------------
[INFO] Building search-ibm 0.24.0-SNAPSHOT                                [7/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ search-ibm ---
[INFO] org.opengroup.osdu:search-ibm:jar:0.24.0-SNAPSHOT
[INFO] +- org.elasticsearch:elasticsearch:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-core:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-secure-sm:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-x-content:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-geo:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-lz4:jar:7.17.12:compile
[INFO] |  +- org.elasticsearch:elasticsearch-cli:jar:7.17.12:compile
[INFO] |  \- org.elasticsearch:elasticsearch-plugin-classloader:jar:7.17.12:runtime
[INFO] +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.17.9:compile
[INFO] \- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:mapper-extras-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:parent-join-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.17.12:compile
[INFO]    +- org.elasticsearch.plugin:rank-eval-client:jar:7.17.12:compile
[INFO]    \- org.elasticsearch.plugin:lang-mustache-client:jar:7.17.12:compile
[INFO] ------------------------------------------------------------------------
Edited by Rustam Lotsmanenko (EPAM)

Merge request reports