Use ElasticSearch's search_after feature to implement pagination query with cursor

Type of change

• Bug Fix
• Feature

Please provide link to gitlab issue or ADR(Architecture Decision Record)
ADR Pagination Query API

Does this introduce a change in the core logic?

• [YES]

Does this introduce a change in the cloud provider implementation, if so which cloud?

• AWS
• Azure
• Google Cloud
• IBM

Does this introduce a breaking change?

• [NO]

What is the current behavior?

The existing solution of the query with cursor has limitation on the max. number of open cursors (500 by default). Exception will be thrown by ElasticSearch once the limit is reached. Applications using the current query with cursor API have no control on such kind of error.

What is the new/expected behavior?

The new feature (using ElasticSearch search_after to support pagination) will share the same signature of the current query with cursor on the input and output. But it uses different ElasticSearch technique/feature, called search_after, to implement the same functionalities in order to avoid the max. cursor limitation and require less resources.

Have you added/updated Unit Tests and Integration Tests?

1. We replicated the unit tests from the ScrollCoreQueryServiceImplTest.java to SearchAfterQueryServiceImplTest.java to demonstrate the compatibility between these two implementations.
2. We created a way to share the integration tests between query with cursor and query with search_after to further demonstrate the compatibility between these two implementations.

Any other useful information

1. The revised query with search_after is implemented on top of java client. Thanks @Stanislav_Riabokon to make that happened.
2. We created a way to make query with search_after to reuse the integration tests of query with cursor. Any change on the integration tests of query with cursor will be applied to query with search_after.
3. If the solution of query with search_after works as expected in terms of functionality as well as overcoming the limitation of query with cursor, we should replace the current query with cursor with this implementation with the following considerations:

• The adoption of the query with search_after should be transparent to the existing applications of OSDU search
• The adoption of the query with search_after should be transparent to developers with no change on API and documentation
• To be able to conduct large scale of tests in production environment using existing applications of OSDU services before the replacement.

We decided to use feature flag to control the switch between the solutions of query with cursor and query with search_after. Two temporary (internal) endpoints were added to support integration tests and other kinds of tests without the feature flag enabled.