Exclude system/metadata indices in searches with wildcard kind

Type of change

• Bug Fix
• Feature

Please provide link to gitlab issue or ADR(Architecture Decision Record)

Does this introduce a change in the core logic?

• [YES]

Does this introduce a change in the cloud provider implementation, if so which cloud?

• AWS
• Azure
• Google Cloud
• IBM

Does this introduce a breaking change?

• [NO]

What is the current behavior?

In searches with wildcard kind, only the ElasticSearch system/internal indices started with dot will be excluded.

What is the new/expected behavior?

In searches with wildcard kind, the ElasticSearch system/internal indices started with dot and the indices with authority as "system-meta-data" will be excluded.

Have you added/updated Unit Tests and Integration Tests?

Yes

Any other useful information

• ADR ADR: Exclude indices of the system/meta data from the search results unless the indices (kinds) of the system/meta data are explicitly specified in the search query
• OSDU data-definitions
  • 6.1.2 Record kind
  • Appendix D.1.3 Schema Identifier kind Limitations

changed milestone to %M20 - Release 0.23

added ADRProposed Common Code labels

assigned to @gehrmann and @zhibinmai

added 1 commit

• d4874e82 - Update SearchService.md

Compare with previous version

added MRNew Feature label

added 10 commits

• d4874e82...3b84949e - 8 commits from branch master
• d170b5c9 - Update artifact
• 295812fe - Merge branch 'master' of https://community.opengroup.org/osdu/platform/system/search-service

Compare with previous version

changed the description

added 1 commit

• 61b9f79b - Add references to OSDU data-definitions documents

Compare with previous version

requested review from @nthakur, @Rustam_Lotsmanenko, @Srinivasan_Narayanan, and @ydzeng

Please review this MR: @nthakur, @Rustam_Lotsmanenko, @Srinivasan_Narayanan, and @ydzeng

CC @gehrmann @chad

Thomas has updated the related documents in the data-definitions section:

• 6.1.2 Record kind
• Appendix D.1.3 Schema Identifier kind Limitations

@omprakash_epam @ydzeng @Rustam_Lotsmanenko - this is the related MR for the system metadata - could you help review this ?

added 17 commits

• 61b9f79b...ef85796c - 16 commits from branch master
• e4bfd3e8 - Merge branch 'master' of https://community.opengroup.org/osdu/platform/system/search-service

Compare with previous version

approved this merge request

approved this merge request

Azure Approved.

@ydzeng Can you let your team look at the failed integration for AWS? The change should not affect the integration tests.

Can we please add/update existing integration tests to validate this feature?