Multi kind search does not work with more than approximately 85 kinds

In ADR: Search data across multiple kinds in one search, we make use of elastic search capability to support multi-kinds to boost the search performance. The solution has been deployed to our clients and the feedback is quite positive in terms of performance improvement. However, one of clients reported an issue that the search returns 400 when the search includes lots of kinds.

The reason is that the elastic search API defines the index names (converted kind names) in the path (as part of URL) and there is length limitation (about 4000 bytes) on the URL in the web server that elastic search uses. Instead letting the elastic search throws exception, in OSDU search service, the validator will validate the length of kinds and throw 400 if it exceeds the limit. For detail, please refer to MultiKindValidator

We did some calculation about the average length of kinds in our test envs, it is about 45 bytes per kind on average. That means the current multi-kinds solution can support up to 85 kinds on average. We did some research on elastic search API. In order to support more kinds, one of the approaches is to make use of Aliases solution in elastic search to reduce the length of index names (converted kinds) passed to the elastic search. The basic idea is to replace the long string index name with short string alias when passing the index names to the elastic search if the index names are too long. If we use hashcode of index name as alias, the alias length is no longer than 10 bytes. That means we can support about 400 kinds which should be enough in most cases.