Add flexible page size option in Storage getRecordsByKind API
Get records by Kind API in storage service returns results in the form of multiple pages in case of large number of records. The page size of these pages is constant - it is equal to the limit
specified in the optional query parameter or the default limit configuration set in config file.
Context
Reindex API in indexer service calls the getRecordsByKind storage API to fetch multiple records. We need an option of querying storage without the constant page size constraint for performance related enhancements on azure provider.
Proposed Solution
Add an optional parameter in storage service API, which when set to true will make the API return results with page size <= limit configured. The default behavior will be to return results with page size == limit configured.