GitLab

Problem

The reindexer service tries to call storage service for fetching records of a given kind. But the catch is that these calls are made in batches. In case the number of records for a given size are more than batch size, there is no handling in place to read rest of the records. Hence each time we call reindex api for a certain kind Only Tope 20 records are getting reindexed

FIX

Putting the storage service related calls in a while loop.

Cursor = null
1. call the storage service with batch size = 20 and batch size
2. If no records are found for this kind then break
3. If there are records in current batch then put those records in indexer queue
4. If this was the last batch- terminate, else update the cursor to location of next batch and repeat from step 1

Tested in dev environment for one kind

cc: @manishk , @kiveerap