Iterate through all data

A single call to “List” will only give you the first 1000 items. Then you get a pageToken which you can use to get the next page of data.

In pseudo code, it works like this:

var data = new List();
var pageToken = null

do
{
   var result = gyxi.GetList(”person”, pageToken);
   data.AddRange(result.List);

   // This will have a value if there are more data and it otherwise null
   pageToken = result.NextPageToken
} while (pageToken != null)

This way it will make a first call to get the first 1000 items. If there are more items to find, it will also return the NextPageToken which is needed in the next call.

Depending on your network connection, you can get 1000 items every half second or less and it rarely gets throttled. If you want to handle throttling as well, you can wrap your GetList call in a check for status codes 429 and 503, which is a first and second level throttling. And then you can retry after 2 seconds and 5 respectively.

Common strategy

The pseudo C# code above indicates a scenario where you want to get ALL the data in a partition. However, it may not be smart to get all the data in a single process in your application.

Another common scenario is that you have an end user, perhaps in a browser, requesting data. In this case, you would show the user the first 1000 rows and pass the NextPageToken back to the user as well. If the user requests more data, the user will pass the NextPageToken back to your application.

The NextPageToken is a simple self contained token which simply contains the id of the next row to fetch along with some technical details encoded.