Add alternative support for Skip parameter to facilitate flexible navigation within small datasets

[CalvinQuark]

When I put this question to ChatGPT:

The Azure Data API Builder doesn't support integer Skip parameters. Instead, it supports endCursor bookmarks with hasNextPage flags. What are the advantages and disadvantages of this latter approach versus Skip and Take?

The ChatGPT bot responded:

The Azure Data API Builder's approach of using endCursor bookmarks with hasNextPage flags, compared to the traditional Skip and Take (or Limit and Offset) pagination methods, offers several advantages and disadvantages:

Advantages of endCursor with hasNextPage:

Performance Efficiency: When dealing with large datasets, Skip can be inefficient as the database needs to count and skip over a large number of rows. Using endCursor allows for faster queries since it directly accesses the point where the last query ended.

Consistency in Data: This method helps avoid issues with data modification during pagination. With Skip and Take, if new data is added or existing data is removed, the pages might contain duplicate or missing items. endCursor ensures a consistent view of data, as each page is based on the state of the data at the time of the query.

Stateful Pagination: It provides a way to bookmark a particular state of data. This is particularly useful in scenarios where users need to resume their browsing from a specific point.

Disadvantages of endCursor with hasNextPage:

Complexity in Implementation: Implementing this method can be more complex than the straightforward Skip and Take. It requires a mechanism to generate and interpret the endCursor values.

Limited Flexibility: This approach does not easily allow users to jump to a specific page. It's designed for sequential access, which might not be ideal in all scenarios.

Dependency on Stable Sorting: This method typically depends on the data having a stable and consistent sorting order, which can be an issue if the underlying data changes frequently.

Cursor Value Exposure: The endCursor values might expose some information about the database structure or data ordering, which could be a concern from a security perspective.

Comparison to Skip and Take:

Performance: Skip can be less efficient on large datasets.
Data Consistency: Skip and Take are more prone to issues with data modification during pagination.
Implementation: Skip and Take are simpler to implement.
User Experience: Skip and Take allow for more flexible navigation, such as jumping to a specific page.
Applicability: Skip and Take are more universal and don't rely as heavily on the underlying data's stability.

In conclusion, the choice between these two methods depends on the specific requirements of the application, such as the size of the dataset, the nature of user interaction with data, and the importance of query performance versus implementation complexity.

---

Clearly there are benefits to both approaches. So why not support them both?

This bullet is the best motivation for doing so:

Limited Flexibility: This approach does not easily allow users to jump to a specific page. It's designed for sequential access, which might not be ideal in all scenarios.