From 854f95f03766a3d11df2ba3dbb9f3f5a6346e2ce Mon Sep 17 00:00:00 2001 From: Domizio Demichelis Date: Sun, 15 Dec 2024 09:23:30 +0700 Subject: [PATCH] Improve keyset docs --- docs/api/keyset.md | 65 +++++++++++++++++++++++------------- docs/api/keyset_for_ui.md | 12 +++---- docs/extras/keyset_for_ui.md | 4 +-- 3 files changed, 49 insertions(+), 32 deletions(-) diff --git a/docs/api/keyset.md b/docs/api/keyset.md index acdad8a11..b2abebff1 100644 --- a/docs/api/keyset.md +++ b/docs/api/keyset.md @@ -52,7 +52,8 @@ If you want the best of the two worlds, check out the [keyset_for_ui extra](/doc | `set` | The `uniquely ordered` `ActiveRecord::Relation` or `Sequel::Dataset` collection to paginate. | | `keyset` | The hash of column/direction pairs. Pagy extracts it from the order of the `set`. | | `keyset attributes` | The hash of keyset-column/record-value pairs of a record. | -| `cutoff` | A point in the `set` where a `page` ended. Its value is a `Base64` encoded URL-safe string. | +| `keyset attributes values` | The array of the values of the `keyset attributes`. | +| `cutoff` | A point in the `set` where a `page` ends and the `next` begins. It is encoded as a `Base64` URL-safe string. | | `page` | The current `page`, i.e. the page of records beginning after the `cutoff` of the previous page. Also the `:page` variable, which is set to the `cutoff` of the previous page | | `next` | The next `page`, i.e. the page of records beginning after the `cutoff`. Also the `cutoff` value retured by the `next` method. | @@ -164,41 +165,56 @@ If you need a specific order: #### Understanding the Cutoffs -A `cutoff` defines a point in the `set` where a `page` ended. All the records AFTER that point are or will be part of the `next` page. +A `cutoff` defines a point in the `set` where a `page` ended. -Let's consider an example of a simple `set`. In order to avoid confusion with numeric ids and number of records, let's assume that -it has an `id` column populated by unique alphanumeric codes, and its order is: `order(:id)`. +Let's consider an example of a simple `set` of 29 records. In order to avoid confusion with numeric ids and number of records, +let's assume that it has an `id` column populated by character keys, and its order is: `order(:id)`. -Assuming a LIMIT of 6, the first page will include the first 6 records in the set: no `cutoff` required so far... +Assuming a LIMIT of 10, the _"first page"_ will just include the first 10 records in the `set`: no `cutoff` required so far... ``` - | page | not yet paginated | -beginning ->|. . . . . .|. . . . . . . . . . . . . . . . . . . . . . . . . . .|<- end of set + │ first page (10) >│ rest (19) >│ +beginning of set >[· · · · · · · · · ·]· · · · · · · · · · · · · · · · · · ·]< end of set ``` -After we pull the first 6 records from the beginning of the `set`, we read the `id` of the last one, which is `F`. So our `cutoff` can be defined like: _"the point up to the value `F` in the `id` column"_. +After we pull the first 10 records from the beginning of the `set`, we read the `id` of the last one, which is `X`. So our +`cutoff` can be defined like: _"the point up to the value `X` in the `id` column"_. -Notice that this is not like saying _"up to the record `F`"_. It's important to understand that a `cutoff` refers just to a value +Notice that this is not like saying _"up to the record `X`"_. It's important to understand that a `cutoff` refers just to a value in a column (or a combination of multiple column, in case of muti-columns keysets). -Indeed, that very record could be deleted right after we read it, and our `cutoff` will still be the valid reference that _"we paginated the `set`, up to the "F" value"_... +Indeed, that very record could be deleted right after we read it, and our `cutoff-X` will still be the valid reference that we +paginated the `set`, up to the "X" value", cutting off the `page` any further record... + ``` - | page | page | not yet paginated | -beginning ->|. . . . . F]. . . . . .|. . . . . . . . . . . . . . . . . . . . .|<- end of set - | - cutoff-F + │ first page (10) >│ second page (10) >│ rest (9) >│ +beginning of set >[· · · · · · · · · X]· · · · · · · · · ·]· · · · · · · · ·]< end of set + ▲ + cutoff-X ``` -For getting the `next` page of records - this time - we pull the `next` 6 records AFTER the `cutoff-F`. Again, we read the `id` of the last one, which is `L`: so we have our new `cutoff-L`, which is the end of the current `page`, and the `next` will go AFTER it... +For getting the `next` page of records (i.e. the _"second page"_) we pull the `next` 10 records AFTER the `cutoff-X`. Again, we +read the `id` of the last one, which is `Y`: so we have our new `cutoff-Y`, which is the end of the current `page`, and the `next` +will go AFTER it... ``` - | page | page | page | not yet paginated | -beginning ->|. . . . . F]. . . . . L]. . . . . .|. . . . . . . . . . . . . . .|<- end of set - | | - cutoff-F cutoff-L + │ first page (10) >│ second page (10) >│ last page (9) >│ +beginning of set >[· · · · · · · · · X]· · · · · · · · · Y]· · · · · · · · ·]< end of set + ▲ ▲ + cutoff-X cutoff-Y ``` - -Pagy encodes the values of the `cutoffs` in a `Base64` URL-safe string that is sent as a param in the `request`. + +When we pull the `next` page from the `cutoff-Y` we find only the remaining 9 records, which means that it's the _"last page"_, +which doesn't have a `cutoff` because it ends with the end of the `set`. + +#### Keynotes + +- A `cutoff` identifies a "cutoff value", for a `page` in the `set`. It is not a record nor a reference to it. +- Its value is derived from the `keyset attributes values` array of the last record of the `page`, converted to JSON and encoded + as a Base64 URL-safe string for easy use in URLs. + - `Pagy::Keyset` embeds it in the request URL; `Pagy::KeysetForUI` caches it on the server. +- All the `page`s but the last, end with the `cutoff`. +- All the `page`s but the first, begin AFTER the `cutoff` of the previous `page`. ## ORMs @@ -248,8 +264,9 @@ Default `nil`. ==- `:jsonify_keyset_attributes` -A lambda to override the generic json encoding of the `keyset` attributes. It receives the keyset attributes to jsonify, and it should return a JSON string of the `attributes.values` array. Use it when the generic `to_json` method would lose -some information when decoded. +A lambda to override the generic JSON encoding of the `keyset attributes`. It receives the `keyset attributes` as an arument, and +it should return a JSON string of the `attributes.values` array. Use it when the generic `to_json` method would lose some +information when decoded. For example: `Time` objects may lose or round the fractional seconds through the encoding/decoding cycle, causing the ordering to fail and thus creating all sort of unexpected behaviors (e.g. skipping or repeating the same page, missing or duplicated records, @@ -260,7 +277,7 @@ etc.). Here is what you can do: jsonify_keyset_attributes = lambda do |attributes| # Convert it to a string matching the stored value/format in SQLite DB attributes[:created_at] = attributes[:created_at].strftime('%F %T.%6N') - attributes.values.to_json # remember to return an array of the values only + attributes.values.to_json # remember to return the array of values, not the attribute hash end Pagy::Keyset(set, jsonify_keyset_attributes:) diff --git a/docs/api/keyset_for_ui.md b/docs/api/keyset_for_ui.md index 1dc99661f..82afc169d 100644 --- a/docs/api/keyset_for_ui.md +++ b/docs/api/keyset_for_ui.md @@ -81,12 +81,12 @@ Querying with the LIMIT again, might cause records to get skipped or to appear t While te accuracy is guaranteed, in case of insertions or deletions of records falling in the range of the visited page, the page will obviously have a number of records different from expected. -That might not be a problem in most cases, however in extreme cases, a complete page of records might get wiped out, resulting in a completely empty (or with just very few records) page. That's not a logical problem, nor a common one, but it may look weird to the users. +That is not a logical nor common problem, however in extreme cases, a page of records might change its size so noticeably and unexpectedly that it may look somehow "broken" to the users. -!!!success We are planning to fix the problem in the future by: +!!!success We plan to implement page-rebalancing: -- Adding automatic compacting of empty (or almost empty) visited pages. -- Adding automatic splitting of eccesively grown visited pages. +- Automatic compacting of empty (or almost empty) visited pages. +- Automatic splitting of eccesively grown visited pages. !!! ## Setup @@ -102,7 +102,7 @@ internally: Pagy::KeysetForUI.new(active_record_set) #=> # -Pagy::Keyset.new(sequel_set) +Pagy::KeysetForUI.new(sequel_set) #=> # ``` @@ -134,7 +134,7 @@ Paginate only `:max_pages` ignoring the rest. ==- `:reset_overflow` -Resets the pagination in case of overflow, instead of raising a `Pagy::OverflowError`. Use it when you don't need to `rescue` and handle the event in any particular way. Notice: it keeps the current `cache_key` +Resets the pagination in case of overflow, instead of raising a `Pagy::OverflowError`. Use it when you don't need to `rescue` and handle the event in any particular way. Notice: it reuses the current `cache_key` === diff --git a/docs/extras/keyset_for_ui.md b/docs/extras/keyset_for_ui.md index 6d024e63f..e4b87c325 100644 --- a/docs/extras/keyset_for_ui.md +++ b/docs/extras/keyset_for_ui.md @@ -53,7 +53,7 @@ def pagy_cache_new_key = my_custom_cache.generate_key ## Understanding the cache This extra uses the `session` object as the cache for the `cutoffs` (not for the records!) by default, because it's simple and -works in any app, at least for prototyping. +works out of the box in any app, at least for prototyping. Notice that the `cutoffs` array can potentially grow big if you don't use `:max_pages`, especially if your `keyset` contains multiple ordered columns and more if their size is big. You must be aware of it. @@ -99,7 +99,7 @@ The key used to locate the `cutoffs` in the cache storage. ==- `:cache_key_param` -The name of the cache key param. It is `:cache_key` by default. Pass a different symbol to change it. +The name of the cache key param. It is `:cache_key` by default. Pass a different symbol to change/shorten it. ===