Merge branch 'main' into collection-limit

_includes/release-history.md

@@ -2,7 +2,8 @@ This table lists the Weaviate core versions and corresponding client library ver
 
 | Weaviate <br/><small>([GitHub][cWeaviate])</small> | First <br/>release date | Python <br/><small>([GitHub][cPython])</small> | TypeScript/ <br/>JavaScript <br/><small>([GitHub][cTypeScript])</small> | Go <br/><small>([GitHub][cGo])</small> | Java <br/><small>([GitHub][cJava])</small> |
 | :- | :- | :-: | :-: | :-: | :-: |
-| 1.28.x | 2024-12-11 | 4.10.x | TBC | TBC | TBC |
+| 1.29.x | 2025-02-17 | 4.11.x | 3.4.x | TBC | TBC |
+| 1.28.x | 2024-12-11 | 4.10.x | 3.3.x | 4.16.x | 5.0.x |
 | 1.27.x | 2024-10-16 | 4.9.x | 3.2.x | 4.16.x | 5.0.x<br/>4.9.x |
 | 1.26.x | 2024-07-22 | 4.7.x | 3.1.x | 4.15.x | 4.8.x |
 | 1.25.x | 2024-05-10 | 4.6.x | 2.1.x | 4.13.x | 4.6.x |

developers/weaviate/api/graphql/additional-operators.md

@@ -130,9 +130,6 @@ Pagination is not a cursor-based implementation. This has the following implicat
 
 ## Autocut
 
-:::info Added in `v1.20`
-:::
-
 The autocut function limits results based on discontinuities in the result set. Specifically, autocut looks for discontinuities, or jumps, in result metrics such as vector distance or search score.
 
 To use autocut, specify how many jumps there should be in your query. The query stops returning results after the specified number of jumps.
@@ -283,15 +280,11 @@ import GraphQLFiltersAfter from '/_includes/code/graphql.filters.after.mdx';
 
 ## Sorting
 
-:::info
-Added in `v1.13.0`.
-:::
-
 You can sort results by any primitive property, such as `text`, `number`, or `int`. 
 
 ### Sorting considerations
 
-Sorting can be applied when fetching objects, but it's unavailable for the `near<Media>` vector search operators. 
+Sorting can be applied when fetching objects, but it's **unavailable when using search operators**. 
 
 Weaviate's sorting implementation does not lead to massive memory spikes. Weaviate does not load all object properties into memory; only the property values being sorted are kept in memory.
 

developers/weaviate/client-libraries/python/index.md

@@ -280,6 +280,10 @@ To create an older, `v3` style `Client` object, use the `weaviate.Client` class.
 
 To create a `v3` style client, refer to the [`v3` client documentation](./python_v3.md).
 
+### Using Custom SSL Certificates
+
+The Python client doesn't directly support passing SSL certificates. If you need to work with self-signed certificates (e.g. for enterprise environments), you can add the certificate to the underlying libraries such as `certifi` that the Weaviate client library uses.
+
 ## Initial connection checks
 
 When establishing a connection to the Weaviate server, the client performs a series of checks. These includes checks for the server version, and to make sure that the REST and gRPC ports are available.

developers/weaviate/concepts/replication-architecture/consistency.md

@@ -51,6 +51,21 @@ A clean (without fails) execution has two phases:
 
 </details>
 
+### Collection definition requests in queries
+
+:::info Added in `v1.27.10`, `v1.28.4`
+:::
+
+Some queries require the collection definition. Prior to the introduction of this feature, every such query led to the local (requesting) node to fetch the collection definition from the leader node. This meant that the definition was strongly consistent, but it could lead to additional traffic and load.
+
+Where available, the `COLLECTION_RETRIEVAL_STRATEGY` [environment variable](../../config-refs/env-vars.md#multi-node-instances) can be set to `LeaderOnly`, `LocalOnly`, or `LeaderOnMismatch`.
+
+- `LeaderOnly` (default): Always requests the definition from the leader node. This is the most consistent behavior but can lead to higher intra-cluster traffic.
+- `LocalOnly`: Always use the local definition; leading to eventually consistent behavior while reducing intra-cluster traffic.
+- `LeaderOnMismatch`: Checks if the local definition is outdated, and requests the definition if necessary. Balances consistency and intra-cluster traffic.
+
+The default behavior is `LeaderOnly` to achieve strong consistency. However, `LocalOnly` and `LeaderOnMismatch` can be used to reduce intra-cluster traffic according to the desired consistency level.
+
 ## Data objects
 
 Weaviate uses two-phase commits for objects, adjusted for the consistency level. For example for a `QUORUM` write (see below), if there are 5 nodes, 3 requests will be sent out, each of them using a 2-phase commit under the hood.

developers/weaviate/config-refs/env-vars.md

@@ -120,24 +120,12 @@ For more information on authentication and authorization, see the [Authenticatio
 
 | Variable | Description | Type | Example Value |
 | --- | --- | --- | --- |
-| `ASYNC_REPLICATION_DISABLED` | Disable async replication. Default: `false` | `boolean` | `false` |
-| `ASYNC_REPLICATION_HASHTREE_HEIGHT` | Height of the hash tree used for data comparison between nodes. If the height is `0` each node will store just one digest per shard. Default: `16`, Min: `0`, Max: `20`<br/> [Read more about potentially increased memory consumption.](/developers/weaviate/concepts/replication-architecture/consistency#memory-and-performance-considerations-for-async-replication) | `string - number` | `10` |
-| `ASYNC_REPLICATION_FREQUENCY` |  Frequency of periodic data comparison between nodes in seconds. Default: `30` | `string - number` | `60` |
-| `ASYNC_REPLICATION_FREQUENCY_WHILE_PROPAGATING` | Frequency of data comparison between nodes after a node has been synced in milliseconds. Default: `10` | `string - number` | `20` |
-| `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` | Frequency of how often the background process checks for changes in the availability of nodes in seconds. Default: `5` | `string - number` | `20` |
-| `ASYNC_REPLICATION_LOGGING_FREQUENCY` | Frequency of how often the background process logs any events in seconds. Default: `5` | `string - number` | `7` |
-| `ASYNC_REPLICATION_DIFF_BATCH_SIZE` | Specifies the batch size for comparing digest information between nodes. Default: `1000`, Min: `1`, Max: `10000` |`string - number`  | `2000` |
-| `ASYNC_REPLICATION_DIFF_PER_NODE_TIMEOUT` | Defines the time limit a node has to provide a comparison response in seconds. Default: `10` | `string - number` | `30` |
-| `ASYNC_REPLICATION_PROPAGATION_TIMEOUT` | Defines the time limit a node has to provide a propagation response in seconds. Default: `30` | `string - number` | `60` |
-| `ASYNC_REPLICATION_PROPAGATION_LIMIT` | Limits the number of out-of-sync objects that can be propagated in one asynchronous replication iteration. Default: `10000`, Min: `1`, Max: `1000000` | `string - number` | `5000` |
-| `ASYNC_REPLICATION_PROPAGATION_DELAY` | Sets a delay period to allow asynchronous write operations to reach all nodes in a shard/tenant before propagating new or updated objects. Default: `30` | `string - number` | `40` |
-| `ASYNC_REPLICATION_PROPAGATION_CONCURRENCY` | Defines the number of workers which will concurrently propagate a batch of objects. Default: `5`, Min: `1`, Max: `20` | `string - number` | `10` |
-| `ASYNC_REPLICATION_PROPAGATION_BATCH_SIZE` | Sets the maximum number of objects to propagate in a single batch. Default: `100`, Min: `1`, Max: `1000` |`string - number`  | `200` |
 | `CLUSTER_DATA_BIND_PORT` | Port for exchanging data. | `string - number` | `7103` |
 | `CLUSTER_GOSSIP_BIND_PORT` | Port for exchanging network state information. | `string - number` | `7102` |
 | `CLUSTER_HOSTNAME` | Hostname of a node. Always set this value if the default OS hostname might change over time. | `string` | `node1` |
 | `CLUSTER_JOIN` | The service name of the "founding" member node in a cluster setup | `string` | `weaviate-node-1:7100` |
 | `HNSW_STARTUP_WAIT_FOR_VECTOR_CACHE` | If `true`, vector cache prefill is synchronous when a node starts. The node reports ready to serve when the cache is hot. Defaults to `false`. Added in 1.24.20 and 1.25.5. | `boolean` | `false` |
+| `COLLECTION_RETRIEVAL_STRATEGY`| Set collection definition retrieval behavior for a data request. <br/><br/> <ul><li>`LeaderOnly` (default): Always requests the definition from the leader node. </li><li>`LocalOnly`: Always use the local definition</li><li>`LeaderOnMismatch`: Requests the definition if outdated.</li></ul> ([Read more](../concepts/replication-architecture/consistency.md#collection-definition-requests-in-queries)) (Added in `v1.27.10`, `v1.28.4`) | `string` | `LeaderOnly` |
 | `RAFT_ENABLE_FQDN_RESOLVER` | If `true`, use DNS lookup instead of memberlist lookup for Raft. Added in `v1.25.15`. ([Read more](../concepts/cluster.md#fqdn-for-node-discovery)) | `boolean` | `true` |
 | `RAFT_FQDN_RESOLVER_TLD` | The top-level domain to use for DNS lookup, in `[node-id].[tld]` format. Added in `v1.25.15`. ([Read more](../concepts/cluster.md#fqdn-for-node-discovery)) | `string` | `example.com` |
 | `RAFT_BOOTSTRAP_EXPECT` | The number of voter notes at bootstrapping time | `string - number` | `1` |
@@ -147,6 +135,29 @@ For more information on authentication and authorization, see the [Authenticatio
 | `RAFT_METADATA_ONLY_VOTERS` | If `true`, voter nodes only handle the schema. They do not accept any data. | `boolean` | `false` |
 | `REPLICATION_MINIMUM_FACTOR` | The minimum replication factor for all collections in the cluster. | `string - number` | `3` |
 
+### Async replication
+
+:::info Added in `v1.29`
+The environment variables for configuring async replication have been introduced in `v1.29`.  
+To learn more about their usage, visit the **[replication how-to guide](/developers/weaviate/configuration/replication#async-replication-settings)**.
+:::
+
+| Variable | Description | Type | Example Value |
+| --- | --- | --- | --- |
+| `ASYNC_REPLICATION_DISABLED` | Disable async replication. Default: `false` | `boolean` | `false` |
+| `ASYNC_REPLICATION_HASHTREE_HEIGHT` | Height of the hash tree used for data comparison between nodes. If the height is `0` each node will store just one digest per shard. Default: `16`, Min: `0`, Max: `20`<br/> [Read more about potentially increased memory consumption.](/developers/weaviate/concepts/replication-architecture/consistency#memory-and-performance-considerations-for-async-replication) | `string - number` | `10` |
+| `ASYNC_REPLICATION_FREQUENCY` |  Frequency of periodic data comparison between nodes in seconds. Default: `30` | `string - number` | `60` |
+| `ASYNC_REPLICATION_FREQUENCY_WHILE_PROPAGATING` | Frequency of data comparison between nodes after a node has been synced in milliseconds. Default: `10` | `string - number` | `20` |
+| `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` | Frequency of how often the background process checks for changes in the availability of nodes in seconds. Default: `5` | `string - number` | `20` |
+| `ASYNC_REPLICATION_LOGGING_FREQUENCY` | Frequency of how often the background process logs any events in seconds. Default: `5` | `string - number` | `7` |
+| `ASYNC_REPLICATION_DIFF_BATCH_SIZE` | Specifies the batch size for comparing digest information between nodes. Default: `1000`, Min: `1`, Max: `10000` |`string - number`  | `2000` |
+| `ASYNC_REPLICATION_DIFF_PER_NODE_TIMEOUT` | Defines the time limit a node has to provide a comparison response in seconds. Default: `10` | `string - number` | `30` |
+| `ASYNC_REPLICATION_PROPAGATION_TIMEOUT` | Defines the time limit a node has to provide a propagation response in seconds. Default: `30` | `string - number` | `60` |
+| `ASYNC_REPLICATION_PROPAGATION_LIMIT` | Limits the number of out-of-sync objects that can be propagated in one asynchronous replication iteration. Default: `10000`, Min: `1`, Max: `1000000` | `string - number` | `5000` |
+| `ASYNC_REPLICATION_PROPAGATION_DELAY` | Sets a delay period to allow asynchronous write operations to reach all nodes in a shard/tenant before propagating new or updated objects. Default: `30` | `string - number` | `40` |
+| `ASYNC_REPLICATION_PROPAGATION_CONCURRENCY` | Defines the number of workers which will concurrently propagate a batch of objects. Default: `5`, Min: `1`, Max: `20` | `string - number` | `10` |
+| `ASYNC_REPLICATION_PROPAGATION_BATCH_SIZE` | Sets the maximum number of objects to propagate in a single batch. Default: `100`, Min: `1`, Max: `1000` |`string - number`  | `200` |
+
 <!-- Docs notes:
 Undocumented environment variables - for internal use only:
 MAINTENANCE_NODES

developers/weaviate/configuration/replication.md

@@ -55,12 +55,12 @@ import ReplicationConfigWithAsyncRepair from '/\_includes/code/configuration/rep
 ### Configure async replication settings {#async-replication-settings}
 
 :::info Added in `v1.29`
-The [environment variables](/developers/weaviate/config-refs/env-vars#multi-node-instances) for configuring async replication (`ASYNC_*`) have been introduced in `v1.29`.
+The [environment variables](/developers/weaviate/config-refs/env-vars#async-replication) for configuring async replication (`ASYNC_*`) have been introduced in `v1.29`.
 :::
 
 Async replication helps achieve consistency for data replicated across multiple nodes.
 
-Update the following [environment variables](/developers/weaviate/config-refs/env-vars#multi-node-instances) to configure async replication for your particular use case.
+Update the following [environment variables](/developers/weaviate/config-refs/env-vars#async-replication) to configure async replication for your particular use case.
 
 #### Logging
 

developers/weaviate/model-providers/_includes/provider.generative.py

@@ -462,7 +462,7 @@
     "DemoCollection",
     # highlight-start
     generative_config=Configure.Generative.nvidia(
-        api_endpoint="https://integrate.api.nvidia.com/v1",
+        base_url="https://integrate.api.nvidia.com/v1",
         model="meta/llama-3.3-70b-instruct",
         temperature=0.7,
         max_tokens=1024

developers/weaviate/model-providers/_includes/provider.reranker.py

@@ -114,7 +114,8 @@
     "DemoCollection",
     # highlight-start
     reranker_config=Configure.Reranker.nvidia(
-        model="nvidia/llama-3.2-nv-rerankqa-1b-v2"
+        model="nvidia/llama-3.2-nv-rerankqa-1b-v2",
+        base_url="https://integrate.api.nvidia.com/v1",
     )
     # highlight-end
     # Additional parameters not shown

developers/weaviate/model-providers/_includes/provider.vectorizer.py

@@ -743,7 +743,8 @@
         Configure.NamedVectors.text2vec_nvidia(
             name="title_vector",
             source_properties=["title"],
-            model="nvidia/nv-embed-v1"
+            model="nvidia/nv-embed-v1",
+            base_url="https://integrate.api.nvidia.com/v1",
         )
     ],
     # highlight-end

developers/weaviate/model-providers/nvidia/index.md

@@ -36,13 +36,16 @@ Generative AI models on NVIDIA can generate human-like text based on given promp
 
 <!-- ### Reranker models
 
+
 ![Reranker integration illustration](../_includes/integration_nvidia_reranker.png)
 
-NVIDIA's reranker models are designed to improve the relevance and ranking of search results.
+NVIDIA's reranker models are designed to improve the relevance and ranking of search results. -->
+
+<!-- TODO - Add link back to reranker.md; removed because the link checker still sees it as a broken link -->
 
-[The Weaviate reranker integration](./reranker.md) allows users to easily refine their search results by leveraging NVIDIA's reranker models.
+<!-- [The Weaviate reranker integration](./) allows users to easily refine their search results by leveraging NVIDIA's reranker models. -->
 
-[NVIDIA reranker integration page](./reranker.md) -->
+<!-- [NVIDIA reranker integration page](./) -->
 
 ## Summary
 
@@ -59,7 +62,8 @@ Then, go to the relevant integration page to learn how to configure Weaviate wit
 - [Text Embeddings](./embeddings.md)
 - [Multimodal Embeddings](./embeddings-multimodal.md)
 - [Generative AI](./generative.md)
-- [Reranker](./reranker.md)
+<!-- TODO - Add link back to reranker.md; removed because the link checker still sees it as a broken link -->
+<!-- - [Reranker](./) -->
 
 ## Questions and feedback