-
Notifications
You must be signed in to change notification settings - Fork 57
Open
Description
Add native support for UNF and NOINDEX field attributes in RedisVL schema
Summary
RedisVL currently lacks native support for the UNF
(Un-normalized Form) and NOINDEX
field attributes that are available in RediSearch. These attributes are important for optimizing search performance and storage:
- UNF: Stores the original form of the text without normalization, useful for exact matching scenarios
- NOINDEX: Excludes a field from being indexed while still allowing it to be stored and retrieved
Current Problem
When using RedisVL with indexes that contain fields with UNF
or NOINDEX
attributes, the library's parse_attrs
function fails to properly parse these standalone flags, leading to:
- Parsing errors when introspecting existing indexes
- Inability to create schemas with these attributes
- Need for monkey patching to work around the limitation
Proposed Solution
Add native support for UNF
and NOINDEX
attributes in RedisVL by:
- Schema Definition: Allow these attributes to be specified in field definitions
- Index Creation: Include these attributes when generating
FT.CREATE
commands - Index Introspection: Properly parse these standalone flags when reading index information via
FT.INFO
Implementation Details
The main changes needed are:
- Field Schema: Update field classes to accept
unf
andnoindex
boolean parameters - Command Generation: Include these flags in the Redis command generation
- Parsing Logic: Update
parse_attrs
to handle standalone flags (not just key-value pairs)
Example Usage
from redisvl.schema import IndexSchema, TextField, NumericField
# Define schema with UNF and NOINDEX attributes
schema = IndexSchema.from_dict({
"index": {"name": "my_index", "prefix": "doc:"},
"fields": [
{"name": "title", "type": "text", "unf": True},
{"name": "content", "type": "text", "noindex": True},
{"name": "score", "type": "numeric", "sortable": True}
]
})
Metadata
Metadata
Assignees
Labels
No labels