Skip to content

Add native support for UNF and NOINDEX field attributes in RedisVL schema #374

@jeremyplichta

Description

@jeremyplichta

Add native support for UNF and NOINDEX field attributes in RedisVL schema

Summary

RedisVL currently lacks native support for the UNF (Un-normalized Form) and NOINDEX field attributes that are available in RediSearch. These attributes are important for optimizing search performance and storage:

  • UNF: Stores the original form of the text without normalization, useful for exact matching scenarios
  • NOINDEX: Excludes a field from being indexed while still allowing it to be stored and retrieved

Current Problem

When using RedisVL with indexes that contain fields with UNF or NOINDEX attributes, the library's parse_attrs function fails to properly parse these standalone flags, leading to:

  • Parsing errors when introspecting existing indexes
  • Inability to create schemas with these attributes
  • Need for monkey patching to work around the limitation

Proposed Solution

Add native support for UNF and NOINDEX attributes in RedisVL by:

  1. Schema Definition: Allow these attributes to be specified in field definitions
  2. Index Creation: Include these attributes when generating FT.CREATE commands
  3. Index Introspection: Properly parse these standalone flags when reading index information via FT.INFO

Implementation Details

The main changes needed are:

  1. Field Schema: Update field classes to accept unf and noindex boolean parameters
  2. Command Generation: Include these flags in the Redis command generation
  3. Parsing Logic: Update parse_attrs to handle standalone flags (not just key-value pairs)

Example Usage

from redisvl.schema import IndexSchema, TextField, NumericField

# Define schema with UNF and NOINDEX attributes
schema = IndexSchema.from_dict({
    "index": {"name": "my_index", "prefix": "doc:"},
    "fields": [
        {"name": "title", "type": "text", "unf": True},
        {"name": "content", "type": "text", "noindex": True},
        {"name": "score", "type": "numeric", "sortable": True}
    ]
})

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions