Skip to content

feat(retrieval): improve search query preprocessing (#162) - #176

Open
prasiddhi-105 wants to merge 2 commits into
Aharshi3614:mainfrom
prasiddhi-105:feature/162-improve-search-query-preprocessing
Open

feat(retrieval): improve search query preprocessing (#162)#176
prasiddhi-105 wants to merge 2 commits into
Aharshi3614:mainfrom
prasiddhi-105:feature/162-improve-search-query-preprocessing

Conversation

@prasiddhi-105

Copy link
Copy Markdown
Contributor

Description

  • Introduced search query preprocessing to normalize raw user input before hitting the retrieval pipeline.
  • Normalized irregular whitespace (newlines, tabs, redundant spaces) and stripped unnecessary surrounding punctuation while preserving essential code symbols.
  • Maintained compatibility with existing semantic and hybrid retrieval systems.

Related Issue

Fixes #162


Type of Change

  • Bug fix
  • New feature
  • Enhancement
  • Documentation update
  • Refactoring
  • Performance improvement
  • Tests
  • Other (please specify)

Changes Made

  • Created preprocess_query() helper function in retriever.py.
  • Integrated automatic query preprocessing inside retrieve() entry point.
  • Added unit tests in tests/test_retriever.py covering whitespace normalization and symbol preservation.

Testing

  • Tested locally
  • Existing tests pass
  • Added new tests (if applicable)

Additional Notes:
Ran python -m pytest tests/test_retriever.py locally; all tests passed cleanly.


Screenshots (if applicable)

N/A - Backend preprocessing logic.


Checklist

  • My code follows the project's coding style.
  • I have linked the related issue.
  • I have tested my changes.
  • I have updated documentation where necessary.
  • My changes do not introduce new warnings or errors.
  • The project builds successfully.
  • This PR only addresses the linked issue.

@Aharshi3614

Copy link
Copy Markdown
Owner

@prasiddhi-105 The build is failing here. Can you please fix this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Search Query Preprocessing

2 participants