refactor: composite retriever and make reranker optional

[a-klos]

This pull request introduces several improvements and updates across the infrastructure and core backend logic, focusing on standardizing configuration, enhancing retriever and reranker functionality, and modernizing cluster setup. The most significant changes include refactoring the retriever/reranker pipeline for better performance and configurability, switching to the official Kubernetes ingress-nginx controller, updating configuration files for clarity and consistency, and making the cluster setup script more robust and idempotent.

Retriever and Reranker Enhancements:

• Refactored CompositeRetriever to execute retriever calls concurrently using asyncio.gather, reducing latency, and improved duplicate filtering to O(1) complexity. Added configuration for global document caps and reranker toggling. [1]R47, [2] [3] [4]
• Updated dependency injection in dependency_container.py to pass new reranker parameters (enabled, score_threshold, total_k_documents, k_documents) for finer control. [1] [2]
• Added logic to treat retrievals returning only summaries as "no documents found" for improved error handling in chat graph.

Infrastructure and Helm Chart Updates:

• Migrated from the Bitnami nginx-ingress-controller to the official ingress-nginx Helm chart with pinned versions for both the chart and controller image, ensuring alignment between scripts and Helm dependencies. [1] [2]
• Updated documentation and values files to reflect the new ingress controller and clarified instructions for certificate issuer email and Windows hosts file editing. [1] [2]

Configuration Consistency and Clarity:

• Replaced legacy retriever and reranker configuration variables with canonical names (e.g., RETRIEVER_TOTAL_K_DOCUMENTS, RERANKER_ENABLED) in both values.yaml and README.md for clarity and future-proofing. [1] [2] [3]

Cluster Setup Script Improvements:

• Overhauled setup-k3d-cluster.sh to be idempotent, check for existing clusters, fail fast on errors, and install/upgrade the official ingress-nginx chart with robust status checks.

Build and Sync Ignore List Refactoring:

• Centralized and standardized Docker build ignore patterns in the Tiltfile to reduce duplication and improve maintainability. [1] [2] [3] [4] [5] [6] [7]