ವಚನ ಸಂಚಯ (Vachana Sanchaya)

A free, open-source digital concordance and research platform for Kannada Vachana literature. The project indexes ~25,000+ vachanas (devotional poems) from ~250+ vachanakaaras (poet-saints) of the 12th-century Lingayata tradition, providing full-text search, keyword concordance, alphabetical browsing, glossary, AI-assisted research, citation tools, and text-to-speech in Kannada.

Built for scholars, students, and enthusiasts of Kannada literature.

License

Copyright © 2026 Vachana Sanchaya

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see www.gnu.org/licenses/.

Technical Stack

Ruby 2.3.7 Rails 3.2.22 MySQL Sunspot Solr (full-text search) Memcached (caching) Bootstrap 2.x (front-end)

Getting Started

Solr Setup

Solr runs as a systemd service (sunspot-solr) on port 8983.

To restart and reindex:

sudo systemctl stop sunspot-solr
rm -rvf /home/vachana/rails_apps/concord/releases/vachana/solr
sudo systemctl start sunspot-solr
cd /home/vachana/rails_apps/concord/current
rvm use 2.3.7 do bundle exec rake sunspot:solr:reindex

Java 11 compat: /home/vachana/rails_apps/concord/shared/solr.in.sh overrides GC tuning and adds --add-exports JVM flags for Lucene module access.

Features

• Search — keyword search with exact/like modes, filtered by vachanakaara

• **Search autocomplete** — typeahead suggestions from key_words table

• **Full-text search** — Solr-powered full-text search across all vachanas

• **Alphabetical concordance** — browse vachanas by starting Kannada letter

• **Vachanakaara browser** — 250+ poets with bios, alphabetically sorted

• **Keyword concordance** — every word linked to the vachanas and poets that use it

• Glossary — Kannada word meanings used in vachana literature

• **Related vachanas** — algorithmic discovery via shared keyword concordance

• Text-to-Speech — Kannada TTS using Web Speech API (Chrome/Edge/Safari)

• Bookmarks — save and organize vachanas into categories (localStorage, no login)

• **Research citations** — MLA, APA, Chicago, and BibTeX export per vachana

• **BibTeX export** — download search results as .bib files for Zotero/Mendeley

• **AI research assistant** — query vachana literature with AI (multiple free providers)

• Feedback — report incorrect information (no login required)

• **Internet Archive integration** — search Vachana books from IA

• Bilingual — Kannada primary, English alternate via locale toggle

• API — JSON API v1 for external consumption

Performance Optimizations

The following performance improvements have been applied:

Home Page (2026-07-01)

• **ORDER BY RAND() eliminated** — Home page no longer executes a slow ‘ORDER BY RAND()` query across 25K+ vachanas. Instead, Ruby’s Random seeded with today’s date picks a deterministic vachana via ‘pluck(:id)` + primary-key lookup (non-cached first request only; subsequent requests read the cached DailyVachana record).

• **N+1 on vachana.vachanakaara eliminated** — DailyVachana is loaded with ‘includes(vachana: :vachanakaara)`, removing the lazy-load query.

• **Stats queries eliminated** — 5 COUNT queries (male poets, female poets, total poets, total vachanas, total keywords) are now read from tmp/site_stats.json instead of hitting the database on every request.

  To regenerate the stats file when data changes:

  rake site:stats
  

Search Results (2026-07-01)

• **N+1 on vachanakaara.name** — Search results now eager-load vachanakaara via ‘.includes(:vachanakaara)` on the paginated relation.

• **Related vachanas removed from search results modals** — The related_vachanas_inline partial was rendered inside every hidden modal (15 per page), each executing its own query. Removed from list views; still rendered on the individual vachana show page.

• **Default scope removed from Vachanakaara** — ‘default_scope order(’name’)‘ was removed. All necessary `.order(:name)` calls are now explicit.

• **Unnecessary eager loading eliminated** — WordList.all and Vachanakaara.all are no longer loaded on every search if no search term is present.

Database (2026-07-01)

• **vachana_first_letter column** — New ‘vachanas.vachana_first_letter VARCHAR(1)` column with an index, populated via `LEFT(TRIM(vachana), 1)`. Replaces slow `GROUP BY LEFT(vachana, 1)` queries for letter-counting with indexed column lookups. Auto-populated via before_save callback on the Vachana model.

• **FULLTEXT index** — MySQL FULLTEXT index added on vachanas.vachana to enable ‘MATCH … AGAINST` queries as a future replacement for `LIKE ’%…%‘` scans.

Bot Protection (July 2026)

Aggressive bot blocking at two layers:

• nginx: 20+ known aggressive crawlers (SemrushBot, GPTBot, Amazonbot, AhrefsBot, MJ12bot, etc.) blocked with HTTP 403 before reaching Rails.

• Rack::Attack: Same blocklist at app layer + per-endpoint rate limits (search: 20 req/min, autocomplete: 10 req/min per IP).

• robots.txt: GPTBot/ChatGPT-User explicitly disallowed; all bots blocked from search, autocomplete, and AI pages.

Glossary Lazy Load (July 2026)

Fixed infinite-scroll “Load More” on /glossaries — missing format.js support and JS.ERB template. Added controller respond_to, index.js.erb, and reusable _glossary_words partial.

Solr Full-Text Search (July 2026)

Re-enabled Sunspot Solr (disabled during prior maintenance). Added systemd unit for auto-start, Java 11 compatibility override, and reindexed 20,929 records.

HTTP Caching (2026-07-01)

• fresh_when (304 Not Modified) added to VachanasController#show and VachanakaarasController#show.