The platform for modern questions

Ask anythingin English

Record a question or type one and press Enter.
<200msretrieval pipelineLOCAL
6languages indexedMSMARCO-XI
CRAGgrades before generateABSTAIN
6chunking strategiesCOMPARE
Voicespeak in, cite outSARVAM
2provider profilesLOCAL · CLOUD
<200msretrieval pipelineLOCAL
6languages indexedMSMARCO-XI
CRAGgrades before generateABSTAIN
6chunking strategiesCOMPARE
Voicespeak in, cite outSARVAM
2provider profilesLOCAL · CLOUD
Capabilities

Query to Context
retrieved - not invented.

01

Streamed, cited answers

Sentences land as they generate. Each one carries a citation marker that must match a retrieved chunk - rejected otherwise, never silent.

02

Hybrid retrieve + CRAG

Dense and sparse legs meet RRF, then a cross-encoder reranks. CRAG grades the top hits and strips or rewrites before the model ever speaks.

03

Voice in, grounded out

Mic capture becomes 16 kHz WAV, Sarvam returns a transcript, and the same query path answers - waveform on the old console, globe pulse here.

AB
04

Guardrails before generate

Off-topic, injection, unsafe input, and language gates run first. Weak evidence abstains. API keys stay on the server behind the proxy.

Process

Three steps.
One grounded pipeline.

ingress.py
1POST /v1/voice/query/stream
2# multipart: question.wav
3 
4# or text
5POST /v1/query/stream
6{ "query": "…", "strategy": "sentence" }
7 
8# guardrails first → allow | refuse
Ready
Latency

Sub-200ms
retrieval.

Guardrails through CRAG are measured as one pipeline. The local profile keeps that under 200ms. Speech-to-text and generation are timed separately so a slow model never hides a retrieval regression.

6
Languages indexed
2
Profiles · local / cloud
<200ms
Retrieval P95 target
Retrieval pipelineLocal profile
Guardrails
Off-topic · injection · language
~2ms
Embedding
ONNX dense query vector
~18ms
Cache
Exact + optional semantic
~3ms
Retrieval
Hybrid dense + sparse
~25ms
Rerank
Cross-encoder top-k
~90ms
CRAG
Correct · strip · rewrite
~5ms
Benchmarks

Numbers the
pipeline owns.

Measured|-
<0ms
Retrieval P95 target · local profile
0
Languages indexed · MSMARCO-XI
0
Chunking strategies to compare
0
Provider profiles · local / cloud
Providers

Swappable free-tier
stack.

Every dependency sits behind a port. Local uses ONNX + Compose; cloud swaps in Jina, Groq, Sarvam, Qdrant Cloud, Neon, and Redis Cloud without rewriting the pipeline.

PostgreSQL
Registry · Neon / Compose
Qdrant
Vector store
Jina
Embeddings + rerank
Groq
LLM generation
Sarvam
Speech-to-text
Redis
Exact + semantic cache
Langfuse
Tracing
Ollama
Local LLM
FastEmbed
Local ONNX embed / rerank
OpenRouter
LLM fallback
Neon
Hosted Postgres
Render
API hosting
Vercel
Web hosting
ElevenLabs
Optional STT
PostgreSQL
Registry · Neon / Compose
Qdrant
Vector store
Jina
Embeddings + rerank
Groq
LLM generation
Sarvam
Speech-to-text
Redis
Exact + semantic cache
Langfuse
Tracing
Ollama
Local LLM
FastEmbed
Local ONNX embed / rerank
OpenRouter
LLM fallback
Neon
Hosted Postgres
Render
API hosting
Vercel
Web hosting
ElevenLabs
Optional STT
ElevenLabs
Optional STT
Vercel
Web hosting
Render
API hosting
Neon
Hosted Postgres
OpenRouter
LLM fallback
FastEmbed
Local ONNX embed / rerank
Ollama
Local LLM
Langfuse
Tracing
Redis
Exact + semantic cache
Sarvam
Speech-to-text
Groq
LLM generation
Jina
Embeddings + rerank
Qdrant
Vector store
PostgreSQL
Registry · Neon / Compose
ElevenLabs
Optional STT
Vercel
Web hosting
Render
API hosting
Neon
Hosted Postgres
OpenRouter
LLM fallback
FastEmbed
Local ONNX embed / rerank
Ollama
Local LLM
Langfuse
Tracing
Redis
Exact + semantic cache
Sarvam
Speech-to-text
Groq
LLM generation
Jina
Embeddings + rerank
Qdrant
Vector store
PostgreSQL
Registry · Neon / Compose
Guardrails

Grounded or
silent.

FastRAG does not invent facts. Answers are generated only from retrieved sources, checked by guardrails and CRAG, and refused when the corpus cannot support them.

API keysGuardrailsCitationsCRAGAbstain

Bearer query keys

Shared secrets stay server-side. The UI proxies through /api/rag so tokens never reach the browser.

Input guardrails

Off-topic centroid check, prompt-injection patterns, unsafe-input filter, and a language gate before retrieval.

Citation-bound answers

Every sentence must cite a retrieved chunk. Markers that do not match the context are rejected, not shown.

Abstain over guess

CRAG grades retrieval before generation. Weak or empty evidence returns no_answer instead of a hallucinated reply.

the developers

Built with love,
for the community.

Three people shipping a voice-enabled, multilingual RAG stack you can run on free tiers - and fork without asking.

Open source

The whole pipeline lives in one public repo.

Cited or silent

Every answer is grounded in sources, or the pipeline abstains.

Local + cloud

Same ports, free-tier providers when you need them.

Ship with us

Issues and PRs welcome on GitHub.

Rohan Sen

Builder

Rohan Sen

GitHubLinkedIn
View on GitHub
What we achieved
01 / 04

“Indexed MSMARCO-XI across Hindi, Bengali, Tamil, Telugu, Marathi, and English - one collection, six languages, comparable side by side.”

01

Multilingual corpus

ai4bharat / MSMARCO-XI

Milestone

Six languages live

Built on this stack

MSMARCO-XICRAGHybrid RRFSarvam STTJina v3QdrantGroqNeonLangfuseSix strategies
MSMARCO-XICRAGHybrid RRFSarvam STTJina v3QdrantGroqNeonLangfuseSix strategies
Hosting

Three ways to
run it

Local for the latency claim, free cloud for the live multilingual demo, and a quality profile when you can spend more tokens and disk.

01

Local

Compose + ONNX on your machine - the sub-200ms retrieval rig

In-processNo cloud hops on retrieve
  • FASTRAG_PROFILE=local
  • Embeddings: BAAI/bge-base-en-v1.5 (ONNX)
  • Rerank: Xenova/ms-marco-MiniLM-L-6-v2
  • Qdrant + Postgres + Redis via Compose
  • LLM: Ollama (OpenAI-compatible)
  • Semantic cache off (plain Redis)
  • STT: Sarvam key optional (text-only without it)
  • Benchmark target: <200ms retrieval P95
See local setup
Current
02

Hosted cloud

What we run on free tiers today - multilingual demo

Free tiersRender + Vercel ready
  • FASTRAG_PROFILE=cloud
  • Embeddings + rerank: Jina (v3 / v2-multilingual)
  • Vectors: Qdrant Cloud (1 GB free)
  • Registry: Neon Postgres
  • Cache: Redis Cloud (+ RediSearch)
  • LLM: Groq free · OpenRouter fallback
  • STT: Sarvam Saaras v3
  • Tracing: Langfuse Cloud
  • Sparse retrieval off (512 MB RAM)
See cloud env
03

Best results

Config that maximizes quality when quota and RAM allow

Quality firstSame ports, fuller stack
  • Jina (or multilingual local) for Indic queries
  • Index all six chunking strategies
  • CRAG on · calibrated thresholds
  • Semantic cache on (Redis Stack / Cloud)
  • Hybrid dense + sparse when RAM allows
  • Stronger Groq / paid LLM + fallback
  • Full MSMARCO-XI ingest (not a tiny subset)
  • Golden gate + bench-latency both profiles
Read providers

Cloud hops will not meet the local 200ms retrieval target - that number is measured on the local profile only. Read latency.md

Give it a star
on Github.

FastRAG is open source. Pipelines, adapters, chunking strategies, CRAG, and the voice UI live in one repo - clone it, swap providers, run it local or cloud.

View on GitHub

github.com/rohansen856/FastRAG