Knowledge workers drown in information. We collect documents at scale—research papers, PDFs, articles, code—but can’t retrieve or synthesize what we’ve gathered. Most solutions force a choice: keep data local and lose AI, or move to cloud and lose privacy. Docify dissolves this false binary through 11 specialized services orchestrated into a complete RAG pipeline.
<h2>Architecture: 11 Services, One Pipeline</h2>
Input Layer: Parsing &amp; Chunking -
Resource Ingestion handles heterogeneous formats (PDF, DOCX, XLSX, Markdown, TXT). Deduplication Service computes SHA-256 hashes on raw content—preventing re-processing when the same research paper arrives from three sources. Chunking Service uses tiktoken for accurate token counting (512 tokens, 50-token overlap) while respecting paragraph boundaries and preserving section hierarchies.
Embedding Layer: Async Vector Generation -
Embeddings block APIs. Docify uses Celery + Redis to decouple: uploads return immediately, workers process embeddings asynchronously. Embeddings Service uses all-minilm:22m (384-dim, 22MB)—aggressively lightweight compared to 768-dim models, but sentence-transformers research shows minimal quality loss. Storage in PostgreSQL pgvector with HNSW indexing enables &lt;200ms vector search across 10K documents.
Search Layer: Hybrid Retrieval -
Semantic search alone fails on exact phrases; keyword search alone fails on synonyms. Hybrid Search combines pgvector cosine distance with BM25 ranking via reciprocal rank fusion—a technique that elegantly merges different ranking philosophies. A chunk ranked #2 by vectors and #5 by keywords scores higher than one ranked #1 by vectors and #100 by keywords.
Ranking Layer: Multi-Factor Scoring -
Re-Ranking Service refines results using five factors: base relevance (40%), citation frequency (15%), recency (15%), specificity (15%), source quality (15%). This produces 5-10 final chunks sent to LLM. Notably, it flags conflicting sources—if multiple documents contradict each other, the service signals this upstream.
Context Layer: Token Budget Management -
LLMs have finite context windows. Context Assembly respects token budgets: 2000-token default split 60% primary sources (top-ranked chunks), 30% supporting context, 10% metadata. Truncates intelligently at sentence boundaries (never mid-sentence gibberish). Most questions need 2-3 high-quality sources, not 100.
Prompt Layer: Anti-Hallucination Engineering -
Prompts with strict rules: “ONLY use provided context. ALWAYS cite sources [Source 1]. If unknown, say not available. When sources conflict, present both sides.” Source markers in context enable citation verification—making it tractable to validate claims post-generation.
LLM Service: Provider Flexibility -
Provider-agnostic architecture. Ollama local Mistral 7B (4-bit quantized) is default, with OpenAI/Anthropic support. Hardware auto-detection adjusts: GPU available? Accelerate. CPU-only? Extend timeouts. Low VRAM? Switch models. Streaming enabled by default for responsive UX.
Verification Layer: Citation Grounding -
LLMs fabricate sources. Citation Verification runs post-generation: extracts <code>[Source N]</code> references, searches for cited claims in source chunks, flags mismatches. Catches egregious errors like citing sources containing no relevant information. Not foolproof, but reduces hallucination significantly.
Orchestration: Message Generation Pipeline -
Message Generation Service coordinates all services:
<pre><code>User Query → Query Expansion (3-5 variants) → Hybrid Search (20-30 candidates)
→ Re-Ranking (5-10 selected) → Context Assembly (token budgeting)
→ Prompt Engineering → LLM Call → Citation Verification → Response with metrics
</code></pre>
Returns structured data: message content, source UUIDs, citations, verification results, pipeline latencies.
<h2>Database Design</h2>
Chunks table optimized for vector retrieval:
<pre><code class="language-sql">CREATE TABLE chunks (
 id UUID PRIMARY KEY,
 resource_id UUID REFERENCES resources,
 content TEXT,
 embedding Vector(384),
 chunk_metadata JSONB
);
CREATE INDEX idx_chunks_embedding ON chunks USING hnsw (embedding vector_cosine_ops);
</code></pre>
HNSW indexing enables approximate nearest-neighbor search in logarithmic time. For semantic search with millions of vectors, this speedup is essential.
Resources table tracks documents with <code>content_hash VARCHAR(64) UNIQUE</code> (SHA-256) and <code>is_duplicate_of</code> foreign key for deduplication.
Conversations &amp; Messages maintain chat history with source tracking, citations as JSONB, model metadata.
Workspaces enable personal/team/hybrid collaboration with data isolation via <code>workspace_id</code> in all queries.
<h2>API &amp; Infrastructure</h2>
REST Endpoints (full documentation at <code>/docs</code>):
<ul>
<li><code>POST /api/resources/upload</code> - Upload documents</li>
<li><code>GET /api/resources/{id}/embedding-status</code> - Poll async embedding progress</li>
<li><code>POST /api/conversations/{id}/messages</code> - Triggers RAG pipeline</li>
<li><code>GET /api/conversations/{id}/export</code> - Export as JSON/Markdown</li>
</ul>
Docker Stack (7 services, <code>docker-compose up</code>):
<ol>
<li>PostgreSQL (pgvector pre-loaded)</li>
<li>Redis (cache + Celery broker)</li>
<li>Ollama (local LLM)</li>
<li>FastAPI backend</li>
<li>Celery worker (async embeddings)</li>
<li>Celery Beat (optional scheduled tasks)</li>
<li>Vite frontend</li>
</ol>
Health checks ensure dependencies before dependent services start. Models persist in volumes (~2GB total).
<h2>Frontend</h2>
React 18 + TypeScript. React Query manages server state (caching, invalidation). Zustand for UI state. API client wrappers shield UI from streaming/polling complexity. Tailwind CSS for styling.
<h2>Performance &amp; Design Patterns</h2>
Key Patterns:
<ul>
<li>Async-First: Embeddings/LLM happen async via Celery; API returns immediately</li>
<li>Content Dedup: SHA-256 hashing prevents re-processing identical documents regardless of source</li>
<li>Hybrid Search: Reciprocal rank fusion merges semantic + BM25 for robustness</li>
<li>Token-Aware Assembly: Respects context windows, prioritizes by relevance, truncates intelligently</li>
<li>Multi-Factor Ranking: Combines recency, specificity, source quality, usage history into unified ranking</li>
<li>Citation Verification: Validates LLM claims against source chunks post-generation</li>
<li>Hardware Adaptation: Auto-detects GPU/CPU/VRAM, adjusts timeouts and models accordingly</li>
</ul>
<a href="https://github.com/keshavashiya/docify" target="_blank" rel="noopener nofollow">Docify</a>

Keshav Ashiya

Docify is an open-source RAG system that processes documents locally while maintaining AI capabilities. The architecture uses 11 specialized services including async embedding generation with Celery, hybrid search combining pgvector and BM25, multi-factor ranking with citation verification, and token-aware context assembly. Built with PostgreSQL pgvector for vector storage, Redis for task queuing, and Ollama for local LLM inference, it supports heterogeneous document formats and implements deduplication via SHA-256 hashing. The system uses HNSW indexing for sub-200ms vector search, reciprocal rank fusion for search result merging, and citation verification to reduce hallucinations.

Docify: Building a Production RAG System for Knowledge Management