Intelligent Personal Memory System

SinkDuce.

Spark. Capture every thought.
Sink. Organize into context-isolated collections.
Educe. Reason over your knowledge with full traceability.

A high-fidelity cognitive filter — not a document dump. One-click deployable RAG agent with 43-tool MCP server. Every answer traces back to its source through three layers of provenance.

Get Started docker compose up -d --build
Scroll
概念 Concept

SinkDuce is a high-fidelity cognitive filter — not a document dump.

It turns meetings, lectures, notes, and files into structured, context-isolated knowledge that you can query, distill, and federate across project boundaries. Every answer traces back to its source through three layers of provenance.

Built around three verbs — Spark (capture), Sink (organize), Educe (reason) — SinkDuce is a one-click deployable RAG agent that runs on your own machine. No cloud required. No subscription. Your memory stays yours.

S.
Spark — 采集
Record meetings with live transcription and auto-sectioning. Write structured notes with a full WYSIWYG editor. Drag, drop, distill. Every input — voice, file, link, text — lands in one index.
S.
Sink — 沉淀
Everything lands in context-isolated Collections — independent Qdrant vector databases. 12 format parsers. Intelligent chunking with parent-child mode. Auto-summarization and consolidation with conflict detection.
E.
Educe — 涌现
Ask questions. The system retrieves, grades, and synthesizes. Hybrid search + Agentic RAG with multi-collection federation. Every answer comes with 3-layer source traceability. Built-in recall evaluation suite.
I.

Spark — 采集 · Capture everything.

Meetings · Notes · Distill

Two independent capture entry points: Meetings for spoken audio with live transcription and auto-sectioning, and Notes for structured writing with cross-collection distillation.

Meetings · 会议

Record. Transcribe. Auto-section.

Record a meeting capturing both mic and system audio, or upload a file. FunASR runs locally for offline transcription; plug in DashScope or OpenAI-compatible cloud models for higher accuracy. Real-time WebSocket streaming shows live captions as you speak.

After transcription, a two-pass LLM pipeline fires: first, a general summary plus Blueprint auto-sectioning that uses your Collection catalog as a classification taxonomy; second, per-section deep summaries pinpointing relevant sentences from the transcript. Every sentence is clickable — jump to its source timestamp with synced audio playback.

🎬 Demo Video
Meeting Capture
录制 → 转写 → 自动分段 → 逐段深摘要
Audio Transcription
File upload or WebSocket realtime streaming. FunASR local offline; DashScope & OpenAI-compatible cloud. Speaker diarization, VAD, punctuation restoration.
Live Captions
Real-time transcription pushed during recording, auto-distinguishing partial vs final text. Transcript scrolls in sync with audio playback.
Blueprint Auto-Sectioning
LLM auto-detects topics using your Collection catalog as a taxonomy, decomposing meetings into semantically independent sections. Suggests new Collections for uncovered topics.
Sentence-Level Provenance
Every sentence clicks through to the source transcript timestamp with synced audio. Each sentence tagged with its topic section.
Editable Summaries
All summaries are editable Markdown. General Summary, section summaries, and meeting notes saved independently. Edits persist.
Hot Words & Language
Attach hot words libraries and multi-language hints to boost domain-specific ASR accuracy.
🎬 Demo Video
Notes & Distill
Tiptap 编辑器 → 拖拽蒸馏 → 变更传播 → 双向引用图
Notes · 笔记

Write. Distill. Propagate.

Create structured notes with a full Tiptap WYSIWYG editor — Markdown, headings, tables, task lists, code blocks, image paste/drag-drop, YouTube embeds. Auto-save.

Distill: drag any note onto the editor — the LLM condenses the source note's essence into a citation block. Results are cached. Propagate: when a source note changes, re-distill into all downstream notes with recursive chain propagation. Preview the full update chain before confirming.

A Bidirectional Reference Graph automatically maintains relationships. The sidebar shows Distill In (sources you reference) and Distill Out (notes referencing you).

Tiptap Editor
Full WYSIWYG with Markdown, headings, tables, task lists, code blocks, image paste/drag-drop, YouTube embeds. Auto-save.
Distill
Drag a note onto the editor — LLM condenses core insights into a citation block. Results auto-cached.
Propagate
Re-distill into all downstream notes → recursively chain-propagate. Preview the full update chain before confirming.
Bidirectional Reference Graph
Auto-maintained Distill In / Distill Out navigation in the sidebar.
II.

Sink — 沉淀 · Organize everything.

Collections · Parsing · Chunking · Consolidation

Everything lands in a Collection — an isolated Qdrant vector database. Each project, course, or domain gets its own Collection. Zero cross-contamination.

Collections · 知识库

Context-isolated. Independently configured.

Each Collection is its own Qdrant vector database with independent configuration: chunk mode, parent strategy, chunk sizes, embedding dimensions, search mode, file type allowlist, contextual enrichment, and MinerU cloud parsing toggles.

A Collection Catalog — per-collection definition, coverage scope, and tags — is maintained automatically and used by the agent to route queries to the most relevant Collections.

🎬 Demo Video
Collection Management
创建知识库 → 上传文档 → 自动分块 & 嵌入 → 跨库检索
12 Format Parsers
PDF (with OCR), DOCX, PPTX, XLSX, Markdown, HTML, CSV, JSON/JSONL, plain text, images (OCR). Optional MinerU cloud parser for higher-quality extraction with layout/table/formula preservation.
Parent-Child Chunking
Three strategies: paragraph-based, heading-based, or fixed-token. Parents carry full context; retrieval matches smaller children but returns parent text. CJK-aware sentence boundary detection.
Contextual Retrieval
LLM enriches each chunk with situating context to fill in missing global information. Large documents support async batch processing.
Auto-Summarization
Structured per-document summaries via LLM (Key Data / Facts / Insights). Collection-level consolidation with conflict detection flagging contradictions between sources.
Semantic Meeting Router
Multi-topic meetings split automatically: each section allocated to its most relevant Collection with one click.
Async Task System
Dual-queue architecture: upload queue + general pool with parallel processing. Cancellable and retryable tasks, live progress via SSE.
III.

Educe — 涌现 · Reason over everything.

Hybrid Search · Agentic RAG · Traceability · Evaluation

Ask a question. The system retrieves, grades, and synthesizes. Two search modes — Direct and Agentic — chosen automatically by the LLM based on query complexity.

Retrieval · 检索

Hybrid search. Agentic reasoning.

Direct mode: single-pass hybrid retrieval — dense vector + BM25 sparse (LLM-extracted keywords with synonym expansion), fused via Reciprocal Rank Fusion in Qdrant. Optional reranker re-scores candidates.

Agentic mode: full multi-step pipeline. Decompose complex questions into atomic sub-queries, route each to relevant Collections using catalog metadata, generate and retrieve multiple semantic variants in parallel, deduplicate, judge relevance in one call, aggregate and synthesize.

🎬 Demo Video
Agentic RAG in Action
复杂查询 → 拆解 → 并行检索 → 评分 → 聚合 → 带溯源的回答
Hybrid Search
Dense vector + BM25 sparse vectors. LLM extracts keywords and expands synonyms. Reciprocal Rank Fusion via Qdrant.
Multi-Provider Reranking
Cohere, DashScope/Qwen, OpenAI-compatible. Pluggable architecture — switch backends as needed.
Agentic RAG Pipeline
Decompose → parallel variant generation → retrieve → combined grade (relevance + gap analysis) → aggregate → synthesize. Fully observable.
Multi-Collection Federation
Query across multiple Collections simultaneously. Catalog metadata routes sub-queries to the most relevant Collections.
3-Layer Source Traceability
Answer → text snippet → full document → original file preview. Verify claims layer by layer.
Session-Based Chat
Persistent multi-turn conversations. LLM agent selects search strategy autonomously. Timeline shows interleaved thinking + tool calls. Think toggle for deep reasoning.
Recall Evaluation
Auto-generated test cases, LLM-as-judge scoring with reasoning. Metrics include recall, MRR, and quality score.
Per-Collection Quick Chat
Floating slide-out panel with SSE streaming, thinking display, and source navigation. Ideal for rapid lightweight Q&A.
IV.

MCP Server — 43 tools across 8 domains.

Collections · Documents · Search · Tasks · Summaries · Notes · Meetings · Hot Words

Your curated memory doesn't stay locked in the Web UI. Connect Claude Code, Cursor, or any MCP-compatible client — your AI coding assistant can directly search your knowledge bases, manage documents, and operate on meetings and notes.

One config · 一行配置

Your memory flows into every tool you use.

Add to .mcp.json at your project root (or ~/.claude/.mcp.json for global access). The MCP server reuses the app's services on the same FastAPI process — no separate server needed.

HTTP Streamable transport. 43 atomic tools across 8 domains. Your knowledge bases become directly queryable from your editor and terminal.

.mcp.json
{
"mcpServers": {
"sinkduce": {
"type": "http",
"url": "http://localhost:18900/mcp"
}
}
}
Collections
5 tools
List, get metadata+config, create (26 params), update, delete.
Documents
6 tools
List, upload, delete, chunk inspection, full-text extraction, toggle definitive flag.
Search
3 tools
Direct retrieval, Agentic RAG (full pipeline), query history.
Tasks
5 tools
List, get status, cancel, retry, clear completed.
Summaries
4 tools
Collection overview, per-document summary, conflicts, trigger consolidation.
Notes
6 tools
List, get, create, update, delete, trigger propagation.
Meetings
9 tools
List, get, transcript, create, update, delete, start summary, upload audio.
Hot Words
5 tools
List libraries, get, create, update, delete.
§

Tech Stack

Python · React · Qdrant · Docker
BackendPython 3.11, FastAPI, Uvicorn, Pydantic v2, PyYAML
FrontendReact 19, TypeScript, Vite 6, Tailwind CSS 4, Zustand, Radix UI, Tiptap, Recharts, Lucide React
Vector DBQdrant v1.13+ (dense + sparse BM25, RRF hybrid search)
LLM / EmbeddingOpenAI-compatible protocol, multi-provider with per-collection override
RerankingCohere, DashScope/Qwen, OpenAI-compatible
Parsingpdfplumber, mammoth, python-docx, openpyxl, python-pptx, markdownify, BeautifulSoup, Tesseract, MinerU cloud API
TranscriptionFunASR (SenseVoiceSmall, Paraformer, FSMN-VAD, CAM++, CT-Transformer), DashScope, OpenAI-compatible Whisper
MCPMCP SDK 1.0+, HTTP Streamable transport
InfrastructureDocker Compose (Qdrant + app), GitHub Actions CI

One command. Your knowledge, alive.

~/sinkduce · zsh
$ git clone https://github.com/superdd-coder/sinkduce.git
$ cd sinkduce
$ docker compose up -d --build
→ Starting SinkDuce...
✓ Qdrant vector database
✓ FastAPI backend + MCP server
✓ React frontend
 
ready · http://localhost:18900
 
# On first launch:
# 1. Download local transcription models (optional)
# 2. Settings → Add LLM provider
# 3. Add Embedding provider → Create first Collection
# Tip: Use DashScope OneShot for one-click setup
 
$
View on GitHub ↑ Back to top