VLM-wiki

A full-modal personal knowledge base built on the Karpathy LLM Wiki concept.

Core Philosophy

• Multi-modal Intake: Support comprehensive recording of images, videos, audio, and text
• VLM + LLM: Use Vision Language Models to understand images/videos, and Language Models to organize knowledge
• Markdown First: All content is stored as standard Markdown and can be used by any editor or knowledge tool
• Continuous Compounding: The wiki is a persistent, ever-compounding personal knowledge base

Quick Start

1. Configure Model

Edit .vlmwiki/config.json to choose your model provider:

Supports local models (Ollama) or cloud APIs (OpenAI, Gemini, Claude).

For machine-specific values such as local paths, WeCom webhooks, or app secrets, put them in .vlmwiki/config.local.json. The repo reads config.json + config.local.json, and config.local.json should not be committed.

2.1 Frictionless source import

scripts/import_data_sources.py imports local export folders into raw/.

• phone photo album exports
• note exports
• Mubu diary exports
• any local folder you use as an inbox

Configure .vlmwiki/config.json under imports.sources, for example:

• photo_album
• note_export
• mubu_diary
• generic_text
• generic_diary
• generic_audio
• generic_video

The importer uses .vlmwiki/runtime/import_manifest.json to avoid importing the same file repeatedly.