One ingest command
docmancer docs add handles URLs, GitHub repositories, and local paths. It writes into the same default local hybrid index Docmancer uses for memory: SQLite FTS5 plus dense vectors from the vendored model in sqlite-vec. Sparse retrieval is available only on the scale retrieval profile, which uses Qdrant and SPLADE.
Add from a URL
docmancer docs add https://docs.example.com
Docmancer auto-detects the docs platform and chooses the best fetching strategy:
| Platform | Detection | Strategy |
|---|---|---|
| GitBook | llms-full.txt endpoint | Full-text download |
| Mintlify | llms.txt or sitemap.xml | Sitemap crawl |
| GitHub | Repository URL | README + docs directory extraction |
| Generic web | sitemap.xml or nav crawl | Page-by-page fetch |
Force a specific provider:
docmancer docs add https://docs.example.com --provider mintlify
Add from a GitHub repo
docmancer docs add https://github.com/owner/repo
Extracts the README and any docs/ directory content.
Ingest local files
docmancer docs add ./my-internal-docs
Supported file formats: Markdown (.md, .mdx), plain text (.txt), PDF (.pdf), DOCX (.docx), RTF (.rtf), and HTML (.html, .htm). Every parser ships in the base install, so none of these needs an extra.
Options
docmancer docs add flags:
| Flag | Default | Description |
|---|---|---|
--provider | auto | auto, gitbook, mintlify, web, github, crawl4ai |
--strategy | auto | Force discovery strategy (e.g. llms-full.txt, sitemap.xml, nav-crawl) |
--max-pages | 500 | Limit pages fetched (web provider) |
--browser | off | Use Playwright for JS-heavy sites (needs docmancer[browser]) |
--fetch-workers | auto | Number of concurrent page fetch workers |
--include / --exclude | unset | Glob patterns for a local path |
--format | all | Restrict local ingest to specific formats |
--recursive / --no-recursive | recursive | Recurse through local directories |
--skip-known | off | Skip local files whose content hash is already indexed |
--no-vectors | off | Index local files with FTS5 only |
--recreate | off | Drop and rebuild the index for this source |
List and update existing sources
docmancer docs list
docmancer docs sync
docmancer docs sync https://docs.example.com
docs list shows every indexed source (add --all for individual pages and files). docs sync with no argument refreshes every source; pass a specific URL or path to update only that one. Updates reuse the content-hash-keyed embeddings cache, so unchanged sections skip re-embedding.
Remove a source when you no longer need it:
docmancer docs remove https://docs.example.com
docmancer docs remove --all