Nano-Collective/get-md

A fast, lightweight HTML to Markdown converter optimized for LLM consumption. Uses proven parsing libraries to deliver clean, well-structured markdown with intelligent content extraction and noise filtering

Summary

get-md is a TypeScript library and CLI from Nano Collective that converts HTML, PDF, DOCX, and Markdown inputs into structured Markdown, with output tuned for LLM consumption. It builds on established parsing libraries to extract main content and filter noise, and ships optional AI-powered conversion through a local LLM plus dedicated Mermaid diagram handling.

What it is useful for

Engineers feeding web pages, PDFs, or Office documents into LLM pipelines (retrieval-augmented generation, summarization, fine-tuning corpora) need a normalized text format. get-md targets that preprocessing step, with content extraction and noise filtering aimed at reducing irrelevant tokens before they reach the model.

The Mermaid handling is a practical differentiator for documentation workflows. Most HTML-to-Markdown converters drop diagrams, but get-md offers four documented strategies: Preserved (fence passes through, default), Recovered (source pulled from rendered SVG, default), Reconstructed (PDF diagrams rebuilt via a remote vision model, opt-in), and Validated (broken blocks flagged while keeping the original source, opt-in). Teams can pick the fidelity level that matches their dependency budget.

How engineers can use it

The README documents installation and a minimal call:

npm install @nanocollective/get-md

import { convertToMarkdown } from "@nanocollective/get-md";

const result = await convertToMarkdown("https://example.com");
console.log(result.markdown);

The CLI accepts a URL or local file and writes Markdown to disk:

npx @nanocollective/get-md https://example.com -o output.md
npx @nanocollective/get-md handbook.pdf -o output.md

For Mermaid validation, install the mermaid peer dependency and pass validateMermaid: true. For PDF diagram reconstruction, install pdfjs-dist and @napi-rs/canvas and opt in. The README also references configuration files, React Native usage, and the full API; those are covered in the linked docs site and the docs/ folder rather than inline.

Documented limitations: the GitHub license field is NOASSERTION, so usage terms are not declared in-repo and should be confirmed before production deployment. The README cites a sub-100ms figure without test conditions, so treat it as anecdotal. The Reconstructed Mermaid mode is described as best-effort and depends on a remote vision model.