Neywa-Labs/mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX

Summary

MLX-VLM is a Python package for running and fine-tuning vision-language models locally on Apple Silicon Macs through MLX. It provides command-line, Python, Gradio chat, and FastAPI server interfaces for working with models that accept text, images, video, or audio.

What it is useful for

• Local multimodal inference: Engineers can run text generation, image description, audio understanding, or combined image-and-audio prompts against locally stored or Hugging Face models.
• Application prototyping: The Python API supports loading a model, applying its chat template, and generating output directly. This makes it suitable for embedding multimodal behavior in scripts, internal tools, or experiments.
• Local service development: mlx_vlm.server provides a FastAPI-based server for applications that need a model endpoint rather than a command-line call.
• Model preparation: mlx_vlm.convert can convert and quantize Hugging Face models for MLX, including documented examples using 4-bit quantization.
• Inference experimentation: The project supports thinking-token budgets and speculative decoding with DFlash, EAGLE-3, and MTP drafters. Model-specific documentation is important because supported pairings and behavior vary.
• Specialized workflows: The README includes examples for speech generation, multi-image chat, OCR-oriented models, and image-to-3D Gaussian splats.

How engineers can use it

The project requires macOS on Apple Silicon and Python 3.10 or newer. Install the base package with:

pip install -U mlx-vlm

Run a documented image-understanding example:

mlx_vlm.generate \
  --model mlx-community/Qwen2-VL-2B-Instruct-4bit \
  --max-tokens 100 \
  --temperature 0.0 \
  --image /path/to/image.jpg \
  --prompt "Describe this image."

For Python code, the README shows the core workflow: call load(), format the prompt with apply_chat_template(), and pass the prepared inputs to generate(). Audio and combined image/audio examples are also provided.

If an interactive interface is needed, install the optional UI dependency and launch the documented chat command:

pip install -U 'mlx-vlm[ui]'
mlx_vlm.chat_ui --model mlx-community/Qwen2-VL-2B-Instruct-4bit

For service-based work, start the server with mlx_vlm.server --model <model>. Check each model’s linked README for its prompt format and supported modalities. The Gradio UI is not included in the base installation, DSpark decoding currently requires greedy sampling, and fine-tuning should be followed through the README’s dedicated section because model-specific setup details are not interchangeable.