lightningpixel/modly: Desktop app to generate 3D models from images or prompt using local AI — runs entirely on your GPU

Desktop app to generate 3D models from images or prompt using local AI — runs entirely on your GPU - lightningpixel/modly

Summary

Modly is a TypeScript desktop application for generating 3D meshes from images with open-source AI models that run locally on a GPU. It provides workflows for connecting generation steps, an extension system for models and processes, and a Python CLI so agents and scripts can drive a running desktop instance.

What it is useful for

Use it when an engineer needs to prototype 3D assets from source images without moving inference to a hosted API. The README’s basic workflow is “Image → Generate Mesh → Add to Scene”: connect the steps, select the workflow on the Generate tab, and run “Generate 3D Model.” Imported meshes can also be smoothed and decimated in the app, with the optimized result written back to the workspace.

Model support is supplied through extensions. Official extension repositories cover Hunyuan3D 2 Mini, Mini Turbo, Mini Fast, TripoSG, and Trellis2 GGUF. An extension is a GitHub repository with a manifest.json; manifests can describe multiple Hugging Face model repositories, shared weights between nodes, and separately installed quantization or precision variants. This is useful for teams that need to choose a local model, install weights on demand, or reuse shared weights across generation nodes.

How engineers can use it

The README lists installers on the repository’s Releases page for Windows, Linux, and Apple Silicon macOS. Alternatively, clone the repository and use launch.bat on Windows or ./launch.sh on Linux/macOS. For development, install JavaScript dependencies with npm install. Set up the Python backend with:

cd api
python -m venv .venv
.venv\Scripts\activate     # Windows
source .venv/bin/activate  # Linux / macOS
pip install -r requirements.txt

From the repository root, start the app with npm run dev. The documented checks are npm test, ./node_modules/.bin/tsc --noEmit -p tsconfig.node.json, and npm run build.

Install a model extension by opening Models, choosing “Install from GitHub,” entering the extension’s HTTPS repository URL, and downloading the model or variant exposed by its node. To automate generation, keep the desktop app running and use:

python tools/modly-cli/agent.py health
python tools/modly-cli/agent.py generate --image ./input.png --output ./export.glb

The CLI keeps final machine-readable JSON on stdout and returns workflow-run recovery information if a run needs inspection.

Important constraints are explicit: macOS support targets Apple Silicon only; the generic artifact endpoint currently accepts only scene; scene input is model-only and must be a single input; and Hugging Face is the only supported model-source provider. The README also notes that dev serve-api starts only the FastAPI backend, not a complete Electron/Desktop bridge. Consult the README and tools/modly-cli/SKILL.md for the current automation contract.