vmlx

by jjang-ai · Python · ★ 725

vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!

#anthropic-api#kvcache-compression#kvcache-optimization#kvcache-reuse#llm#lmstudio#macbook#mcp-server#mlx#mlxllm#mlxstudio#omlx#omlx-alternative#openai-api#openclaw#openclaw-agent#persistent-memory#prefix-cache#vmlx

Install

pip install git+https://github.com/jjang-ai/vmlx.git

Auto-generated from the repo's primary language — check the repo's own README for its exact install instructions.

Claude Desktop config

A starting point for your claude_desktop_config.json (auto-generated — the repo's README may specify a published package name or required env vars):

{
  "mcpServers": {
    "vmlx": {
      "command": "uvx",
      "args": [
        "git+https://github.com/jjang-ai/vmlx.git"
      ]
    }
  }
}

From the README

MLX Inference Server for Apple Silicon Self-hosted inference server for LLMs, VLMs, and image generation on Apple Silicon. OpenAI + Anthropic + Ollama compatible HTTP API. Self-hosted; no third-party API keys required. Native MTP artifact detection and family-specific cache policy gates keep speculative/cache settings explicit and model-safe. Looking for a native Swift macOS app or Swift inference engine? See osaurus.ai. Quickstart • Models • Features • Image Gen • API • Desktop App • JANG • CLI • Config…

Read full README on GitHub →

Related ai & ml servers

gemini-cli

google-gemini

★ 105,607

An open-source AI agent that brings the power of Gemini directly into your terminal.

AI & ML · TypeScript

ruflo

ruvnet

★ 61,807

🌊 The leading agent meta-harness for Claude. Deploy intelligent multi-agent swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning swarm intelligence, RAG integration, and native Claude Code / Codex Integration

AI & ML · TypeScript

context7

upstash

★ 58,210

Context7 Platform -- Up-to-date code documentation for LLMs and AI code editors

AI & ML · TypeScript

JeecgBoot

jeecgboot

★ 46,903

AI 低代码平台「低代码 + 零代码」双驱动！低代码可一键生成前后端代码;零代码可 5 分钟搭建系统;AI Skills 一句话画流程、设计表单、生成整套系统。内置 AI聊天、知识库、流程编排、MCP插件等，兼容主流大模型。引领「AI 生成 → 在线配置 → 代码生成 → 手工合并->AI修改」开发模式，消除 Java 项目 80% 的重复工作，提效而不失灵活。

AI & ML · Java

UI-TARS-desktop

bytedance

★ 37,350

The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra

AI & ML · TypeScript

gpt-researcher

assafelovic

★ 27,938

An autonomous agent that conducts deep research on any data using any LLM providers

AI & ML · Python