Apple has new Mac app coming to make running local AI models easy
Learn how Apple’s app uses model distillation and silicon acceleration to make local AI inference practical.
Apple unveiled a Mac app in private preview that runs local AI models without sending data to the cloud, targeting tiny judgment models such as TypeSafe’s Jev. The most striking fact is that the app promises to handle classification‑style tasks, email importance, request routing, risk assessment, directly on the machine. It aims to replace heavyweight LLM calls with fast, on‑device decisions.
The app leans on model distillation, trimming parameters so that lean models run efficiently on Apple’s M7 Max and Ultra silicon. It uses the same low‑level runtime tricks found in llama.cpp, tapping both CPU and GPU cores for inference. By shrinking the model size, Apple can deliver comparable accuracy with far fewer FLOPs, cutting latency and power draw.
Apple is shipping the tool under its revived Claris brand, the name of its old subsidiary that re‑emerged in 2019. The preview is limited to invited users, but the interface mirrors the clean design of llama.app, offering simple prompts and instant feedback. This branding signals Apple’s intent to integrate AI utilities into its broader productivity suite.
For developers, the app opens a path to embed on‑device inference in macOS software. You package a distilled model, call the app’s API, and let the silicon handle the compute, eliminating network round‑trips and preserving user privacy. It also means you can prototype AI‑enhanced features without building a custom runtime from scratch.
TakeawayDistilled models can now run on‑device via Apple’s preview app, letting macOS developers add fast, private AI features without cloud services.