Meet Core AI — Apple's iOS 27 runtime for running modern neural networks locally, right on Apple silicon. Not a cosmetic rename of anything. It's a brand-new framework, and the same engine that powers Apple Intelligence's on-device work is now open to developers. 🧵
Core AI vs Core ML: it's a split, not a replacement. Core AI handles the big tensor-heavy neural networks — LLMs, VLMs, diffusion, segmentation. Core ML stays the right tool for decision trees, tabular models, and other classic ML. Both live on. Pick the lane that fits your workload.
Core AI isn't a single load-a-model call — it's a whole toolchain. A research model flows through export, optimization, compilation, device specialization, and a Swift runtime onto hardware. Wrapped around it: profiling, debugging, caching, custom kernels, and a converted-model repository. The full deployment story.
Core AI runs completely on-device. No server round-trip, no per-token cloud bill. That marginal inference cost falls to zero. But it's not literally free — the user's hardware, power, and your engineering time are still the price. The bill just moves onto the device.
Core AI and Foundation Models are complementary levels, not rivals. Foundation Models is the capability-level door: Apple manages the models (including Private Cloud Compute). Core AI is the low-level door: you bring your own architecture, weights, compression, and execution — a Qwen or Mistral of your choosing.
The new .aimodel format is portable — deliberately not the final per-device binary. One .aimodel gets specialized into different executables for different devices. That portability is the point: Apple can optimize the same model differently across hardware generations without you re-exporting.
Specialization is expensive — so Core AI caches it. First run: the model is specialized and the device-specific executable is stored. Every launch after: load straight from cache, skip the recompile. Caches can even be pre-warmed, persisted, and shared across apps in the same App Group.
Ahead-of-time compilation moves the heavy lifting to your Mac. The coreai-build tool compiles .aimodel into .aimodelc at build time, so the user's phone only does a lightweight final specialization step. First-run cost drops sharply — shipping large models gets much smoother.
Memory management is a first-class feature here. Redundant buffer copies waste latency, bandwidth, and battery. Core AI's zero-copy paths feed a shared buffer directly into a tensor. Crucial for generative and streaming work — LLMs hold state across steps, vision apps rerun every frame.
One runtime, one abstraction — scaling from a compact phone VLM up to multi-billion-parameter models, even a 70B LLM on high-memory Macs. Core AI reframes local inference as ordinary OS infrastructure. The tradeoffs shift from cloud bills to device RAM, bandwidth, and battery. (Beta as of Aug 2026 — APIs may change.)