Core AI in iOS 27: Apple's On-Device AI Runtime

Core AI arrives in iOS 27 as a substantial addition to Apple's ML stack. It is neither a rebranded Core ML nor the same thing as Foundation Models. Instead it is a dedicated deployment and inference system built for modern neural networks and generative AI running locally on Apple silicon — the same execution architecture that powers the on-device part of Apple Intelligence, now opened to developers' own models.

Core AI is neither a renamed Core ML nor a synonym for Foundation Models. Apple describes it as the framework driving the on-device side of Apple Intelligence, and developers can now build on that same execution architecture. The key claim is that Core AI stands on its own as a new deployment and inference system, sharing Apple's own production infrastructure rather than repackaging an older tool.

Core ML is not going away. Apple advises keeping it for non-neural workloads like decision trees and tabular models. Core AI, by contrast, is built for contemporary neural architectures — LLMs, VLMs, diffusion, segmentation, audio and other large tensor-heavy networks. The two coexist, each suited to a different class of model.

Core AI is best understood as an entire model-deployment toolchain rather than a single API call. Around its Swift runtime, Apple has built Xcode integration, Instruments profiling, a dedicated debugger, model caching, device specialization, custom Metal kernel support, and a repository of converted models. This is far richer than the old "MLModel loads a neural network" idea — it is a complete production inference platform.

Core AI runs models entirely on Apple silicon using the CPU, GPU and Neural Engine, so user input need not leave the device. Apple advertises zero server dependencies and zero token costs, plus privacy and responsiveness. It isn't literally free — the user paid for hardware and the app burns power, storage and memory — but the marginal cloud inference bill can drop to zero.

Apple now offers multiple AI API layers. Foundation Models is a high-level, capability-based interface to models Apple or providers manage, including the on-device model and Private Cloud Compute. Core AI is much lower level: you supply and control the model, its weights, compression, execution, memory, specialization, caching and even custom kernels. Apple frames the two as complementary rather than competing choices.

Apple has published a Core AI Models repository containing four parts: export recipes for supported models, reusable Python authoring utilities, Swift runtime utilities, and AI coding-agent skills. Its WWDC demo covers models like Qwen, Mistral and SAM3. The point is that this is more than a model zoo — Apple provides the recipes for turning research and open-source models into production Apple-silicon deployments.

Core AI introduces a new artifact, the .aimodel, distinct from Core ML's .mlmodel and friends. The key word is portable: the file isn't the final executable for any single iPhone. Instead Core AI takes it and specializes it for whatever hardware will run it, so one source representation can target different Apple silicon configurations across device generations.

When an app creates an AIModel, Core AI inspects the device and builds an optimized representation, by default choosing the CPU/GPU/Neural Engine combination that minimizes latency. Developers can override with options like .cpuOnly or preferred compute units, but Apple recommends benchmarking first since the default usually wins. The philosophy: describe the model instead of hardcoding which layer runs where.

Because specializing a large model is expensive, Core AI provides an AIModelCache. The first run specializes and caches a device-specific executable; later runs just load from cache and infer. Caching is automatic by default, and developers can inspect caches, control persistence, pre-specialize downloaded models, and share specialized models across apps in the same App Group — crucial UX for multi-gigabyte generative models.

Beyond caching, Apple offers ahead-of-time compilation through the coreai-build tool, converting a .aimodel into architecture-specific .aimodelc assets at build time. This moves the most expensive specialization work onto the developer's Mac, leaving the user's device to perform only lightweight final specialization at first run — a significant deployment improvement for apps shipping large models.

A Core AI model isn't limited to a single input-to-output path — it can expose multiple named inference functions. Developers inspect model.functionNames and load a specific one. Crucially, the AIModel object is deliberately lightweight and does not own the weights or intermediate buffers; those resources belong to the inference functions loaded from it. That separation matters for managing memory in very large networks.

Core AI treats memory as a dominant constraint, offering fine-grained memory control, stateful execution, asynchronous compute and zero-copy data paths. Generative models hold large autoregressive state, and camera apps run vision models repeatedly over frames — chains of buffer copies from camera to CPU to framework to GPU become extremely expensive. A zero-copy architecture avoids those redundant transfers, improving not just latency but memory bandwidth and battery efficiency.

Core AI Optimization is a separate component for model compression. Quantization stores parameters in fewer bits (e.g. FP16 to INT8), while palettization has weights reference a smaller palette of shared values. Crucially, compression can be configured per layer — mild on embeddings, aggressive elsewhere, precise on critical outputs. The aim is to trade off quality, model size, speed and memory bandwidth, reducing size and boosting performance with minimal accuracy loss.

Apple makes the pipeline researcher-friendly: the workflow can begin in PyTorch, pass through Core AI PyTorch Extensions, and become a Core AI model. The extensions can export multiple inference functions into a single artifact and include hardware-optimized implementations of common ops like attention and normalization, plus support for custom Metal 4 kernels when built-ins fall short. For specialized models, Apple even discusses re-authoring parts around target-aware layouts.

Because AI research moves faster than OS frameworks, a new architecture may need an operation Apple's runtime doesn't yet implement well. Rather than waiting for Apple, developers can supply their own Metal 4 kernels, plugging the missing op straight into their Core AI model. This escape hatch acknowledges that today's transformer won't dominate forever, keeping Core AI adaptable to fast-moving architectures.

The dedicated Core AI Debugger does far more than draw the model graph: it inspects tensors, compares intermediate values, identifies sync points, computes similarity metrics like PSNR, and — most interestingly — traces runtime tensors back to the original Python source. This tackles one of the nastiest conversion problems: finding exactly where numerical divergence entered a converted model, by comparing reference values against Core AI results.

Core AI is deeply wired into Apple's developer tools: Xcode inspects Core AI graphs and validates artifacts, and Instruments has a dedicated Core AI instrument for profiling runtime inference. So instead of vaguely noting "the model feels slow," developers can drill down through model, function, operation, compute device, execution duration and memory behavior — treating inference like any CPU or graphics workload, which matters because architecture alone doesn't determine real performance.

Underneath Core AI, iOS 27 changes Neural Engine behavior. Loading models larger than 1 GB is faster on Apple Intelligence-capable devices. ANE memory is now attributed to the app process rather than to system memory, so it shows up in Instruments' Allocations — good for debugging but a reminder to watch large-model footprint. Separately, background ANE inference now requires a specific entitlement, gating power-hungry continuous background generation.

Apple's examples reveal the range: from a small speaker-diarization model up to a 70-billion-parameter LLM where sufficient Apple silicon is available. Not every iPhone runs 70B comfortably — the idea is a spectrum, with compact models on iPhone, larger ones on iPad, multi-billion-parameter models on MacBook and Mac Studio, and potentially very large LLMs on high-memory Macs. Core AI gives all these hardware classes a common runtime abstraction.

The big shift is that Apple treats on-device AI inference as a normal OS capability, moving features from cloud GPUs and per-token billing toward local hardware accelerators — improving economics, privacy, offline operation, latency and model choice. But it doesn't remove constraints, it relocates them: from GPU cost, token cost and rate limits toward RAM, bandwidth, storage, thermals, battery, specialization time and device fragmentation.

Where to go next: explore the Core AI Models repository and its recipes, try the PyTorch export path and Core AI Optimization, lean on caching and AOT compilation for large models, and profile with Instruments and the Debugger. Remember the APIs are beta as of August 2026 and may change before release.