Core AI in iOS 27: Apple's On-Device AI Runtime
This is the text the comic was made from, published by its author. ‹ back to the comic
Core AI in iOS 27: Apple’s New On-Device AI Runtime
Apple’s Core AI framework, introduced for iOS 27, is a major change in Apple’s machine-learning stack. It is not simply a rename of Core ML, nor is it the same thing as the Foundation Models framework. Core AI is a new deployment and inference system specifically designed for modern neural networks and generative AI running locally on Apple silicon.
Apple describes it as the framework that powers the on-device portion of Apple Intelligence itself. Developers can now use essentially the same underlying execution architecture for their own models.
Apple — Core AI overview
The important conceptual shift is:
Core ML was Apple’s general-purpose machine-learning deployment framework. Core AI is Apple’s modern high-performance neural-network inference stack.
Core ML is not disappearing. Apple specifically says developers should continue using Core ML for things such as decision trees, tabular models, and other non-neural-network workloads. Core AI targets contemporary neural architectures: LLMs, VLMs, diffusion models, segmentation models, audio models and other large tensor-heavy networks.
⸻
1. What Core AI actually provides
Core AI should be thought of as an entire model-deployment toolchain, rather than merely a Swift API.
The stack looks roughly like this:
PyTorch / research model ↓ Core AI PyTorch Extensions ↓ Core AI Optimization quantization / palettization ↓ .aimodel ↓ optional ahead-of-time compilation ↓ .aimodelc ↓ device specialization ↓ Core AI Swift runtime ↓ CPU / GPU / Neural Engine
Around that runtime Apple has built Xcode integration, Instruments profiling, a dedicated Core AI Debugger, model caching, device specialization, custom Metal kernel support and an open repository of already-converted models.
This is much closer to a complete production inference platform than the comparatively simple “MLModel loads a neural network” mental model associated with traditional Core ML.
⸻
2. Core AI is completely on-device
This is probably the most commercially important property of the framework.
Apple explicitly describes Core AI applications as having:
zero server dependencies and zero token costs.
Models execute locally on Apple silicon using combinations of the:
* CPU * GPU * Neural Engine
User input therefore doesn’t inherently have to leave the device.
For an AI application, this changes the economics considerably.
A conventional generative application might look like:
User ↓ iPhone ↓ Internet ↓ API server ↓ LLM/VLM inference provider ↓ tokens billed ↓ network ↓ iPhone
Core AI makes this possible instead:
User ↓ iPhone ↓ local Core AI model ↓ Apple Neural Engine / GPU / CPU ↓ result
There is consequently no inference API bill associated with executing that model.
Apple explicitly emphasizes the resulting benefits: privacy, responsiveness, zero server dependency and zero per-token inference cost.
That doesn’t mean on-device inference is literally economically free—the user bought the hardware, the app consumes power/storage/memory, and developers still bear engineering and distribution costs—but the marginal cloud inference bill can fall to zero.
⸻
3. Core AI isn’t the Foundation Models framework
This distinction is extremely important in iOS 27.
Apple now effectively has multiple levels of AI API.
Foundation Models
The Foundation Models framework gives developers access to models Apple or other providers operate/manage, including Apple’s on-device model and Private Cloud Compute models.
It is primarily a capability-level interface.
You say, conceptually:
"Use an appropriate foundation model to reason/generate/tool-call."
Apple handles much of the underlying model infrastructure.
Core AI
Core AI is much lower level.
You provide and control the model.
"I want THIS Qwen/Mistral/vision/ segmentation/custom PyTorch model running locally."
You control:
* model architecture * model weights * compression * execution * inference state * memory * specialization * caching * potentially custom GPU kernels.
Apple summarized the distinction at WWDC26: Foundation Models gives applications access to Apple’s model ecosystem; when a developer wants to bring a specific model into an application and run it on-device, Core AI is the appropriate framework.
So the two frameworks are complementary rather than competitors.
⸻
4. Apple is explicitly supporting open models
One of the biggest changes is that Apple has published a Core AI Models repository.
Apple Core AI Models repository on GitHub
Apple’s WWDC demonstration discusses models including Qwen, Mistral and SAM3.
The repository provides four major pieces:
Component Purpose models/ Export recipes for supported models python/ Reusable Python authoring/export utilities swift/ Runtime utilities for applications skills/ AI coding-agent skills for working with Core AI
The repository is therefore more than a model zoo.
Apple is providing recipes for taking research/open-source models and transforming them into production Apple-silicon deployments.
⸻
5. The new .aimodel format
Core AI introduces a new model artifact:
.aimodel
This is distinct from the familiar Core ML:
.mlmodel .mlpackage .mlmodelc
A .aimodel is a portable Core AI model representation.
The key word is portable.
The file is not necessarily the final executable representation for an individual iPhone. Instead, Core AI takes that model and specializes it for the hardware on which it will execute.
That gives Apple considerable freedom to optimize models differently across generations of hardware.
For example:
model.aimodel │ ├── iPhone architecture A │ ↓ │ specialized executable │ ├── iPhone architecture B │ ↓ │ specialized executable │ └── Mac architecture ↓ specialized executable
The same source representation can therefore target different Apple silicon configurations.
⸻
6. Hardware specialization is fundamental
When an application creates an AIModel, Core AI can automatically specialize it for the current device.
Conceptually:
let model = try await AIModel(contentsOf: modelURL)
Core AI examines the hardware and creates an optimized representation for it. By default, Apple allows the system to choose the CPU/GPU/Neural Engine combination that minimizes inference latency.
Developers can override that behavior.
For example, Core AI exposes specialization options including:
.default .cpuOnly preferredComputeUnitKind allowedComputeUnitKinds
Apple cautions developers that the default configuration will normally produce the best performance and recommends benchmarking before manually constraining execution.
This is an important architectural philosophy.
Instead of developers hardcoding:
Run layer X on GPU. Run layer Y on ANE.
the developer can describe the model while Core AI performs device-aware specialization.
⸻
7. Specialization is cached
Specializing a large model can be expensive.
Apple therefore provides an AIModelCache.
The first run might look approximately like:
.aimodel ↓ specialization ↓ device-specific executable ↓ cache
Subsequent runs become:
cache ↓ load ↓ inference
rather than repeating specialization every launch.
Core AI automatically caches specialized models by default. Developers can also explicitly check caches, control persistence, pre-specialize downloaded models and share specialized models between applications belonging to the same App Group.
This becomes particularly important for multi-gigabyte generative models where waiting for compilation/specialization at first launch would produce terrible UX.
⸻
8. Ahead-of-time compilation
Apple has another solution to the first-run problem: ahead-of-time compilation.
The command-line tool is:
coreai-build
It can convert:
.aimodel
into architecture-specific:
.aimodelc
assets during the build process.
Instead of making an iPhone perform the expensive compilation work, much of that process occurs on the developer’s Mac.
At runtime:
Xcode/build machine ↓ expensive compilation ↓ architecture-specific .aimodelc ↓ app distribution ↓ iPhone ↓ lightweight final specialization
Apple says this moves the most expensive portion of specialization to the build machine, substantially reducing the work necessary when the model first runs on the user’s device.
For applications shipping large models, this is a significant deployment improvement.
⸻
9. Models can contain multiple inference functions
Another architectural improvement is that Core AI models aren’t necessarily simply:
input → model → output
A Core AI model can expose multiple named inference functions.
The Swift object model looks roughly like:
AIModel ↓ InferenceFunction ↓ InferenceValue ↓ NDArray / image / etc.
Developers can inspect:
model.functionNames
and load a particular function:
let function = try model.loadFunction(named: "main")
The model itself is deliberately lightweight: Apple says AIModel doesn’t itself own the weights or intermediate buffers. Those resources are associated with the inference functions loaded from it.
That separation matters for memory management when applications are dealing with very large networks.
⸻
10. Memory management is a first-class feature
Apple repeatedly emphasizes that Core AI has been designed for workloads where memory is one of the dominant constraints.
The framework supports:
* fine-grained inference memory control * zero-copy data paths * stateful execution * explicit inference functions * asynchronous compute * multidimensional arrays * hardware specialization.
This is especially relevant for generative models.
An LLM isn’t merely:
prompt → inference → response
It maintains considerable state during autoregressive generation.
Likewise, a camera application may repeatedly execute a vision model over incoming frames.
Unnecessary copies become extremely expensive:
camera ↓ COPY CPU buffer ↓ COPY framework buffer ↓ COPY GPU buffer
A zero-copy architecture attempts to avoid these redundant memory transfers.
Conceptually:
camera/shared buffer ↓ Core AI tensor ↓ GPU / Neural Engine
That is important not only for latency but for memory bandwidth and battery efficiency.
⸻
11. Core AI Optimization
Apple has separated model compression into another component:
Core AI Optimization.
Its primary techniques currently include:
Quantization
Represent model parameters using fewer bits.
Conceptually:
FP16 ↓ INT8 / lower precision
This can dramatically reduce model storage and memory bandwidth.
Palettization
Instead of storing every individual floating-point weight, weights can reference a smaller palette of shared values.
Apple allows developers to configure compression differently at the layer level.
That means a developer could theoretically do something like:
Embedding → mild compression Attention → stronger compression Critical output → preserve precision Other layers → aggressive compression
rather than applying one universal compression policy.
The goal is finding the appropriate trade-off among:
quality ↕ model size ↕ speed ↕ memory bandwidth
Apple specifically says these optimizations are designed to reduce model size and improve inference performance with minimal accuracy degradation.
⸻
12. PyTorch becomes a much more direct part of Apple’s deployment pipeline
Apple is also making the workflow much friendlier to ML researchers.
The expected workflow can start with PyTorch:
PyTorch ↓ Core AI PyTorch Extensions ↓ Core AI model
Apple’s extensions support exporting one or more inference functions into a single model artifact.
They also include hardware-optimized implementations of common neural-network operations such as:
* attention * normalization
and support custom Metal 4 kernels when built-in operations aren’t sufficient.
For highly specialized models, Apple even discusses re-authoring parts of a network around target-aware data layouts and execution patterns.
That is a much deeper level of control than “convert a model and hope Core ML runs it efficiently.”
⸻
13. Custom Metal kernels
One particularly significant capability is custom GPU execution.
Core AI PyTorch Extensions allow developers to bring their own Metal 4 kernels.
This matters because the AI research frontier moves much faster than OS frameworks.
A new architecture might introduce an operation that Apple’s runtime doesn’t yet implement optimally.
Rather than waiting for Apple:
new research architecture ↓ unsupported operation ↓ custom Metal kernel ↓ Core AI model
This makes Core AI considerably more adaptable to new model architectures.
It is an acknowledgement that a modern AI runtime cannot assume that today’s transformer implementation will remain the dominant architecture indefinitely.
⸻
14. Core AI Debugger is unusually sophisticated
Apple also introduced a dedicated Core AI Debugger application.
It goes well beyond simply displaying the model graph.
The debugger can:
* visualize model structure * inspect tensors * compare intermediate tensor values * identify synchronization points * calculate similarity metrics * trace runtime tensors back to the original Python source.
That last capability is particularly interesting.
The intended workflow becomes:
Python source ↓ PyTorch graph ↓ Core AI conversion ↓ device execution ↓ wrong result ↓ Core AI Debugger ↓ intermediate tensor ↓ original Python operation
This addresses one of the nastiest problems in model conversion: determining where numerical divergence entered the converted model.
The debugger can compare reference intermediate values with Core AI results and surface measures such as PSNR to help locate divergence.
⸻
15. Xcode and Instruments understand AI inference
Core AI is deeply integrated into Apple’s normal developer tools.
Xcode can inspect Core AI graphs and validate model artifacts, while Instruments provides a dedicated Core AI instrument for profiling runtime inference.
That means AI inference increasingly becomes something developers profile like CPU or graphics workloads.
Instead of:
"The model feels slow."
developers can investigate:
model ↓ function ↓ operation ↓ compute device ↓ execution duration ↓ memory behavior
That is important for production AI engineering because model architecture alone doesn’t determine actual application performance.
⸻
16. iOS 27 changes Neural Engine behavior
There are also some important OS-level changes underneath Core AI.
Apple’s current iOS/iPadOS 27 release notes say that iOS 27 includes Neural Engine improvements on Apple Intelligence-capable devices, including improved performance when loading models larger than 1 GB.
Memory accounting also changes.
Neural Engine memory is now attributed to the application process rather than appearing as system memory, making it visible in Instruments’ Allocations tool.
That’s good for debugging but also means developers need to pay much closer attention to the real memory footprint of large models.
⸻
17. Background Neural Engine inference is now explicitly controlled
There is another important iOS 27 restriction.
Apple says background Neural Engine access now requires the entitlement:
com.apple.developer.background-tasks .continued-processing.inference
The OS restricts background ANE access similarly to existing GPU restrictions.
So Core AI should not be interpreted as permission for arbitrary continuous background LLM execution.
Apple is maintaining control over power-intensive accelerator usage.
That makes sense on phones where unconstrained background generative inference could destroy battery life.
⸻
18. Core AI scales from small models to very large models
Apple’s examples reveal the intended scope.
At one end:
small speaker-diarization model
At another:
vision-language model
And Apple explicitly discusses running something as large as a:
70-billion-parameter LLM
where sufficient Apple silicon is available.
Obviously, that doesn’t imply running a 70B model comfortably on every iPhone.
The philosophy is instead:
iPhone → compact VLM / LLM / vision model iPad → larger local models MacBook / Mac Studio → multi-billion-parameter models high-memory Mac → potentially very large local LLMs
Core AI provides a common runtime abstraction across those hardware classes.
⸻
19. Device requirements
The Core AI Models repository currently specifies:
iOS 27+ / macOS 27+
and:
Xcode 27+
for running and integrating its Core AI models.
The dedicated Core AI Debugger requires macOS 27 or later, and connected devices must run at least iOS 27, iPadOS 27 or macOS 27 to specialize and execute Core AI models.
The APIs remain beta as of August 2026, and Apple’s documentation explicitly warns that they are preliminary and may change before the final OS release.
⸻
20. Core AI vs Core ML vs Foundation Models vs MLX
The easiest way to understand Apple’s new AI stack is as four different layers.
Technology Primary purpose Who controls model? Typical use Foundation Models High-level generative intelligence Apple/provider summarization, generation, reasoning, tool use Core AI Production on-device neural inference Developer LLMs, VLMs, diffusion, vision/audio models Core ML Traditional/general ML deployment Developer classifiers, regressors, trees, conventional ML MLX AI research/training experimentation Developer/researcher training, fine-tuning, experimentation on Apple silicon
Apple explicitly directs developers with non-neural models such as decision trees or tabular feature engineering toward Core ML rather than Core AI.
The architecture therefore increasingly resembles:
APP │ ┌───────────┴────────────┐ │ │ Foundation Models Core AI │ │ Apple / PCC / providers Your neural model │ CPU / GPU / ANE Core ML │ traditional ML MLX │ research / train / experiment
⸻
21. Why this is strategically significant
Core AI represents a bigger shift than another machine-learning framework.
Apple is effectively saying that local AI inference is becoming a normal operating-system capability.
Historically, an AI startup’s architecture often assumed:
client ↓ cloud ↓ expensive GPU ↓ model
Apple is pushing toward:
client ↓ local model ↓ hardware accelerator
That changes several dimensions simultaneously.
Economics
If a feature can be moved from an API model to Core AI:
1 million generations × cloud inference cost
can potentially become:
1 million generations × $0 marginal server inference
Apple explicitly advertises zero token costs and zero server dependencies for local Core AI inference.
Privacy
Sensitive inputs can remain on the user’s device.
Offline operation
Applications can potentially continue providing AI functionality without network connectivity.
Latency
There is no network round trip.
Distribution
The app developer can select a specific open or proprietary model rather than depending entirely on Apple’s system model.
⸻
22. But local AI introduces a different set of bottlenecks
Core AI doesn’t eliminate engineering constraints. It moves them.
Cloud AI is constrained heavily by:
GPU cost token cost network latency rate limits server scaling
On-device AI is constrained by:
RAM memory bandwidth model storage thermal envelope battery ANE/GPU availability specialization time model download size device fragmentation
That is why so many of Core AI’s APIs revolve around:
compression → specialization → caching → memory management → zero-copy execution → profiling.
Apple isn’t merely providing an inference API. It is providing infrastructure for managing those constraints.
⸻
23. The most important architectural takeaway
The really significant thing about iOS 27 isn’t simply:
“Apple can run AI models on the iPhone.”
Apple has been doing that for years.
The change is that Apple is exposing a modern, general-purpose, performance-oriented inference stack capable of tracking contemporary AI research.
The workflow is now much closer to:
Hugging Face / PyTorch ↓ choose architecture ↓ optimize / quantize ↓ Core AI PyTorch Extensions ↓ .aimodel ↓ AOT compile ↓ device specialization ↓ Core AI runtime ↓ CPU + GPU + Neural Engine ↓ Swift application
And Apple is simultaneously supplying the model repository, Swift runtime utilities, conversion libraries, compression system, Metal escape hatch, device specialization, caching system, debugger and profiling infrastructure needed to make that workflow practical.
That makes Core AI one of the most consequential developer changes in iOS 27. It turns Apple silicon from something that merely happens to accelerate ML into something much closer to a developer-accessible local AI inference platform.
Primary Apple sources
Core AI — Apple Developer
Meet Core AI — WWDC26
Integrate on-device AI models using Core AI — WWDC26
Dive into Core AI model authoring and optimization — WWDC26
Core AI framework documentation
Core AI Models — Apple GitHub repository
Core AI Debugger
iOS & iPadOS 27 release notes