Coming soon Emerging

On-device AI

AI that runs directly on your phone or laptop. It exists today and is expected to grow.

Forward-looking: this describes work in progress, not a finished product. Nothing here is a promised date or release.

What is it?

On-device AI means the model runs on your own phone, laptop or other device instead of on a remote server. This already happens for tasks such as photo enhancement, speech recognition, text suggestions and some smaller chat models.

As chips gain dedicated AI hardware and models become more efficient, more features are expected to run locally. Benefits include better privacy, working offline and faster responses. However, local models are usually smaller and less capable than the largest cloud models, so many products mix on-device and cloud processing.

Which models use it?

Apple Intelligence runs some features on-device and sends others to Apple's cloud servers.
Google's Gemini Nano is a small model designed to run on some Android phones.
Small open models such as Gemma, Phi, Llama and Mistral variants can run on laptops with tools like Ollama or LM Studio.
Many phones include neural processing units (NPUs) designed to speed up AI tasks.

How to use it

Check your device's settings to see which AI features run locally and which use the cloud.
Use on-device features when privacy matters, such as for personal notes or photos.
On a capable computer, try a small open model with a desktop tool like Ollama or LM Studio.
Compare results with a cloud model to understand the quality trade-off.
Keep your device updated, since new on-device features often arrive through updates.

Benefits

Data can stay on your device, which helps privacy.
Some features work without an internet connection.
Responses can be faster because nothing is sent to a server.

Limits and risks

On-device models are smaller and usually less capable than large cloud models.
Running AI locally uses battery, memory and storage.
Older devices may not support new on-device features.
How quickly local AI will match cloud quality is uncertain.

Prompt template

Copy it, replace the parts in [brackets] and paste it into any AI chat.

Help me decide whether to use on-device or cloud AI for this task. Task: [describe]. Device: [phone/laptop model and memory]. Privacy needs: [low/medium/high]. Internet access: [always/sometimes/never]. Compare the two options for quality, privacy, speed and cost, and suggest a setup I can try. Say clearly where you are unsure.