App Development

Intelligent apps in 2026: LLM features and on-device models in mobile apps

Updated 13 min readBy Sandeep Mahajan

Two phones in front of an orange glowing circle on a dark navy background

In 2019 "intelligent app" meant an app with analytics, some machine learning and a sensor feed. In 2026 it means an app with a language-model feature: it summarises, extracts, classifies, answers from the app's own data, or takes an action through a tool call, and it does so either on the phone or through a cloud model. The decision that matters is where the model runs. Apple's Foundation Models framework and Google's ML Kit GenAI APIs give apps a free, offline, private on-device model for bounded tasks; cloud models from Anthropic, Google, OpenAI or Amazon Bedrock handle anything that needs world knowledge, long context or heavier reasoning, at a per-token cost and with privacy obligations. Most good intelligent apps use both.

Key takeaways

  • Apple's on-device model is "a large language model with 3 billion parameters, each quantized to 2 bits", built for "summarization, extraction, classification" and "not designed for world knowledge or advanced reasoning". It runs offline, costs nothing per call and adds nothing to app size. iOS 27 (released 14 September 2026) brought a rebuilt on-device model, image input and a larger Private Cloud Compute model with a 32,000-token context.
  • Google's ML Kit GenAI APIs run Gemini Nano on-device for summarisation, proofreading, rewriting, image description and a general Prompt API; "input, inference, and output data is processed locally". Device support is uneven and inference is allowed only while the app is in the foreground.
  • Voice is still the fastest input: in a Stanford and Baidu study, speech entry was "2.93 times faster" than keyboard typing in English (153 vs 52 words per minute), with fewer errors during entry, in laboratory conditions.
  • Store rules changed: since 13 November 2025 Apple requires apps to "clearly disclose where personal data will be shared with third parties, including with third-party AI, and obtain explicit permission before doing so". Google Play's AI-generated content policy covers chatbot and generation apps and requires in-app reporting of offensive output for the apps in scope.
  • The feature is only as good as its evaluation set and its hand-off to a human. That was true of 2017 chatbots and is true of 2026 LLM features.

What changed since the 2019 version of this article

The 2019 post listed five characteristics of an intelligent app: results without keyboard input, immediate assistance, behavioural analysis, data-driven reports and automation of repetitive tasks. All five are still what buyers want. What changed is the engine. Each used to need a bespoke model, a data-science team and months of training data; a general-purpose language model now does the language parts of all five out of the box, and the engineering effort moved to grounding, guardrails, evaluation and cost control.

Two things did not change. First, an app still needs a clear job per feature: "summarise this inspection report into five fields" ships; "add AI" does not. Second, the data the feature sees is the product. A model that answers from your help centre is only as good as the help centre (see our RAG buyer's guide).

Where the model runs: the decision that drives everything else

Criterion On-device model (Apple Foundation Models, Gemini Nano) Cloud model (Claude, Gemini, OpenAI, Bedrock-hosted) Hybrid
Cost per request None Per input and output token; price from the provider's pricing page Small tasks local, large tasks cloud
Works offline Yes No Degrades to local features
Latency Low, no network hop Network plus model time; streaming hides some of it Best of both if routed well
Privacy Data stays on the phone; Apple: "all data going into and out of the model stays private" Data leaves the device; disclosure and consent needed (Apple 5.1.2(i)); provider training terms matter Keep personal data local, send the rest
Capability Bounded tasks on short inputs; Apple's model is "not designed for world knowledge or advanced reasoning" Long documents, reasoning, multi-step tools, world knowledge Route by task
Device coverage Apple Intelligence-enabled devices in supported regions; a changing list of Android devices with AICore Any device with a connection Fallback to cloud on unsupported devices
Structured output Apple guided generation "guarantees structural correctness"; ML Kit Prompt API Provider JSON/tool schemas Both

Apple: the Foundation Models framework

Introduced with iOS 26 at WWDC 2025, the framework "gives you access to the on-device Large Language Model that powers Apple Intelligence, with a convenient and powerful Swift API" on iOS, iPadOS, macOS and visionOS. Everything "runs on-device, so all data going into and out of the model stays private. That also means it can run offline! And it's built into the operating system, so it won't increase your app size." The model "can only run on Apple Intelligence-enabled devices in supported regions", and the API exposes an availability check so the app can hide or degrade the feature.

Two features matter for product work. Guided generation (@Generable and @Guide macros) uses constrained decoding so the model returns a valid Swift type rather than free text: you ask for a struct with a date, an amount and a category, and you get one. Tool calling lets the model call code you define; because arguments are generable, "the model will never produce invalid tool names or arguments". Apple's research note says the on-device model is "approximately 3-billion-parameter", that the models are "designed to support 15 languages", and that Apple does "not use our users' private personal data or user interactions when training our foundation models".

iOS 26.4 added APIs to read the model's context size and count tokens (Apple's example prints a context size of 8,192), so an app can chunk input to fit. iOS 27 brought "a new on-device model, rebuilt from the ground up", image attachments in prompts ("the model supports images in any size and aspect ratio"), a PrivateCloudComputeLanguageModel that is "a much bigger model than the on-device models" with "a 32,000 token context window" and reasoning levels, usage reporting with token counts, a LanguageModel protocol so a session can be backed by a local or a server model, Swift packages from Anthropic and Google for their frontier models, Vision-backed barcode and OCR tools, a Spotlight-backed search tool for fully local retrieval, and an Evaluations framework for measuring prompt changes. Apple states the cloud model is free of API cost for developers with fewer than two million first-time downloads; check Apple's current terms before planning around that.

Android: ML Kit GenAI APIs and Gemma on-device

Google's ML Kit GenAI APIs "harness the power of Gemini Nano", running on AICore, "an Android system service that enables on-device execution of GenAI foundation models". As of October 2026 the APIs are Summarization, Proofreading, Rewriting and Image description (beta), Speech recognition (alpha) and a Prompt API (beta) for custom text or multimodal prompts. Google's privacy statement is direct: "input, inference, and output data is processed locally". Two constraints: inference is allowed "only when the app is the top foreground application", and support depends on the device and the Gemini Nano version it carries (Google's list names Pixel 9 to 11, Galaxy S25 and S26, Xiaomi 15 and OPPO Find X9 among others; the Prompt API needs newer "nano" versions and, per Google's October 2025 post, "works best on the Pixel 10 series"). For other hardware, Google's MediaPipe LLM Inference API runs Gemma 3n (E2B and E4B, text, image and audio input) on Android and the web from a bundled model file, at the cost of app size and your own benchmarking.

Cloud models

For anything that needs long context, world knowledge, reasoning or a tool chain across services, the app calls a hosted model, usually through your own backend so that keys, logging, rate limits and consent checks live server-side. The cost model is tokens in and out multiplied by volume; take prices from the provider's pricing page, not from an article, and build a cost estimate at three volumes before committing. If you are already on AWS, Amazon Bedrock lets you pick among several model families behind one API and keeps data in your region.

Five feature patterns that ship

  1. Capture without typing. Voice, camera or a photo of a form becomes structured data through speech recognition plus guided generation. The 2019 article's claim that speech beats typing holds: Ruan and colleagues measured English speech entry at 153 words per minute against 52 for the keyboard (2.93x), with a corrected error rate of 5.30% against 11.22%, though speech "left slightly more errors in the final transcribed text" (1.30% vs 0.79%), all in laboratory conditions. Design for a quick correction step.
  2. Summarise and draft inside the app. Meeting notes, inspection reports, support threads, product listings: bounded inputs, bounded outputs, a good fit for the on-device model and the ML Kit summarisation and rewriting APIs.
  3. Answer from the app's own data. A retrieval-augmented assistant over your help centre, policies or product catalogue, with citations and a "talk to a person" route. Apple's Spotlight-backed tool makes a fully local version possible for personal data; cloud retrieval is the norm for shared content.
  4. Act, with approval. Tool calling lets the model fill a reorder, draft a reply or schedule a visit; the app shows the proposed action and the person confirms. Anything that spends money or sends messages needs an approval step, limits and an audit log (see our AI agents service).
  5. Personalise from on-device signals. Recent activity, location and preferences shape suggestions without leaving the phone, which is both the better experience and the simpler privacy story.

Store rules and privacy

  • Apple. The 13 November 2025 update to App Review Guideline 5.1.2(i) "clarifies that you must clearly disclose where personal data will be shared with third parties, including with third-party AI, and obtain explicit permission before doing so." In practice: a clear in-app disclosure and consent before a feature sends personal data to a cloud model, with the on-device path needing none.
  • Google Play. The AI-Generated Content policy covers "text-to-text AI chatbot apps, in which the AI generated chatbot interaction is a central feature of the app", AI image generators and apps that create voice or video of real people; apps that only use AI to improve an existing feature are out of scope. Apps in scope must give users a way to report or flag offensive output without leaving the app, and generated content must follow the existing content policies.
  • Privacy law. For a business operating from Canada and India, the Office of the Privacy Commissioner's generative-AI principles (December 2023) and PIPEDA apply to personal data an assistant handles; India's DPDP Act and its 2025 Rules apply on a phased timeline that should be checked against the official notification. Say what is sent where, keep logs minimal, and get the provider's training-use terms in writing.

Scoping an AI feature: a six-step procedure

  1. Write the job as an input and an output. "Given a photo of a utility bill, return account number, amount, due date." If you cannot write it this way, it is not ready.
  2. Decide where it runs. Personal data, offline use, high volume or tiny outputs point on-device; long documents, reasoning or cross-system tools point to the cloud; most apps route per task.
  3. Build the evaluation set first. Fifty to a hundred real inputs with the expected output, scored on every build. Apple's Evaluations framework and a plain spreadsheet both work.
  4. Add the guardrails. Structured output schemas, refusal behaviour, a human route, rate limits, an audit log for any action.
  5. Design the disclosure. The in-app notice and consent for cloud processing, the Play reporting control if in scope, the privacy policy update.
  6. Measure in production. Task completion rate, correction rate, hand-off rate, cost per request, and thumbs-down review weekly.

What it costs

Build cost is driven by the number of features, the integrations and the evaluation work, not by the model. Run cost is close to zero for on-device features and is tokens multiplied by volume for cloud features, plus the weekly human review. We do not publish a typical price: the cost depends on scope, and the drivers are the number of AI features, on-device versus cloud models, the integrations with your backend and data, the size of the evaluation set, and the platforms (iOS, Android or both). Describe the features you have in mind and we will quote against them.

FAQ

What is an intelligent app in 2026?

An app with at least one language-model feature: summarising, extracting structured data, classifying, answering from the app's own content, or proposing actions through tool calls. The model runs on the device (Apple Foundation Models, Gemini Nano) for bounded private tasks, or in the cloud for long inputs, reasoning and cross-system work.

Can an AI feature work offline?

Yes, if it uses the on-device model. Apple's Foundation Models framework runs entirely on Apple Intelligence-enabled devices and "can run offline"; Google's ML Kit GenAI APIs process input and output locally on supported Android devices. Features that need a cloud model need a connection and should degrade gracefully.

Does on-device AI cost anything per request?

No. Apple's on-device model is part of the operating system and adds nothing to app size; Google's ML Kit GenAI APIs run on the device through AICore. Cloud models are billed per token by the provider; estimate at three volumes using the provider's pricing page before committing.

Do I need user consent to send data to an AI model?

If personal data leaves the device to a third-party AI service, Apple's guideline 5.1.2(i) (November 2025) requires clear disclosure and explicit permission first, and privacy law in Canada and India applies to how it is processed and kept. On-device processing avoids the transfer, which is one reason to prefer it for personal data.

Which phones support on-device models?

On iOS, devices that support Apple Intelligence in supported regions; the framework exposes an availability check. On Android, a vendor list of devices with AICore and a compatible Gemini Nano version (Pixel 9 and later, recent Galaxy S models and several others); the Prompt API needs newer devices. Plan a cloud or non-AI fallback for the rest.

Impinge Solutions builds native and cross-platform apps and adds grounded AI features to them as part of its mobile app development and AI assistants services. To scope a two-feature pilot, request an estimate.

Sources (13 links)
Share this article