How to add AI features to your app (a practical guide)

Start with the job, not the model

Most AI features that fail had no clear job to do. Someone adds a chat bubble because competitors have one, users try it twice and never open it again. The features that last take a task people already do by hand, often a slow or boring one, and make it faster.

A useful test: can you describe the feature without the word "AI"? "Turn a 40-minute call recording into five action items" is a feature. "An AI assistant for your workspace" is a hope. Write the first kind of sentence before you choose a model, a vendor or a framework.

I build these features for clients and for our own products at Loha.dev, and the pattern holds: the narrow version ships in weeks and gets used, while the broad version drags on and gets ignored.

AI features that actually help users

These are the categories I see earn their place in real apps, roughly ordered from easiest to hardest to get right:

• Semantic search. Users type what they mean ("invoice from the plumber last spring") and get results even when the words don't match. Built on embeddings, often blended with ordinary keyword search.
• Summaries. Long threads, documents, meeting notes or support tickets reduced to what matters. Low risk, because the source is right there to check.
• Drafting. First drafts of replies, descriptions, reports or listings that the user edits before sending. The human stays in charge, which makes mistakes cheap.
• Extraction from documents and images. Pull fields out of receipts, PDFs, IDs, forms or photos into structured data. Modern vision models handle messy scans far better than old OCR pipelines.
• Classification and routing. Tag incoming messages, detect urgency, sort leads, route tickets to the right queue. Invisible to users, but often the biggest time saver for a business.
• Chat assistants. A conversational layer over your product's data and help docs. Useful when grounded in real data; frustrating when it just guesses.
• Agents and tool use. The model takes actions: books the appointment, updates the record, files the report. Powerful, and the place where you need the most guardrails.
• Voice. Speech-to-text for dictation and notes, text-to-speech for hands-free use, or full real-time voice conversations. Great for field workers and anyone whose hands are busy.

Picking a model: hosted, open or on-device

You have three realistic options, and many apps end up using more than one.

Hosted APIs (Claude, OpenAI, Gemini)

For most products this is the right default. You get the strongest models, no GPUs to manage, and pay-per-use pricing. Each provider offers a range of sizes, from large models for hard reasoning to small, fast, cheap ones for classification and extraction. Quality differences between the top providers are real but task-specific, so test your actual prompts on two of them before committing. Keep your code provider-agnostic behind a thin interface so switching later is a config change, not a rewrite.

Open-weight models (Llama, Mistral, Qwen and others)

You can run these on your own servers or through hosting providers. They make sense when you have strict data residency rules, very high and steady volume where per-token pricing gets expensive, or a narrow task where a fine-tuned small model beats a general one. The tradeoff is operations work: serving, scaling, monitoring and upgrades become your problem.

On-device models

Phones now ship with small built-in models: Apple exposes one to developers through its Foundation Models framework on recent iOS versions, and Google offers Gemini Nano on supported Android devices. They are free per call, work offline and keep data on the phone, but they are much less capable than hosted models and not available on every device. Good for quick rewrites, short summaries and classification; not for complex reasoning. Plan a server fallback.

Architecture: how the pieces fit together

The architecture matters more than the model choice. Here is the shape I use for almost every AI feature in a custom app.

Call the model from your server, never from the app

An API key shipped inside a mobile app or web bundle will be extracted, and someone will run up your bill. The app calls your backend; your backend checks who the user is, applies rate limits and usage quotas, adds context, and then calls the model. This is also where you log requests, switch models and enforce content rules without shipping an app update.

Stream the response

Large models can take several seconds to finish. Streaming tokens to the UI as they arrive makes the same response feel fast. All major providers support streaming, and on mobile apps it works fine over server-sent events or a realtime backend. In our own stack, Convex makes it simple to write streamed output to the database and have every open client update live.

Retrieval (RAG) for your own data

Models don't know your users' documents, your product catalog or your policies. Retrieval-augmented generation fixes that: split content into chunks, store an embedding (a numeric fingerprint of meaning) for each chunk in a vector index, find the most relevant chunks for each question, and pass them to the model with an instruction to answer only from them. Most of the quality work in RAG is unglamorous: clean source data, sensible chunk sizes, hybrid keyword plus vector search, and filtering by user permissions so nobody sees another customer's data.

Tool calling and structured output

Tool calling lets the model request an action ("look up order 1234", "create a calendar event") that your code executes and validates. Structured output makes the model return JSON matching a schema you define, so you can save it straight to the database or render it in the UI. Together they turn a chatbot into a feature: the model decides, your code does, and you check everything in between.

Cost control: tokens, caching and smaller models

Hosted models bill per token, roughly per word fragment, for both input and output. Input is usually what grows quietly: a long system prompt, retrieved documents and chat history get resent on every call. A few habits keep costs predictable:

• Use the smallest model that passes your evals. Classification, tagging and extraction often run fine on a provider's small tier, at a fraction of the cost of the flagship.
• Cache. Providers offer prompt caching that discounts repeated prompt prefixes such as long instructions or a shared document. Also cache your own results: the same document should never be summarized twice.
• Trim context. Send the five most relevant chunks, not fifty. Summarize old chat turns instead of resending them.
• Batch non-urgent work. Batch APIs process jobs asynchronously at a discount, often around half price, which suits nightly tagging or bulk extraction.
• Set per-user limits. Quotas and rate limits on your server protect you from abuse and from one heavy user wiping out your margin.

For the bigger picture of what a full build costs, including design, backend and launch, see how much it costs to build an app.

Quality: evals, guardrails and a human in the loop

Models are probabilistic. The same input can produce different output, and a prompt tweak that fixes one case can break three others. You need a way to know whether a change made things better.

1. Build an eval set early. Collect 30 to 100 real examples with the answer you expect. Run every prompt or model change against them. Some checks are exact (did it extract the right total?), others use a second model as a grader against a written rubric.
2. Add guardrails in code. Validate structured output against its schema, reject out-of-range values, cap tool permissions, and block actions the model should never take on its own.
3. Keep a human in the loop for anything consequential. Drafts are reviewed before sending. Payments, deletions and messages to customers need a confirm step. The model proposes; the user approves.
4. Log and review in production. Sample real conversations (with consent and redaction), track thumbs up and down, and feed failures back into the eval set.

Privacy and data handling

Users and enterprise buyers will ask where their data goes. Have answers ready before launch:

• Know your provider's terms. The major API providers state that they don't train on API data by default and keep it only for limited retention periods, with zero-retention options for some customers. Read the current terms and sign a data processing agreement if you handle personal data.
• Send the minimum. Strip names, emails and IDs that the task doesn't need before the prompt leaves your server.
• Respect permissions in retrieval. Filter search results by the user's access rights before they reach the model, not after.
• Disclose it. Say in your privacy policy and in the UI when content is processed by an AI service. For health, finance or children's data, get proper legal advice; on-device or self-hosted models may be the only acceptable route.

UX patterns that make AI features feel good

• Put AI where the work is. A "Summarize" button on the long thread beats a separate chat screen. Inline actions get used; destinations get forgotten.
• Show progress immediately. Stream text, show skeletons, and let users cancel long jobs.
• Make output editable. Drafts go into a normal text field. Extracted fields land in a form the user can correct before saving.
• Show sources. For search and chat answers, link to the documents used so users can verify.
• Let users retry and give feedback. A regenerate button and a thumbs down cost little and give you your best eval data.
• Fail gracefully. Timeouts, provider outages and refusals happen. Fall back to the non-AI path rather than a dead end.

How to start: a realistic first step

Pick one feature, prototype it against real data in a few days, and put it in front of a handful of users. If they use it, harden it: evals, limits, logging, privacy review. If they don't, you've lost a week instead of a quarter. The same logic drives a good MVP, and AI features are a natural fit for that approach.

If you'd like help scoping or building it, our AI app development work covers everything from a single feature in an existing product to AI-first apps on iOS, Android and the web. The fastest way to start is to get a free app plan: describe the job you want AI to do, and we'll tell you honestly whether it's a good fit.