Gains:
- Ability to decide on-device or cloud and choose the right tool (ML Kit, Core ML, TensorFlow Lite) based on privacy, offline need, model size and battery criteria
- Ability to prevent silent errors by verifying input preprocessing (size and normalization) from the model's document in model integration
- Ability to evaluate the confidence score and measure the result with user approval and on the real device, without presenting low confidence predictions as absolute truth.
So far, we have used AI as an aid to speed up the development process. Now we move on to the second role of AI: the talent embedded in the application. Modern phones have the power to run AI models such as image recognition, text translation, speech transcription, etc. directly on the device (on-device — in the phone's own processor without going to the server). AI on the device; It offers great advantages over cloud solutions in terms of speed, privacy and offline operation. In this unit, we will learn how to embed AI into the application with iOS's Core ML, cross-platform TensorFlow Lite (now known as LiteRT) and Google's ready-made solution ML Kit, and how to use AI as an assistant in this integration.
On-device or cloud?
This is the first and most important architectural decision. On-device AI does not remove data from the phone — a huge win for privacy. It is also instantaneous and works offline as there is no network latency. However, it is limited by the device's processing power and memory; Very large models (e.g. giant tongue models) will not fit in the phone or will drain the battery. Cloud AI, on the other hand, offers unlimited power, but sends data to the server, requires network and creates latency.
criterion
On-device
Cloud (cloud API)
Privacy
Data stays on device, strong
Data goes to server, attention needed
speed
Instant, no network
Depends on network latency
offline
It works
Does not work
Model size
Limited (phone resource)
unlimited
battery/heat
Effects with heavy use
Server under load, device relaxed
Cost
Free (device source)
Fee per use
Decision rule: Choose on-device if personal/sensitive data is being processed, needs to work offline, or instant response is essential. If you need a very large model, turn to the cloud. This unit is on-device focused; We will cover cloud AI in the next unit.
Tip: Always make on-device your default for a feature that handles sensitive data (health, biometrics, location). The phrase “data does not leave the device” is invaluable for both privacy compliance and user trust, and makes a big difference in the store privacy label.
Three ways: ML Kit, Core ML, TensorFlow Lite
ML Kit (Google) is the easiest way to start: it gives ready-made capabilities such as text recognition (OCR — reading text in an image), face detection, barcode reading, translation in a few lines. You don't need to train your own model. Core ML (Apple) is the most efficient way to run your own model or a ready-made model on iOS; It uses Apple's Neural Engine (artificial neural network processor) hardware. TensorFlow Lite/LiteRT is a cross-platform solution that allows you to run your own trained model on both Android and iOS.
The general integration flow with AI goes like this:
- Talent definition. A clear goal, such as "I want to read the text in the photo."
- Path selection. If there is ready talent, ML Kit; Core ML/TF Lite if special model available.
- Model format. .mlmodel (Core ML), .tflite (TF Lite). Explains the AI transformation steps.
- Integration code. Loading the model, preprocessing the input, interpreting the output.
- Performance test. Speed, memory, battery measurement on real device.
Caution: The most common AI mistake in on-device model integration is input preprocessing — converting the image to the size and color format the model expects. If the model expects 224x224 pixels and you give it 300x300, the result will be meaningless, but you will not receive an error message. Verify the preprocessing values from the model's document.
Knowing the limits of the model
An on-device model makes decisions based on the data it was trained on. An object recognition model trained only on photos taken during the day will be wrong on night images. The model has a confidence score (confidence — how confident the model is about its answer, usually between 0 and 1); It is dangerous to present low-confidence results to the user as accurate. For example, a skin spot scanning application should not say "definitely benign", but should say "the model's prediction is this, please consult a physician". The model result is a recommendation, not a diagnosis.
three mini cases
Case 1 — Acceleration with OCR. An expense tracking app has removed the burden of manually entering receipts with ML Kit text recognition. The user takes a photo of the receipt, and the amount and date are filled in automatically. Manual entry time decreased from 40 seconds to 8 seconds per receipt. The team always had the user confirm the amount the AI read; because wrinkled receipts had a 6% margin of error. Automation + human approval was the right balance.
Case 2 — Preprocessing error. One team integrated a plant recognition model with TensorFlow Lite; On the tester, the results were random. The problem was that the code the AI generated did not normalize the image to the [0,1] range the model expected (pixel values were left at 0-255). When normalization was added, accuracy increased from 30% to 89%. Lesson: preprocessing is silent but deadly.
Case 3 — Privacy gain. A health application detected anomaly from heart rate data with the on-device Core ML model. The data never went to the server. This choice enabled the application to receive the phrase "does not collect data" in the App Store privacy label and increased its download rate compared to competitors. The on-device choice was both ethical and commercially profitable.
Weak prompt / Strong prompt
Weak prompt: "Add image recognition to my app."
Powerful prompt: "Add the amount and date reading feature to my Android/Kotlin application. - Use Google ML Kit Text Recognition (on-device, offline) - Take image from camera or gallery - Extract the amount and date from the recognized text with regex - Present the result to the user for approval in the EDITABLE field, auto-save - Handle camera permission flow and rejection. Write down pre-processing and error situations, explain the steps."
Copiable templates
Path selection template: "I want to make the following feature: [feature]. Should it be on-device or cloud? Compare based on: privacy, offline need, model size, battery, cost. Recommend the appropriate tool (ML Kit / Core ML / TF Lite) and justify."
Integration template:"Write [model/capability] integration for [platform]:1) Model loading2) Input preprocessing (expected size and normalization)3) Inference call4) Output interpretation and confidence score checking5) Warning to user on low-confidence resultRemind me to verify the preprocessing values from the model's documentation."
Confidence score template:"Consider confidence score in this inference code:- Present result 'exact' below threshold (e.g. 0.6)- Show 'this is an estimate' note to user- Refer to expert if critical area (health, safety)[code]"
Performance verification template: "List the metrics I need to measure on the actual device for this on-device model integration: inference time, memory increase, battery impact, heating. Tell the measurement method for each."
Common mistakes
- Skipping preprocessing or doing it incorrectly. Wrong size/normalization silently produces wrong result.
- Ignoring the confidence score. Presenting a low-confidence estimate as accurate will mislead the user.
- Testing the model in the emulator. Actual device speed and battery are very different; always measure on real hardware.
- Sending sensitive data to the cloud unnecessarily. Choosing cloud when on-device is possible is a privacy risk.
- Ignoring model size. Large model apps inflate download size and crash on low hardware.
- Forgetting the training limit of the model. The model is mistaken in the condition where it does not see (night, different language); Make this clear to the user.
In summary
On-device AI provides privacy, speed and offline operation by keeping data on the phone; The limit is the device power and model size. ML Kit is used for out-of-the-box capabilities, Core ML (iOS) and TensorFlow Lite (cross-platform) are used for custom models. The silent killer of integration is improper preprocessing; The input size and normalization are verified from the model's documentation. Each result comes with a confidence score, and low confidence predictions are not presented as absolute truth. Decisions are measured on the real device, not the emulator.
Application task
For a “text reading from photo” or “barcode reading” feature, ask the AI whether it should be on-device or cloud with the “Path selection template”, then ask for an ML Kit-based blueprint with the “Integration template”. Verify that the preprocessing step and user approval/edit flow are present in the code. Set a trust score threshold and write what you will do if the result is low trust.
checklist
- [ ] I made the on-device/cloud decision based on criteria
- [ ] I chose the right tool (ML Kit / Core ML / TF Lite)
- [ ] I verified the preprocessing dimension and normalization from the model's documentation
- [ ] I checked the trust score and warned of low trust results
- [ ] I presented the result to the user with approval/edit, I did not blindly save it
- [ ] I measured performance on the real device, not the emulator