Gains:
- Ability to establish a secure cloud LLM architecture that does not keep the API key on the client but goes through a back-end proxy
- Ability to write robust integrations that increase the perceived speed with streaming and gently handle situations such as timeouts, network errors and speed limits
- Ability to reduce the cost by shortening the token sent and question the necessity of personal data before it goes to the cloud
On-device AI is powerful but limited. When you want to add a truly “smart chat assistant,” long text summarization, or complex creative production to an app, you need models that are too big to fit on a phone. This is where cloud AI comes into play: your application connects to a large language model (LLM) via an API (Application Programming Interface – the standard interface where two software send and receive data to each other). In this unit we will learn how to integrate cloud LLM into a mobile application in a safe, fast and cost-conscious manner. The critical emphasis will be on security: an incorrectly installed LLM integration could leak your API key and result in bills worth thousands of pounds.
The golden rule of architecture: keep the key on the client
The most dangerous mistake that can be made in cloud AI integration is to embed the API key (the secret password that authorizes using the service) directly into the mobile application code. Mobile applications are downloaded to the user's device and the code can be read by reverse engineering — parsing the compiled application and seeing what's inside it. If your key is inside the app, someone can extract it and make unlimited requests from your account.
The correct architecture is this: the mobile application sends requests to your own backend server (the proxy server you control); The key resides only on the server; The server goes to the LLM service and returns the response to the application. This middleware also provides speed capping, abuse prevention, and cost control.
Approach
where is the key
Security
The key is in the application (FALSE)
In client, public
It leaks, the bill explodes
The key is in the backend (TRUE)
On the server, hidden
Safe, controllable
Caution: When you ask AI for cloud LLM integration, it may produce an example that writes the key directly into the application code for your convenience. Never take this live. Be sure to include the sentence "The API key should not be on the client, go through the backend proxy" in the prompt.
Streaming: increasing the perceived speed
LLM answers can be long and take seconds to produce in their entirety. Leaving the user waiting on a blank screen is a bad experience. The solution is streaming — displaying the answer word by word, as it is generated. The user monitors the spelling of the text, as in ChatGPT; this dramatically increases perceived speed and fluency. Flow on mobile means adding pieces (tokens — the piece of text produced by the model) from the server to the interface as they arrive. Explicitly request the flow when printing integration to AI.
Tip: Add a "pause" button in the streaming response. The user should be able to stop production when he gets the answer he wants; This both improves the experience and reduces cost by cutting unnecessary token generation. In the middle of the long answer, the user may have already found their answer.
Cost, delay and error management
Cloud LLM carries money cost (fee per token) and time cost (latency) with each request. Three disciplines are essential. Cost: limit prompt and response length, do not send unnecessarily long system instructions, default to small and cheap model if possible. Latency: use streaming, set timeout, notify user if network is slow. Error: network outage, service may return 429 (too many requests) or 500 (server error); handle each one gently, don't crash the app. Also, LLM sometimes gives meaningless or incorrect (hallucination) answers; Add a layer of verification of the answer in critical areas.
three mini cases
Case 1 — Leaked key. A startup embedded the OpenAI key directly into its React Native app to get out fast. Three weeks after the app was released, the key was reverse engineered and $2,400 worth of usage was made overnight. The team had to revoke the key and set up a backend proxy. Lesson: the shortcut taken for convenience became the most expensive route.
Case 2 — Dropout decreased with flow. An education app first released its Q&A feature without streaming; users were exiting after 6 seconds of idle waiting. When flow was added, the first word started appearing in 0.8 seconds, and the abandonment rate dropped from 48% to 12%. Same model, same speed — just a difference in presentation.
Case 3 — Cost control. One app was sending the entire chat history to the model with every user message; In long conversations, a single request reached 8,000 tokens, inflating the cost. By sending just the last few messages and a summary, the team reduced tokens per request by 70%, reducing the monthly bill to a third. Lesson: measure what you send.
Weak prompt / Strong prompt
Weak prompt: "Add a chat like ChatGPT to my app."
Powerful prompt: "Add a chat assistant to my iOS/Swift application. Architecture: the application sends a request to my own backend, the LLM API key is NOT on the CLIENT, it goes through the proxy. - The response comes streaming, displayed word by word - The 'Stop' button interrupts the production - Handle timeout, network error, 429 and 500 situations gracefully - Shorten the chat history: send the last 6 messages + summary (cost control)Explain the architectural diagram first, then give the client and proxy code separately."
Copiable templates
Secure architecture template:"Design cloud LLM integration into my [platform] application. Rule: API key only in backend. Client -> my proxy -> LLM. In proxy: authentication, per-user rate limit, request logging. List client and proxy responsibilities separately, then export the code."
Streaming template: "Add a streaming response to this chat screen:- Add snippets to the message bubble as they arrive- Show a cursor/animation while typing- Have 'Stop' button cancel the stream- Preserve partial text and warn if there is an error while the stream is ending[existing code]"
Cost-latency template:"Reduce cost and latency in this LLM integration:- How do I reduce token sent (history abbreviation, summary)?- In which case smaller/cheaper model is enough?- Suggest timeout and retry strategy[code]"
Fault tolerance template: "Make this LLM call resilient:- Separate behavior for no network, timeout, 429 (rate limit), 500 (server)- Non-technical, polite message to user- Verification note against risk of hallucination in critical replies[code]"
Common mistakes
- Embedding the API key into the application. The most expensive and common security bug; The key definitely lies at the back end.
- Not using flow. Leaving the user waiting for long answers will drive the user away.
- Sending the entire chat history with every request. It multiplies token cost and latency.
- Bypassing error conditions. If 429/500/timeout is not addressed the application will crash or freeze.
- Considering the LLM answer as correct without question. The hallucination is real; Add verification layer in critical area.
- Sending user data to unnecessary LLM. Ask whether personal data is required or should be masked before it goes to the cloud.
In summary
Cloud LLM brings great capabilities that do not fit on the device to mobile, but requires security and cost discipline. Golden rule: The API key is never on the client, it goes through the backend proxy. Flow greatly increases perceived speed and retention; Supported by "stop" button. The cost is determined by shortening the token sent; Resilience is achieved by handling all error cases gracefully. LLM answers may include hallucinations; In critical areas, verification is essential and personal data is reviewed before sending it to the cloud.
Application task
Request a client + backend proxy design from the AI using the “Secure architecture template” for a “text summarization” or “chat” feature. Verify that the API key only resides in the backend in the generated design. Then extract at least two ways to reduce the token sent with the "Cost-delay pattern" and write the polite message to be displayed to the user for an error condition (e.g. 429).
checklist
- [ ] I verified that the API key resides in the backend and not on the client
- [ ] I made the response streaming and added a 'pause' button
- [ ] I handled timeout, network error, 429 and 500 situations
- [ ] I reduced the submitted token with the past abbreviation/summary
- [ ] I considered validation against the risk of hallucinations in the LLM answer
- [ ] I checked the necessity/masking of personal data before going to the cloud