"Edge or cloud?" is usually the wrong question. Most production systems use both, and the useful decision is which parts of the pipeline run where. This guide compares the two, lists the questions that settle the choice, and describes hybrid patterns that work in practice.
Edge and cloud compared
| Factor | Edge AI (on the device or site) | Cloud AI (remote servers) |
|---|---|---|
| Response time | Local, with no network round trip | Adds network latency and depends on link quality |
| Without a connection | Keeps working | Stops, or falls back to something simpler |
| Data movement | Raw data stays local; results are sent | Raw data must be uploaded |
| Compute ceiling | Fixed by the device's power and cost budget | Scales up on demand |
| Model size | Must be compressed to fit | Large models are practical |
| Updating models | Needs a fleet update mechanism | Change once on the server |
| Cost profile | Hardware up front per device, low running cost | Little up front, ongoing usage charges |
| Privacy exposure | Less personal data leaves the site | More data in transit and in central storage |
| Monitoring | Harder; devices must report back | Centralised logs and metrics |
Five questions that usually decide it
- What happens when the link drops? If the answer is "a safety function stops" or "the machine can't work", that function belongs at the edge. The 2024 Regional Telecommunications Review noted ongoing connectivity challenges in regional and remote Australia, so don't design for the best day's coverage.
- How fast must the system act? Machine control and safety alerts need local inference. A daily report doesn't.
- How much data does each sensor produce? Continuous video from many cameras is expensive to move; occasional photos or text are not.
- Is the data personal or commercially sensitive? Processing locally and sending results limits exposure. See the on-device AI and privacy guide.
- How often will the model change, and how big is it? Models that change weekly or need very large compute are easier to run centrally. Stable models that compress well suit the edge.
Hybrid patterns that work
Train in the cloud, infer at the edge
The most common split. AWS describes it in its IoT Greengrass documentation: models trained in the cloud are deployed to edge devices, giving the low latency and cost savings of local inference while still using cloud computing power for training and complex processing. Most edge platforms support some version of this.
The edge guides, the cloud decides
A small model on the device handles the real-time part, and a larger model on a server does the heavy analysis. DiUS used this split for insurer bolttech: according to its case study, a MobileNetV3 model under 1 MB ran on the customer's phone to guide them through taking photos, and the captured photos were then analysed by a backend model to check that the phone's screen was uncracked.
Send results, not raw data
The device sends detections, counts and short event clips instead of continuous streams. Spiral Blue applies this in orbit: its Space Edge computers process imagery on board and downlink results rather than raw pixels. On the ground, the same idea keeps mobile and satellite data bills under control.
Escalate the hard cases
The edge model handles confident cases on its own and queues uncertain ones for a larger model or a human reviewer when a connection is available. That keeps the device model small without forcing it to be right about everything.
Close the loop with fleet learning
Devices flag inputs where the model struggled. Those examples are reviewed, labelled and used to retrain centrally, and a new compressed model is tested and pushed back to the fleet. This loop is how a deployed model keeps improving, and it needs a secure update path and an evaluation gate before every release.
On-premises: the middle option
Between the device and the public cloud sits a server you control on site or in your own data centre. It can run larger models than a device while keeping data inside your network. Vedametric, for example, deploys either to the device or to the server room, and builds private inference where nothing leaves the client's network.
Thinking about cost
There's no general answer to whether edge or cloud is cheaper, and a firm figure offered without your numbers is a guess. Build a simple model over the expected life of the system:
- Edge costs: hardware per device, enclosures and installation, power, maintenance visits, spares, and the one-off engineering to compress the model and build or integrate the hardware.
- Cloud costs: compute or per-request charges, storage, data transfer, the connectivity plan at each site, and growth in all of these as usage rises.
- Shared costs: data collection and labelling, training, monitoring, security and the people who look after the system.
As a rule of thumb, systems that process continuous streams around the clock lean towards the edge, because cloud charges grow with every hour of data. Systems used occasionally, or by a handful of people, lean towards the cloud, because idle edge hardware still had to be bought. The number of devices and their service life decide where the crossover sits.
Mistakes to avoid when splitting edge and cloud
- Designing for the best day's connectivity. Test what happens on the worst day, and for how long the device can buffer results.
- Forgetting the update path. An edge model with no secure way to update it is frozen on day one.
- Sending everything "just in case". Uploading raw data by default brings back the bandwidth, cost and privacy problems the edge was meant to solve.
- Testing only one side. Evaluate the edge model, the cloud model and the hand-off between them, including what happens when they disagree.
When the cloud is the better choice
- You're still working out what the model needs to do, and it changes weekly.
- The model is too large to compress without unacceptable accuracy loss.
- Connectivity is reliable and latency isn't critical.
- The data isn't sensitive, or it already lives in the cloud.
- You have a few users rather than a fleet of devices.
When the edge is the better choice
- The system must keep working without a connection.
- Decisions are needed in milliseconds.
- Sensors generate more data than the link can carry economically.
- Raw data is personal or commercially sensitive.
- You're deploying many identical devices over a long service life.
If the answer is the edge, the hardware guide and the compression guide cover the next decisions, and the directory lists Australian firms that build both sides.
FAQ
- Is edge AI cheaper than cloud AI?
- Sometimes. Edge AI costs more up front for hardware and engineering but has low running costs; cloud AI is cheap to start, but charges grow with usage and data volume. Continuous, high-volume workloads across many devices tend to favour the edge, while occasional or low-volume use tends to favour the cloud.
- What is a hybrid edge-cloud AI architecture?
- A design where some processing runs on the device and some on remote servers. Common examples are training in the cloud and running inference on the device, or running a small real-time model on a phone and a larger model on a server, as DiUS did for bolttech.
- Can edge AI run without any cloud at all?
- Yes. Models can be trained on local servers and deployed offline, with updates installed over a private network or by hand. Fully offline set-ups are common where data must stay on site or there is no internet link, although monitoring and updates take more work.
- Is on-premises AI the same as edge AI?
- Not quite. Edge usually means the device or machine itself, while on-premises means servers you control at your own site. Both keep data out of public cloud services, and many systems use both: small models on devices and larger models on a local server.