An on-premise computer vision deployment has five layers: camera ingest, a GPU inference node, a rules layer that turns detections into events, an alerting and storage layer, and an operator console. All five run inside the site's own network boundary, and the only thing that ever needs to leave it is an event summary, never raw video. That constraint is what makes the architecture different from a cloud CV product, and it is the whole reason the design looks the way it does.
The forcing function is bandwidth and regulation together. A single 1080p camera stream is a continuous outbound flow; a site with fifty of them cannot ship raw footage to an external API at all, and in most regulated environments it would not be permitted to even if the uplink existed. So inference moves to the footage rather than the footage moving to inference.
Everything below assumes cameras that already exist. Replacing a camera estate is a capital project with its own approval cycle, and a reference architecture that starts by requiring one is a reference architecture nobody deploys.
Layer 1: camera ingest
The ingest layer reads the streams the existing cameras already publish, normally RTSP or ONVIF from the recorder or the cameras directly, and decodes them into frames. It runs on the site network, alongside the recorder rather than in place of it, so the existing recording and retention policy stays exactly as the compliance function already approved it.
Two decisions matter here. The first is whether to decode on CPU or on the GPU: GPU decode keeps the frames on the accelerator and removes a copy, which is what allows one node to carry more streams. The second is frame sampling. Most detection tasks do not need every frame of every camera, and sampling per camera by rule is the single largest lever on how much hardware a site needs.
Layer 2: GPU inference
The inference layer runs the detection models on a GPU node inside the building. One node serves many cameras, because the models are batched across streams rather than pinned one-per-camera. This is the layer that decides the hardware bill, and it is the layer to size against the rules a site actually enforces rather than against its camera count.
Model choice belongs here too. Object detection for people, vehicles, and equipment is one model family; number plate recognition is another; pose and interaction detection is a third. A deployment loads only what its rules reference, which keeps the node's memory footprint tied to the work being asked of it.
Layer 3: the rules layer
Detections are not events. A person detected in a walkway is routine; the same person detected inside a machine guard while the machine is running is an event. The rules layer holds the site's own definitions - zones drawn on each camera, which roles may enter which zone, which shifts a rule applies on, and which equipment must be present - and it evaluates every detection against them.
Keeping the rules separate from the models is what makes a deployment configurable without retraining. A safety officer redrawing a zone or changing a shift window is editing configuration, not commissioning a model.
Layer 4: alerting and the audit record
Confirmed events go two places at once. They raise an alert to the operator responsible for that area, on the channel that area already uses, and they write a record: the frame, the timestamp, the camera, the rule that fired, and the outcome. The record is the part a regulator or an internal investigation actually asks for, and it is why storage sits inside the boundary alongside inference rather than in an external log service.
Retention on that record is a policy setting, not a product default. Footage retention is regulated in most of the jurisdictions this architecture is deployed in, and the event store has to inherit the site's existing policy rather than invent a second one.
Layer 5: the operator console
The console is where a person watches live feeds, reviews an event with its triggering frame, and closes it out. For multi-site operators it aggregates across sites while each site keeps its own cameras, rules, and access boundary, so a regional supervisor sees the estate without any site's footage leaving that site.
Where the boundary sits
Three deployment surfaces satisfy the same architecture. On-premise puts every layer on hardware inside the building. A private cloud tenant puts them in a cloud subscription the customer owns, which suits operators who have already centralised infrastructure and want one tenant across sites. An edge node runs the whole stack on a single self-contained appliance next to the cameras, for remote sites with a thin uplink or none at all.
The layers do not change between them. What changes is who holds the hardware and where the network boundary is drawn, which is exactly the decision a compliance function wants to make explicitly rather than inherit from a vendor's default.
See how this shows up in practice: VMS + Omniscient