Offline is the normal case
Field hardware has to answer when the link is bad or gone. A round trip to a datacenter is not always on the table.
Edge AI · ModelOps · Policy-gated cloud
Deployable, governed edge AI for real-world device fleets.
Small models run locally when latency, privacy, or offline operation matters. Larger AI work crosses to the cloud only through policy, audit, and approval controls. ByteHub is the model lifecycle and policy layer between AI capability and device operations.
The problem
A chat window is the wrong interface for a device that can go dark.
Today, device teams assemble local inference, cloud APIs, scripts, and OTA tooling themselves — until a failed update, a data-policy block, or a field incident makes AI unusable in production.
The data cannot leave
Logs, video, and firmware artifacts often cannot leave the site. Cloud-only AI gets blocked in procurement before a pilot starts.
Updates need proof, not prose
A model that recommends a firmware action is not enough. Production systems need signed artifacts, policy checks, staged rollout, observability, and a tested rollback path. For covered products, the EU Cyber Resilience Act raises the bar on secure vulnerability handling and lifecycle updates.
What operators see
Model releases managed like firmware.
Every device reports which signed model artifact it runs, under which execution profile and policy, and whether a verified rollback target exists. The table below is illustrative of the control plane view — not measured fleet data.
| Device | Silicon | Profile | Model artifact | Signature | Policy | Rollback |
|---|---|---|---|---|---|---|
| insp-line-a-07 | i.MX95 · Neutron NPU | imx95-npu-int4 | vlm-inspect 1.4.2 | verified | local-only | 1.4.1 ready |
| insp-line-a-08 | i.MX95 · Neutron NPU | imx95-npu-int4 | vlm-inspect 1.4.2 | verified | local-only | 1.4.1 ready |
| gw-plant-east | Jetson Orin NX 16GB | orin-trt-int4 | fleet-copilot 0.9.0 | verified | escalate: metadata only | 0.8.3 ready |
| gw-plant-west | Jetson Orin NX 16GB | orin-trt-int4 | fleet-copilot 0.9.0 → staged 0.9.1 | staging 20% | escalate: metadata only | 0.9.0 ready |
| bench-thor-01 | Jetson Thor | thor-trt-fp8 | vlm-inspect 1.4.2 (bench) | verified | local-only | 1.4.1 ready |
| hmi-android-02 | Snapdragon · Hexagon NPU | android-npu-int4 (planned) | — | not deployed | — | — |
Illustrative. ByteHub publishes measured latency, memory, power, and rollback data only after it is measured on our bench and validated on a customer workflow.
What we build
Small work stays on the chip. Big work goes through a gate.
Product infrastructure for that split — not a one-off integration project.
01
EdgeRuntime
Quantized local inference for low latency, privacy, offline operation, and device-specific execution profiles.
02
ModelOps control plane
Build, evaluate, sign, deploy, observe, and roll back model artifacts like firmware.
03
Policy-gated cloud intelligence
Escalate only approved, minimized context to a frontier model when the task exceeds the device's capability.
04
Business sandbox
Tenant-isolated context, tools, and audit boundaries for each customer deployment.
How it works
Policy picks the plane. The model does not.
Build / quantize
Package for the target device: one signed artifact format, per-SoC execution profile.
Evaluate
Benchmark on real hardware first: accuracy, latency, memory, power, failure recovery.
Sign
Sign the artifact and its metadata: compatibility rules, policy, evaluation result.
Verify / run
Check the signature on device. Stage the rollout. Keep the previous artifact ready.
Edge plane · small scope
On-device runtime
- Local perception, short prompts, control loops
- Purpose-trained specialist models: an open base model under ~1B parameters, distilled to the device’s one job
- Offline continuity on the silicon already in the product
- Execution profiles per SoC: NPU, GPU, CPU fallback
⏻
policy gate
default-off
metadata only
append-only audit
default-off
metadata only
append-only audit
Cloud plane · large scope
Authorized frontier escalation
- Long-context reasoning, multimodal analysis, multi-step work
- Minimized, approved context only; every crossing logged
- Provider-aware: Vertex AI, Amazon Bedrock, Microsoft Foundry
Specialized on-device models
The model on the device is trained for that device’s job.
A frontier model teaches; a small open model learns one narrow skill; the device runs it offline and escalates only what the judge rejects. This is how ByteHub builds the model inside the signed artifact — camera event understanding, inspection explanation, fleet-operations copilots, and other single-job specialists.
Silicon ladder
One runtime across the edge silicon industrial products ship.
One signed artifact format, per-SoC execution profiles.
| Target | Status | Role | Stack we build on |
|---|---|---|---|
| i.MX95-class | in hand · validating | Low-power industrial and automotive-grade compute. Integrated NPU for quantized inference, CPU fallback for unsupported operators, offline-first behaviour. | NXP eIQ, Neutron NPU, ONNX Runtime, Yocto BSP |
| Jetson Orin | in hand · validating | GPU-accelerated vision, multimodal, and language workloads within a defined power and thermal envelope. | JetPack, CUDA, TensorRT / TensorRT Edge-LLM, JetPack OTA |
| Jetson Thor | in hand · validating | Higher-capability edge compute for products that need larger local multimodal or language-model capacity: physical AI, robotics, multi-camera inspection. | JetPack 7, TensorRT Edge-LLM, same signed artifact format, new execution profile |
| Android devices | planned | Device runtime for rugged handhelds, operator terminals, and Android-based HMIs that sit next to the fleet — same signed artifact, same policy gate, NPU-accelerated where the SoC allows. | Android NNAPI / LiteRT, Qualcomm AI Engine Direct, ExecuTorch |
Initial beachhead
We start with the team responsible when a device release fails.
Land with firmware and fleet operations; expand into every AI workload that must be deployed and governed on the same device fleet.
Primary
Private on-prem inspection AI
For industrial operators and equipment makers running vision-language inspection within the plant. Images and local operational context remain on site by default; the appliance keeps operating when connectivity is degraded or unavailable. Cloud is used only for policy-authorized evaluation, model improvement, or deeper analysis when customer controls allow it.
Durable layer
Firmware and fleet operations copilot
Grounded diagnosis across Yocto/BitBake logs, RAUC or SWUpdate outcomes, device telemetry, and engineering runbooks. Model releases are managed with the same signed-delivery, staged-rollout, and rollback discipline as firmware.
Roadmap
Proof before performance claims.
Every edge AI vendor quotes tokens per second. Almost none can show what happens when a model release fails on a device in the field. Our proof standard is different: measured on our own bench, then validated on a customer workflow — and only then published.
For each execution profile we measure accuracy on the customer's task set, time-to-first-token and sustained throughput under thermal load, resident memory including KV cache, board-level power, and rollback time under fault injection. The result is a compatibility matrix a buyer can plan against and an investor can diligence — not a benchmark slide.
- i.MX95-class, Jetson Orin, and Jetson Thor hardware on the bench today
- Signed artifact, policy gate, and rollback architecture defined
- First measured baselines land with the Q4 2026 prototype
Q4 2026
Architecture lock + EdgeRuntime prototype
Quantized inference on i.MX95-class, Jetson Orin, and Jetson Thor hardware; signed artifact format; per-target execution profiles; local policy gates; baseline latency, power, memory, and rollback measurements.
Q1 2027
Authorized frontier-cloud connection
Controlled escalation for work that exceeds the edge envelope: policy routing, context minimization, audit trails, unit-economics measurement.
Q3 2027
Design-partner validation
An early on-prem inspection or firmware/fleet-operations pilot on real hardware, with signed model artifacts, staged rollout, audit trails, and human approval for consequential actions.
Research
Why now: edge models are useful, hybrid is necessary, releases are risk.
01
Edge models are now useful
1–9B models at INT4 run on industrial NPUs and GPUs with usable quality. The model is no longer the blocker; deployment is.
02
Frontier models create a hybrid necessity
Devices need local autonomy but still benefit from cloud reasoning. The missing layer is policy-controlled routing and auditable context boundaries.
03
Model releases are becoming production risk
As AI reaches regulated, operationally critical products, model artifacts need the same release discipline as firmware: compatibility, signing, staged rollout, observability, rollback.
Design partners · investors · cloud partners
ByteHub AI makes edge AI deployable, governable, and safe to operate.
We are selecting 2–3 industrial Linux-device design partners and raising a pre-seed round to prove a repeatable operating layer for governed edge AI fleets.