Frigate could not see the iGPU inside WSL2, so we gave it a detector over ZMQ

Inside the Frigate container, OpenVINO lists one device: CPU. In a stock pip install openvino on the WSL host, outside the container, it lists ['CPU', 'GPU']. Everything below follows from that gap: the design, the protocol as we observed Frigate 0.17.2 speaking it, the full ov_zmq_server.py, the systemd unit, the Frigate detectors: block, and the numbers from 3.41 million inferences (as of 2026-09-03). If the question is whether you need a Coral or a GPU at all, the detector entry page answers it. The WSL2 plumbing this sits on is the Windows setup files.

The problem

WSL2 gives Linux a paravirtualised GPU through /dev/dxg and a set of Microsoft libraries in /usr/lib/wsl/lib (libd3d12.so, libdxcore.so). There is no /dev/dri. Frigate's container image ships OpenVINO and expects to enumerate the iGPU the ordinary Linux way, through /dev/dri/renderD128 and the Intel compute runtime. We passed /dev/dxg and the WSL library directory into the container (see docker-compose.yml); the OpenVINO inside Frigate still reported only a CPU device. Frigate ran detection on the CPU from the day it moved to this machine until 2026-08-29, and the CPU detector competed with the CPU decode of eleven streams. It worked, but it was not free.

Outside the container, in a plain Python 3.12 virtualenv on the WSL host with pip install openvino, ov.Core().available_devices returns ['CPU', 'GPU'], and the GPU is reported as Intel(R) Graphics [0x3e92] (iGPU). That is the UHD 630 in the i7-8700, reachable from the host and not from where Frigate was looking.

The design

Frigate 0.17 added a detector type called zmq (frigate/detectors/plugins/zmq_ipc.py; it is not in 0.16): instead of loading a model in-process, Frigate sends each detect frame to a ZeroMQ endpoint and expects a fixed-shape block of detections back. The docs introduce it under the Apple Silicon detector, where the NPU is visible to the host but not to the container, and an official out-of-process client, apple-silicon-detector, answers on the host. WSL2’s /dev/dxg is the same shape of problem for an Intel iGPU, and this page is the OpenVINO server for that case. The container keeps doing what it is good at (decoding, motion, tracking, recording); a separate process that can see the iGPU does the one thing the container cannot.

Frigate (docker, python3 -m frigate)
   |  detect frame, 300x300x3 uint8, one per motion region
   v
tcp://172.17.0.1:5555   (docker bridge gateway = the WSL host)
   |
   v
ov_zmq_server.py  (WSL host, venv312, openvino 2025.3, device GPU, f16)
   |  ZMQ REP: recv_multipart -> infer -> send (20,6) float32
   v
Frigate  ->  tracked objects, events, snapshots, clips

The endpoint address is docker's default bridge gateway. From inside a container on the default bridge network, 172.17.0.1 is the host; the server binds 0.0.0.0:5555 on the host so the bridge can reach it. Under WSL2 mirrored networking that port is also visible on the Windows side, and because this box runs .wslconfig with firewall=false (see the setup page for what that really means), Windows Firewall does not filter it: any device on the LAN can reach the socket, and the router's refusal to forward the port is the only thing keeping it off the internet.

The protocol, as observed from the server side

The request and reply framing is in the docstring of Frigate's zmq_ipc.py, and the Apple Silicon client implements the same exchange; this is what Frigate 0.17.2 sends and what this server answers, as seen from the server side. Read Frigate's own zmq detector source for your version before you trust it; this is what our server handles and Frigate has been happy with for 3.41 million round trips.

The SSDLite MobileNet v2 IR that Frigate bundles emits (1, 1, 100, 7) rows of [image_id, class_id, score, xmin, ymin, xmax, ymax], so the only real work after inference is reordering the box corners into Frigate's [ymin, xmin, ymax, xmax] and keeping the first twenty. Because REP sockets must answer every request, an exception anywhere in the loop sends an all-zero (20, 6) block rather than nothing; Frigate sees "no objects" instead of a timeout.

The Frigate side: config.yml

This is the top of the real config. The model block still points at the copy of the model that ships inside the Frigate image; Frigate uses it to know the input size, layout and labels. The detector never reads it — the host process has its own copy of the same three files.

mqtt:
  enabled: false

detectors:
  ov_gpu:
    type: zmq
    endpoint: tcp://172.17.0.1:5555   # docker's default bridge gateway = the WSL host
    request_timeout_ms: 1000

model:
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

A steady frame takes 8 to 14 ms on the host and about 12.5 ms as Frigate measures it, so the one-second ceiling of request_timeout_ms: 1000 only trips if the server is down, and then Frigate logs and moves on rather than stalling a camera process. The rest of the file, with all eleven cameras, is on the annotated config page.

The server: ov_zmq_server.py

About 170 lines, three dependencies (openvino, numpy, pyzmq), no framework. Everything configurable is an environment variable so start.sh and the unit file stay trivial.

#!/usr/bin/env python3
"""Frigate type:zmq OpenVINO GPU detector. Runs on the WSL host, not inside Frigate."""
from __future__ import annotations

import json
import os
import sys
import time
import traceback

import numpy as np
import openvino as ov
import zmq

BIND = os.environ.get("OV_ZMQ_BIND", "tcp://0.0.0.0:5555")
MODEL = os.environ.get(
    "OV_MODEL",
    "/mnt/d/frigate/ov-scratch/openvino-model/ssdlite_mobilenet_v2.xml",
)
DEVICE = os.environ.get("OV_DEVICE", "GPU")
READY_PATH = os.environ.get("OV_READY_PATH", "/mnt/d/frigate/ov-detector/ready")
STATS_PATH = os.environ.get("OV_STATS_PATH", "/mnt/d/frigate/ov-detector/stats.json")


def load_compiled():
    core = ov.Core()
    print("openvino", ov.__version__, "devices", core.available_devices, flush=True)
    if DEVICE not in core.available_devices and DEVICE != "AUTO":
        raise RuntimeError(f"{DEVICE} not in {core.available_devices}")
    model = core.read_model(MODEL)
    configs = [
        {"INFERENCE_PRECISION_HINT": "f16"},
        {"INFERENCE_PRECISION_HINT": "f32"},
        {},
    ]
    last = None
    for cfg in configs:
        try:
            t0 = time.time()
            compiled = core.compile_model(model, DEVICE, cfg)
            print(f"compiled {DEVICE} {cfg} in {time.time()-t0:.2f}s", flush=True)
            return compiled
        except Exception as exc:
            last = exc
            print(f"compile failed {cfg}: {exc}", flush=True)
    raise last


def postprocess_ssd(raw, max_dets=20):
    dets = np.zeros((max_dets, 6), np.float32)
    arr = np.asarray(raw)
    if arr.ndim == 4:
        rows = arr[0][0]
    elif arr.ndim == 3:
        rows = arr[0]
    elif arr.ndim == 2:
        rows = arr
    else:
        return dets
    for i, row in enumerate(rows):
        if i >= max_dets:
            break
        # SSDLite: [image_id, class_id, score, xmin, ymin, xmax, ymax]
        if row.size < 7:
            continue
        _, class_id, score, xmin, ymin, xmax, ymax = row[:7]
        dets[i] = [class_id, float(score), ymin, xmin, ymax, xmax]
    return dets


def handle_model_request(compiled, header):
    name = header.get("model_name", "")
    expected = os.path.basename(MODEL)
    ok = (not name) or (name == expected) or name.endswith(".xml")
    body = {
        "model_available": bool(ok and compiled is not None),
        "model_loaded": bool(ok and compiled is not None),
    }
    print(f"model_request name={name} ok={ok}", flush=True)
    return [json.dumps(body).encode("utf-8")]


def infer(compiled, header, payload):
    shape = header.get("shape")
    dtype = np.dtype(header.get("dtype", "uint8"))
    tensor = np.frombuffer(payload, dtype=dtype)
    if shape:
        tensor = tensor.reshape(shape)
    inp = compiled.inputs[0]
    if inp.element_type == ov.Type.f32 and tensor.dtype != np.float32:
        tensor = tensor.astype(np.float32)
    req = compiled.create_infer_request()
    key = inp.get_any_name() or inp
    req.infer({key: tensor})
    out = req.get_output_tensor(0).data
    dets = postprocess_ssd(out)
    return [dets.tobytes(order="C")]


def write_stats(n_infer, n_model, last_ms):
    tmp = STATS_PATH + ".tmp"
    with open(tmp, "w", encoding="utf-8") as f:
        json.dump(
            {
                "infer": n_infer,
                "model": n_model,
                "last_ms": last_ms,
                "ts": time.time(),
            },
            f,
        )
    os.replace(tmp, STATS_PATH)


def main():
    print(f"loading {MODEL} on {DEVICE}", flush=True)
    compiled = load_compiled()
    inp = compiled.inputs[0]
    dummy_shape = [int(x) if x > 0 else 1 for x in inp.shape]
    dummy = np.zeros(dummy_shape, np.uint8)
    if inp.element_type == ov.Type.f32:
        dummy = dummy.astype(np.float32)
    req = compiled.create_infer_request()
    req.infer({inp.get_any_name() or inp: dummy})
    print("warmup ok", flush=True)

    os.makedirs(os.path.dirname(READY_PATH), exist_ok=True)
    with open(READY_PATH, "w", encoding="utf-8") as f:
        f.write(f"ready {time.time()}\n")
    write_stats(0, 0, 0.0)

    ctx = zmq.Context()
    sock = ctx.socket(zmq.REP)
    sock.bind(BIND)
    print(f"listening {BIND}", flush=True)

    n_infer = 0
    n_model = 0
    last_ms = 0.0
    while True:
        try:
            frames = sock.recv_multipart()
        except Exception:
            traceback.print_exc()
            continue
        try:
            header = json.loads(frames[0].decode("utf-8"))
            if header.get("model_request"):
                n_model += 1
                sock.send_multipart(handle_model_request(compiled, header))
                write_stats(n_infer, n_model, last_ms)
                continue
            if header.get("model_data"):
                sock.send_multipart(
                    [
                        json.dumps(
                            {"model_saved": True, "model_loaded": True}
                        ).encode("utf-8")
                    ]
                )
                continue
            payload = frames[1] if len(frames) > 1 else b""
            t0 = time.time()
            sock.send_multipart(infer(compiled, header, payload))
            last_ms = (time.time() - t0) * 1000
            n_infer += 1
            if n_infer <= 5 or n_infer % 25 == 0:
                print(f"infer #{n_infer} {last_ms:.1f}ms shape={header.get('shape')}", flush=True)
            write_stats(n_infer, n_model, last_ms)
        except Exception:
            traceback.print_exc()
            try:
                sock.send(np.zeros((20, 6), np.float32).tobytes(order="C"))
            except Exception:
                pass


if __name__ == "__main__":
    try:
        main()
    except Exception:
        traceback.print_exc()
        sys.exit(1)

Things worth noticing in that file:

• Precision fallback. It tries INFERENCE_PRECISION_HINT: f16 first, then f32, then the plugin default. On the UHD 630 the f16 compile succeeds in about 3.6 s. f16 is where the iGPU is fast; SSDLite does not need f32 for a cat.
• The input key. inp.get_any_name() or inp — the tensor in this IR is called image_tensor:0; some IRs have no friendly name, in which case the port object itself is the key.
• A fresh infer request per call. Simple and correct for a single-threaded REP loop. If you ever go to a ROUTER/DEALER pattern with concurrency, reuse requests.
• The ready file. Written only after the warm-up inference passes, so anything that wants to wait for "the detector is really up" can wait on the file rather than the port.
• stats.json is written atomically (write to .tmp, os.replace) after every inference. That is 20-ish tiny writes a second to an NTFS drive through 9p; it has not been a problem, but it is the first thing to throttle if you run this on a slow disk.

start.sh, the venv, and the model files

#!/bin/bash
set -euo pipefail
export PYTHONUNBUFFERED=1
ROOT=/mnt/d/frigate/ov-detector
export OV_ZMQ_BIND="${OV_ZMQ_BIND:-tcp://0.0.0.0:5555}"
export OV_MODEL="${OV_MODEL:-/mnt/d/frigate/ov-scratch/openvino-model/ssdlite_mobilenet_v2.xml}"
export OV_DEVICE="${OV_DEVICE:-GPU}"
export OV_READY_PATH="$ROOT/ready"
rm -f "$OV_READY_PATH"
exec "$ROOT/venv312/bin/python" "$ROOT/ov_zmq_server.py"

# inside the WSL distro, as root
# The openvino wheel does not ship Intel's GPU compute runtime. Without the OpenCL ICD the GPU
# plugin stays silent and available_devices prints ['CPU']. Installed here on 08-29, before the venv:
apt-get install -y intel-opencl-icd clinfo   # Ubuntu 26.04: intel-opencl-icd 26.05, pulls libigc2/libigdfcl2/libigdgmm12
clinfo -l
# Platform #0: Intel(R) OpenCL Graphics
#  `-- Device #0: Intel(R) Graphics [0x3e92]

mkdir -p /mnt/d/frigate/ov-detector && cd /mnt/d/frigate/ov-detector
python3.12 -m venv venv312
./venv312/bin/pip install --upgrade pip
./venv312/bin/pip install openvino numpy pyzmq

# does the GPU plugin see the iGPU from here? (it does; the container cannot)
./venv312/bin/python -c "import openvino as ov; c=ov.Core(); print(c.available_devices); print(c.get_property('GPU','FULL_DEVICE_NAME'))"
# ['CPU', 'GPU']
# Intel(R) Graphics [0x3e92] (iGPU)

The model is Frigate's own bundled OpenVINO IR: ssdlite_mobilenet_v2.xml (450 kB), ssdlite_mobilenet_v2.bin (8.9 MB) and coco_91cl_bkgr.txt. Copy them out of a running container with docker cp frigate:/openvino-model /mnt/d/frigate/ov-scratch/openvino-model. Using the identical files on both sides means the label ids in Frigate's labelmap_path line up with what the server returns.

The systemd unit and boot order

[Unit]
Description=Frigate OpenVINO iGPU ZMQ detector
After=network.target

[Service]
Type=simple
WorkingDirectory=/mnt/d/frigate/ov-detector
ExecStart=/mnt/d/frigate/ov-detector/start.sh
Restart=always
RestartSec=3
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target

cp ov-gpu-detector.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now ov-gpu-detector
systemctl is-active ov-gpu-detector      # active
ls /mnt/d/frigate/ov-detector/ready       # exists once warm-up passed

Because the distro boots with systemd=true (see wsl.conf), the unit comes up whenever the distro does, which the Windows-side boot script triggers after every reboot. A scheduled task at system start runs systemctl start ov-gpu-detector as well, so two things try to start it and neither depends on the other.

# Windows, elevated PowerShell. Belt and braces: the unit is enabled, so systemd
# starts it whenever the distro boots; this task just makes sure.
schtasks /Create /TN HelenOvGpuDetector /SC ONSTART /RL HIGHEST /RU <YOUR_USER> /TR "wsl.exe -d Ubuntu -u root -- systemctl start ov-gpu-detector"

Start order matters, in one direction. When Frigate’s detector process starts it sends a single model_request and waits up to 30 seconds for the answer (zmq_ipc.py raises the receive timeout to 30,000 ms for that one exchange). A REQ socket does not report connection refused; the request sits in the queue until something answers. If nothing has answered after 30 seconds, _check_and_transfer_model returns false, _model_ready stays false, and every detect_raw call from then on returns the zero block with a “Model not ready” warning before it touches the socket. Nothing retries. Detection is silently dead until Frigate restarts, and the watchdog does not catch it because the detector is answering, just with zeros. On this box the unit is enabled and Frigate comes up later through Docker, so the server is always listening first; that is an ordering dependency, not the absence of one. Server restarts mid-run are the recoverable case: a frame times out after request_timeout_ms, Frigate resets the socket and re-sends model_request (blocking up to 30 s again), and once the server is back the next frame goes through. With Restart=always and RestartSec=3 that costs a few seconds of zero detections.

# bring the server up outside the 30 s window, then watch detection_fps for two minutes
systemctl stop ov-gpu-detector
docker restart frigate
sleep 90
systemctl start ov-gpu-detector
# now walk in front of a camera and poll; the port is whatever you mapped to 5000 (8971 for the authenticated UI)
for i in $(seq 1 12); do
  curl -s http://127.0.0.1:5000/api/stats | python3 -c "import sys,json; s=json.load(sys.stdin); print({c: v['detection_fps'] for c, v in s['cameras'].items()})"
  sleep 10
done
# detection_fps pinned at 0.0 on every camera with motion present = the dead state; docker restart frigate recovers it.
# Reverse the order (server first, then Frigate) and the same loop shows non-zero within seconds.

This reading is from the v0.17.0 source; the loop above is the test, and the earlier version of this page, which said start order did not matter, was wrong.

What it measures

version 2025.3.0-19807-44526285f24-releases/2025/3
devices ['CPU', 'GPU']
CPU Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
GPU Intel(R) Graphics [0x3e92] (iGPU)
inputs [('image_tensor:0', (1, 300, 300, 3), "<Type: 'uint8_t'>")]
--- GPU {'INFERENCE_PRECISION_HINT': 'f16'}
compiled 3.606672763824463
infer1_ms 32.19342231750488
infer2_ms 12.607812881469727
out (1, 1, 100, 7)
SUCCESS GPU {'INFERENCE_PRECISION_HINT': 'f16'}

$ journalctl -u ov-gpu-detector -n 8 --no-pager -o short-iso
2026-09-02T20:17:21-04:00 <HOSTNAME> start.sh[10631]: infer #3408600 14.3ms shape=[1, 300, 300, 3]
2026-09-02T20:17:22-04:00 <HOSTNAME> start.sh[10631]: infer #3408625 8.4ms shape=[1, 300, 300, 3]
2026-09-02T20:17:24-04:00 <HOSTNAME> start.sh[10631]: infer #3408650 8.1ms shape=[1, 300, 300, 3]
2026-09-02T20:17:25-04:00 <HOSTNAME> start.sh[10631]: infer #3408675 8.3ms shape=[1, 300, 300, 3]
2026-09-02T20:17:27-04:00 <HOSTNAME> start.sh[10631]: infer #3408700 9.9ms shape=[1, 300, 300, 3]
2026-09-02T20:17:29-04:00 <HOSTNAME> start.sh[10631]: infer #3408725 9.0ms shape=[1, 300, 300, 3]
2026-09-02T20:17:31-04:00 <HOSTNAME> start.sh[10631]: infer #3408750 8.4ms shape=[1, 300, 300, 3]
2026-09-02T20:17:33-04:00 <HOSTNAME> start.sh[10631]: infer #3408775 9.6ms shape=[1, 300, 300, 3]

$ cat /mnt/d/frigate/ov-detector/stats.json
{"infer": 3408788, "model": 11, "last_ms": 10.118246078491211, "ts": 1788394653.7592132}

$ curl -s http://127.0.0.1:5000/api/stats | python3 -c "import sys,json; print(json.load(sys.stdin)['detectors'])"
{'ov_gpu': {'inference_speed': 12.57, 'detection_start': 0.0, 'pid': 1073}}

Read the three together: the journal prints every 25th inference, its last line is #3408775, and stats.json written the same second says 3408788, so the two agree to within one journal interval. Frigate's inference_speed is a rolling average of its own round trip, which is why it sits about 3 ms above the host's last_ms.

$ journalctl -u ov-gpu-detector --no-pager -o short-iso | head -4
2026-08-29T20:38:43-04:00 <HOSTNAME> systemd[1]: Started ov-gpu-detector.service - Frigate OpenVINO iGPU ZMQ detector.
2026-08-29T20:38:54-04:00 <HOSTNAME> start.sh[87884]: loading /mnt/d/frigate/ov-scratch/openvino-model/ssdlite_mobilenet_v2.xml on GPU
2026-08-29T20:38:56-04:00 <HOSTNAME> start.sh[87884]: openvino 2025.3.0-19807-44526285f24-releases/2025/3 devices ['CPU', 'GPU']
2026-08-29T20:39:09-04:00 <HOSTNAME> start.sh[87884]: compiled GPU {'INFERENCE_PRECISION_HINT': 'f16'} in 12.33s

Failure modes handled

• Server down mid-run: Frigate's request_timeout_ms: 1000 bounds the wait, the timeout resets the socket and re-sends model_request, and systemd restarts the server in 3 s. Server down at Frigate start is the one case that does not recover; see the boot-order section above.
• Bad frame or model hiccup: the except in the loop replies an all-zero (20, 6) so the REP socket never gets stuck in the "owe a reply" state.
• GPU not present at start (driver update, WSL kernel change): load_compiled raises, the process exits 1, systemd retries every 3 s, and the journal says why. Set OV_DEVICE=CPU in the unit to limp along on the CPU without touching Frigate's config.
• Frigate restarts: each restart sends a fresh model_request; the server answers the same way every time (eleven so far, counting mid-run timeouts).
• Reboot: unit is enabled; the boot script brings the distro up; the scheduled task starts it again anyway.

Caveats, and what I would change

• One REP socket means serialised inference. Every request waits for the previous one. At 16 fps and 12 ms that is fine; at 60 fps it would not be. The fix is a ROUTER front end with N worker threads sharing one compiled model, but we have not needed it.
• f16 only. We never measured f32 on this iGPU because f16 compiled first and the detections were correct. If you see odd scores on a different model, try the f32 hint.
• SSDLite only. postprocess_ssd knows the SSD output layout. Swapping in another OpenVINO IR (YOLO-NAS, YOLOv9 exports) means rewriting that function and matching Frigate's model: block for the new input size and tensor layout.
• No auth on the socket. It binds 0.0.0.0 so docker can reach it, and mirrored networking exposes the port on the Windows side. With firewall=false in .wslconfig, which this box uses, Windows Firewall does not filter it either, so it is open to the whole LAN and the router is the only thing keeping it off the internet. On any network you do not fully control, bind to the docker bridge address (OV_ZMQ_BIND=tcp://172.17.0.1:5555) instead of 0.0.0.0, or leave firewall=true and add a Hyper-V firewall rule that admits 5555 from the bridge only.
• AI’s partA verification note has a shelf life. Twelve minutes after the service came up, an assistant checked whether the live loop was really reaching it, found detection_fps pinned at 0.0 on every camera, and wrote it down (the cause was the 320×180 detect size, fixed a quarter of an hour later; see the config page). Half an hour after that, working from its own note, it stopped the service to investigate, saw infer #14650 scroll past in the shutdown log, realised the note was stale, and started it again; about a minute of detection was lost. Re-read the counter before acting on anything written earlier in the evening.
• Decode is still CPU. This fixes detection, not decode. There is no /dev/dri, so Frigate's VAAPI/QSV presets are not available in the container. At 640×360 and 2 fps on seven detecting cameras plus three record-only ones, the whole Linux side of a six-core CPU read 13.6 % in the capture above, which is why we stopped worrying.

If a Frigate maintainer is reading: the zmq type, added for Apple Silicon, turned out to be the right abstraction for this case too. The docs describe it for an NPU the container cannot reach; an Intel iGPU under WSL2, where /dev/dri does not exist, is the same situation with a different accelerator, and this server is the OpenVINO half of it. Corrections belong in the discussion linked below.

Whether you need any of this · The config it plugs into · The WSL2 files underneath · Back to the Frigate hub