Input Latency Explained: Console, Capture Card and PC Vision Chain

The chain, stage by stage

Latency is the time between a cause and its visible effect. A console computer-vision setup has two paths that meet at the PC. The video path carries the console's picture through a capture card into the PC, where models read each frame for analysis, stats or overlays. The input path carries anything the PC sends back to the console, such as keyboard and mouse input translated into controller input. The stages, in order:

1. Controller polling: how often the controller reports its state over USB or Bluetooth.
2. Adapter processing: the time a USB adapter such as a Titan Two needs to translate and forward a report.
3. Console frame time: the game reads input and renders frames; at 60 fps each frame takes 16.7 ms.
4. HDMI and capture: the frame leaves the console, the capture card passes it to the TV and digitizes a copy.
5. USB transfer: the copy travels to the PC.
6. Decoding and conversion: compressed video is decoded, raw video is converted to the model's color format.
7. Inference: the vision model processes the frame, plus pre- and post-processing.
8. Output back to the console: input from the PC goes through a USB adapter or a Remote Play session.
9. Network: only with Remote Play, covering encoding on the console, transit and decoding on the PC.

Typical ranges per stage

The table shows typical, illustrative ranges to give you orders of magnitude. They are not measurements of any specific product; your numbers depend on hardware, settings and the game. Where a figure comes from a maker or a specification, the source is named.

Two lessons stand out. First, the stages tied to frames, like the game loop, capture buffering and Remote Play, dominate; microsecond differences between adapters rarely matter. Second, queues are the hidden cost. If frames pile up anywhere, in a capture buffer, a decode queue or an inference backlog, every queued frame adds a full frame period.

Input side: polling, adapters and the trip back

USB devices don't send data whenever they like; the host polls them. Microsoft's USB documentation defines a full-speed bus interval as a 1 ms frame and a high-speed interval as a 125 µs microframe, which limits how often a report can move. A controller polled every 8 ms can hold a press for up to 8 ms before the console sees it; at 1 ms, the worst case drops to 1 ms.

Adapters add their own processing. ConsoleTuner states an average response time of 50–250 µs for the Titan Two, small next to a 16.7 ms frame. The output leg back to the console follows the same rules: the PC hands a report to the adapter over USB, then the console polls the adapter at its own rate.

• Use a wired controller connection instead of Bluetooth when you can.
• Plug adapters directly into the PC and the console, not through unpowered hubs.
• Keep the output path short. A USB adapter keeps the network out of the input path, while Remote Play adds encoding and network time; our Remote Play on PC guide covers how to keep that part tight.

Video side: frame time, capture, USB and decoding

The console's frame rate sets the pace for everything after it. At 60 fps a new frame arrives every 16.7 ms, at 120 fps every 8.3 ms and at 30 fps every 33.3 ms. Games also need time between reading input and showing the result, often more than one frame, and that part is outside your control.

A capture card's passthrough sends the signal on to your TV, so on the TV side the display's own processing matters most: turn on its Game Mode. The capture path is different. The card buffers and digitizes each frame and sends it over USB or PCIe. Uncompressed 1080p60 in YUY2 (2 bytes per pixel) is 1920 × 1080 × 2 bytes × 60 ≈ 249 MB/s, about 2 Gbit/s. That fits a 5 Gbps USB 3 port but is far beyond USB 2.0's 480 Mbit/s. Cards that compress to MJPEG or H.264 save bandwidth but add encoding and decoding time.

• Plug the card into a USB 3 port directly on the motherboard. The Elgato HD60 X, for example, uses a USB-C 3.2 Gen 1 (5 Gbps) link.
• Capture at 1080p60, or 1080p120 if the card supports it, instead of 4K. Most vision models work on much smaller inputs anyway.
• Prefer uncompressed formats such as YUY2 or NV12 over MJPEG when the card offers them.
• Set the console to an output mode the card can capture; our capture card guide lists the modes of current cards.

Inference: the stage you control most

Inference time depends on model size, input resolution, numeric precision and the hardware it runs on. Ultralytics' published figures show the spread: YOLO11n at 640 pixels takes 1.5 ms on an NVIDIA T4 with TensorRT but 56.1 ms on a CPU with ONNX, and the largest variant, YOLO11x, takes 11.3 ms on the same GPU. Pre-processing (resizing, color conversion, normalization) and post-processing add their own time.

On Windows, ONNX Runtime can run the same model on the CPU or, through the DirectML execution provider, on any DirectX 12-capable GPU. The ONNX Runtime docs now list DirectML as in sustained engineering, with new feature work moving to Windows ML. Our ONNX Runtime and DirectML explainer covers the options, and the PC requirements guide covers which GPU and CPU to pick.

• Run inference on the GPU; for vision models a CPU is often many times slower.
• Pick the smallest model and input size that are accurate enough for your task.
• Try half precision (FP16) if your runtime and GPU support it, then check that accuracy holds.
• Always process the newest frame and drop stale ones instead of queueing them.
• Close other GPU-heavy apps; games, encoders and browsers on the same GPU compete for it.

Remote Play and the network

When Remote Play is the output route, input travels over your home network to the console, and the console encodes a video stream for the Remote Play app. Neither Sony nor Microsoft publishes a latency figure. Sony asks for at least 5 Mbps up and down, recommends 15 Mbps and a wired connection. Microsoft asks for 7–10 Mbps on 5 GHz Wi-Fi and warns that VPNs, Wi-Fi extenders and casting to a second screen add latency.

• Use Ethernet for both the console and the PC.
• If you must use Wi-Fi, use 5 GHz and stay close to the router.
• Pause downloads and video streams on other devices during a session.

How to measure latency yourself

Published numbers are a starting point, but your chain is unique. Measure end to end first, then split the PC's share into stages.

1. High-speed camera. Film the input, a button press or an LED, and the screen in one shot. At 240 fps each video frame is 4.17 ms; at 1,000 fps it is 1 ms. Count frames from the press to the first visible change, repeat at least 20 times and use the median.
2. Passthrough vs capture. Film the TV (fed by passthrough) and the PC's preview window side by side while something changes on screen. The frame difference is the cost of capture, USB transfer, decoding and display on the PC.
3. Software timestamps. Log a high-resolution timestamp, for example QueryPerformanceCounter on Windows or time.perf_counter_ns() in Python, when a frame arrives, after decoding, after inference and when output is sent.
4. PresentMon and FrameView. Intel's open-source PresentMon traces CPU, GPU and display frame times and latencies for apps that render on the PC, and NVIDIA FrameView reports frame rates, frame times and power. Use them for your preview or overlay window and GPU load; they measure the PC's rendering, not the console.

Change one thing at a time, measure again and note your settings. A different capture format or model size can move the total more than any cable.

Frequently asked questions

Sources

• Microsoft Learn — How to transfer data to USB isochronous endpoints (bus intervals)
• ConsoleTuner — Titan Two
• Elgato — Game Capture HD60 X
• Ultralytics — YOLO11 performance metrics
• ONNX Runtime — DirectML Execution Provider
• Intel GameTechDev — PresentMon
• NVIDIA — FrameView
• PlayStation — When Remote Play isn't available
• Xbox Support — Troubleshoot remote play gaming