Feature request: optional websocket reporter to stream real-time test results

[dneu2]

I propose adding an optional test completion callback function that can report test results over a WebSocket connection. Features:

• Opt-in only: It would be invoked with a special URL parameters, ws_host=<ip address> and ws_id=<unique id>, ensuring no change to default test behavior.

• Allows for new use cases: live reporting for nightly CI/CD builds, letting developers see test failures in real time without waiting for full JSON/HTML reports. Useful for devices that can’t host WPT themselves.

• Extensible: Could serve as a framework for other streaming/reporting utilities in the future.

I’m building this initially to run WPT tests on smart TVs in my dev environment, but I believe it can benefit a broader set of workflows where live feedback or remote reporting is valuable.

[jonathan-j-lee]

I think testharnessreport.js already makes your request possible for testharness tests without major changes to tests or the shared infrastructure. (Non-testharness tests might need a different solution.)

As an example, Chromium runs some WPTs with content shell, which is a minimal browser for testing that's controlled through a nonstandard protocol instead of WebDriver.

1. The run_web_tests.py harness sets up a custom wptserve config that will serve this script.
2. That testharnessreport.js registers a callback that will forward the results to content shell-specific testRunner.* JS APIs that dump the results to content_shell's stdout.

[dneu2]

Thank you for your reply!

I’m asking for a way to automatically enable a WebSocket reporter on the public WPT site without requiring a custom server or configuration. This would make it much easier to run tests on remote or constrained devices (smart TVs, embedded browsers) and get live feedback as tests run.

The feature would be strictly opt-in and fully backward-compatible, so it wouldn’t affect existing test behavior.

I’d be happy to submit a PR with a minimal, opt-in implementation for review.