Skip to content

feat(tracer): add configuration for connection mode#3573

Open
Leiyks wants to merge 24 commits intomasterfrom
leiyks/sidecar-threaded-fallback
Open

feat(tracer): add configuration for connection mode#3573
Leiyks wants to merge 24 commits intomasterfrom
leiyks/sidecar-threaded-fallback

Conversation

@Leiyks
Copy link
Contributor

@Leiyks Leiyks commented Jan 13, 2026

Description

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@datadog-official
Copy link

datadog-official bot commented Jan 13, 2026

⚠️ Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

❄️ 4 New flaky tests detected

tmp/build_extension/tests/ext/background-sender/sidecar_thread_mode_permissions.phpt (Thread mode sidecar: uses abstract Unix socket (no filesystem permissions needed)) from PHP.tmp.build_extension.tests.ext.background.sender (Datadog) (Fix with Cursor)
001+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
     Sidecar uses abstract socket
tmp/build_extension/tests/ext/pcntl/pcntl_fork_thread_mode_orphan.phpt (Thread mode sidecar: orphaned child process promotes itself to master after parent exits) from PHP.tmp.build_extension.tests.ext.pcntl (Datadog) (Fix with Cursor)
001+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
002+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
     Child span submitted
tmp/build_extension/tests/ext/background-sender/sidecar_thread_mode_permissions.phpt (Thread mode sidecar: uses abstract Unix socket (no filesystem permissions needed)) from php.tmp.build_extension.tests.ext.background.sender (Datadog) (Fix with Cursor)
001+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
     Sidecar uses abstract socket
tmp/build_extension/tests/ext/pcntl/pcntl_fork_thread_mode_orphan.phpt (Thread mode sidecar: orphaned child process promotes itself to master after parent exits) from php.tmp.build_extension.tests.ext.pcntl (Datadog) (Fix with Cursor)
001+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
002+ [ddtrace] [error] There is no user-span on the top of the stack. Cannot close.
     Child span submitted
View all

🧪 1038 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integration\PHPInstallerTest::testSearchPhpBinaries
Test code or tested code printed unexpected output: Searching for available php binaries, this operation might take a while.
testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a9991a00000000833980d0afec726a
tid: 69a9991a00000000
hexProcessTraceId: 833980d0afec726a
hexProcessSpanId: e1f6a564226de103
processTraceId: 9455730526434128490
processSpanId: 16282383352297545987

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a9992e0000000092707456a55091d0
tid: 69a9992e00000000
hexProcessTraceId: 92707456a55091d0
hexProcessSpanId: f97a3264a6192835
processTraceId: 10552061842418602448
processSpanId: 17976736270467541045
View all
This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8b7a5b3 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@codecov-commenter
Copy link

codecov-commenter commented Jan 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.08%. Comparing base (8cd0131) to head (8b7a5b3).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3573      +/-   ##
==========================================
- Coverage   62.22%   62.08%   -0.14%     
==========================================
  Files         141      141              
  Lines       13352    13352              
  Branches     1746     1746              
==========================================
- Hits         8308     8290      -18     
- Misses       4253     4269      +16     
- Partials      791      793       +2     

see 4 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8cd0131...8b7a5b3. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link

pr-commenter bot commented Jan 13, 2026

Benchmarks [ tracer ]

Benchmark execution time: 2026-03-05 16:01:30

Comparing candidate commit 8b7a5b3 in PR branch leiyks/sidecar-threaded-fallback with baseline commit 8cd0131 in branch master.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 188 metrics, 3 unstable metrics.

scenario:ComposerTelemetryBench/benchTelemetryParsing-opcache

  • 🟥 execution_time [+0.977µs; +1.823µs] or [+7.940%; +14.824%]

scenario:PHPRedisBench/benchRedisBaseline-opcache

  • 🟥 execution_time [+3.380µs; +5.054µs] or [+2.425%; +3.626%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4-opcache

  • 🟥 execution_time [+275.822ns; +509.578ns] or [+2.486%; +4.592%]

@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch 4 times, most recently from ce20a9e to 2eb0609 Compare January 14, 2026 14:41
@pr-commenter
Copy link

pr-commenter bot commented Jan 14, 2026

Benchmarks [ profiler ]

Benchmark execution time: 2026-01-14 16:24:26

Comparing candidate commit 05ebcae in PR branch leiyks/sidecar-threaded-fallback with baseline commit 88a859c in branch master.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 28 metrics, 6 unstable metrics.

scenario:php-profiler-timeline-memory-with-profiler-and-timeline

  • 🟥 cpu_user_time [+36.124ms; +78.620ms] or [+2.324%; +5.058%]
  • 🟥 execution_time [+32.293ms; +51.335ms] or [+2.645%; +4.204%]

@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch 9 times, most recently from 06223eb to cda73e8 Compare January 16, 2026 15:24
@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch 9 times, most recently from 211d162 to 0b1eb24 Compare January 23, 2026 12:42
@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch 2 times, most recently from 58e8671 to e4a603a Compare February 18, 2026 14:20
Leiyks added 10 commits March 2, 2026 15:37
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch from e4a603a to a2f512c Compare March 2, 2026 14:39
Leiyks added 4 commits March 2, 2026 16:41
- crashtracker_segfault: filter crash pings by absence of os_info instead
  of kind == "Crash ping" (libdatadog 27de9f37d changed crash ping kind
  from "Crash ping" to "UnixSignal")
- debugger_span_decoration_probe: update expected URL from /debugger/v1/input
  to /debugger/v2/input (libdatadog 29e00628b routed logs to snapshots endpoint)
…lter

With newer libdatadog (29e00628b), debugger logs are routed to
/debugger/v2/input instead of /debugger/v1/diagnostics. The filter
in replayDebuggerData() still included /debugger/v1/diagnostics with
body (added in c3be656 for older routing), which caused it to return
a diagnostics response before the actual /debugger/v2/input snapshot,
breaking debugger_span_decoration_probe.phpt.

Remove the diagnostics clause — only /debugger/v1/input and
/debugger/v2/input are snapshot/log endpoints.
…back routing

libdatadog 29e00628b routes logs to /debugger/v2/input but falls back to
/debugger/v1/diagnostics when the agent doesn't support v2 (as is the
case with the request replayer in tests). Update the expected URI in
debugger_span_decoration_probe.phpt accordingly.

Also restore the /debugger/v1/diagnostics filter in replayDebuggerData()
which was incorrectly removed in the previous commit.
Leiyks added 5 commits March 3, 2026 17:03
…ailable

In ddtrace_sidecar_setup_thread_mode, a forked child detecting is_child_process=true
would try to connect to the parent's thread listener. If the parent used subprocess
mode (no thread listener), the connect would fail and the child returned with no
sidecar and no fallback.

Mirror the existing fallback logic from ddtrace_sidecar_handle_fork: when the
parent's listener is unavailable, reset ddtrace_sidecar_master_pid to the current
process and fall through to start a new master listener in this process.
…ct path

When a parent process initializes the sidecar in thread mode, forks, and
then exits, the child inherits a broken transport (parent's listener thread
is dead). In dd_sidecar_connect(), if ddog_sidecar_connect_worker() fails
and current_pid != master_pid, promote the child to master so it can still
submit traces.

The existing fallback in ddtrace_sidecar_setup_thread_mode covers the
initial-setup path, but the reconnect path (ddtrace_sidecar_connect_callback
-> dd_sidecar_connect) had no equivalent fallback, causing a silent failure
for orphaned children that already had an inherited transport.

Add a .phpt test that verifies the orphaned child can create and submit
spans after the parent exits.
… compatibility

Thread-mode sockets now include the master's effective uid in the filename
(libdd.<ver>@<uid>-<pid>.sock in /tmp/libdatadog/). A worker process that later
drops privileges via setuid() (e.g. www-data under PHP-FPM) still computes the
same socket path as the master listener, and ensure_dir_exists now best-effort
chmods the directory world-writable to allow socket creation by any user.

Also fixes a double-dot bug in socket/lock path construction (Rust >=1.87
no longer strips leading dots from with_extension arguments).

Adds test: sidecar_thread_mode_permissions.phpt verifies the socket is created
with the correct uid-pid encoding.
@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch from 04d237e to 0d0c164 Compare March 4, 2026 13:48
Leiyks added 2 commits March 4, 2026 16:36
Update libdatadog submodule: thread mode sidecar now uses abstract Unix
sockets on Linux (no filesystem permissions needed, any user can connect)
and a single-threaded Tokio runtime (no extra OS threads, fixes LSan
"Running thread was not suspended" ASAN warnings at process exit).

Update sidecar_thread_mode_permissions.phpt to verify abstract socket
usage (no filesystem socket created) instead of checking file permissions.
Signed-off-by: Alexandre Rulleau <alexandre.rulleau@datadoghq.com>
@Leiyks Leiyks force-pushed the leiyks/sidecar-threaded-fallback branch from fd58c34 to 31e8976 Compare March 4, 2026 15:56
Leiyks added 3 commits March 4, 2026 17:14
After fork, the child inherits the parent's heap including the Tokio
current_thread runtime allocations and Rust stdlib once-cell inits from
the sidecar thread. Since threads don't survive fork, these are orphaned
allocations that LSan incorrectly reports as leaks. This is the same
reason daemonize() already sets LSAN_OPTIONS=detect_leaks=0 for the
subprocess sidecar.
When the PHP-FPM master process runs as root, the sidecar thread (thread
mode) creates named SHM objects with 0600 by default, making them
inaccessible to worker processes running as www-data.

Call ddog_sidecar_set_shm_open_mode(0644) before starting the master
sidecar listener when geteuid()==0, so workers can open the SHM regions
read-only. The mode is set in both ddtrace_sidecar_setup_thread_mode()
and ddtrace_sidecar_minit() to cover all code paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants