Support python contextvars based context propagation

[aabmass]

Based on #903, there is no specific python trace context correlation heuristic. The thread based correlation should work well for traditional WSGI frameworks like Flask and Django but not for asyncio.

Rather than focus on asyncio, I'm wondering if correlation can be done by the builtin contextvars api which provides generic context local storage and is well integrated with asyncio and can be plugged into other concurrency frameworks with context.run(). There is a C API for entering/exiting a context https://docs.python.org/3/c-api/contextvars.html#c.PyContext_Enter.

Let me know if you need more details or it's infeasible. We also use contextvars for the OTel Python context which might make integration with a live SDK feasible.

[grcevski]

Cool! Thanks @aabmass, I'll take a look. I think it might be possible with PyContext_Enter. I'll let you know as soon as I have dug into it a bit.

[aabmass]

I did some poking around and found cpython usually calls _PyContext_Enter/_PyContext_Exit internally, instead of the public PyContext_Enter which just wrap those functions. That's including calls to contextvars.Context.run() from python code. I had some success with a bpftrace script on this function, but it looks it looks like _PyContext_Enter/_PyContext_Exit gets stripped on many python builds, including on Debian and UV's managed python binaries. I checked and the underscore version are present at in all python versions 3.9-3.14 which is promising (I didn't check any earlier than that).

I'll try to share a working example with bpftrace later today.

[aabmass]

I threw together a runnable example with some explanations https://github.com/aabmass/opentelemetry-python/tree/uprobe-ctxvars/uprobe-contextvars/README.md.

[grcevski]

Awesome, that's really helpful! I'm going to give this a shot tomorrow to see if I can replicate it in OBI. I'll ask any question if I get stuck here, I hope that's ok :) .

[aabmass]

Thank you and yes please reach out. I'm not 100% confident on the approach so let me know if it seems infeasible. Python re-using the Context objects seems a little tricky, but doable.

[grcevski]

This is great, @marctc and I tried it today and we were able to see context propagation work. Essentially put 2 of the probes you suggested, context_run and PyContext_CopyCurrent. Those appear sufficient for the examples we made so far. We are working on a POC and we'll see how far this takes us. I started a branch https://github.com/grcevski/opentelemetry-ebpf-instrumentation/tree/python_async_io and Marc will continue tomorrow in his timezone to add the maps and the correlation code.

[aabmass]

That's awesome news! I can also try to come up with more example cases and test it out.

[marctc]

hey @aabmass, thanks! I manged to make it work with a simple example. If you have a more complex example, i'd great if you can share so I can add it to integration tests.

[marctc]

You can take a look a draft PR here in case you want to thest or add feedback: #966