Skip to content

awatch() should take an optional ready: asyncio.Event #350

@owtaylor

Description

@owtaylor

The awatch() interface is a little tricky to use robustly because there's no way to wait until the underlying filesystem watch has been set up. For example, the following code needs an await asyncio.sleep(0) to work correctly.

import asyncio
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from pathlib import Path
from tempfile import TemporaryDirectory

from watchfiles import awatch, Change


@asynccontextmanager
async def log_file_changes(path) -> AsyncIterator[asyncio.Queue]:
    """
    Async context manager for watching file changes.

    Usage:
        async with filesystem.watch_files():
            # File changes will be automatically detected and processed
            await some_other_work()
    """

    change_queue = asyncio.Queue()

    async def _watch_files() -> None:
        try:
            async for changes in awatch(path):
                for change in changes:
                    change_queue.put_nowait(change)
        except asyncio.CancelledError:
            # Expected when stopping the watcher
            pass

    watch_task = asyncio.create_task(_watch_files())
    try:
        # This gives _watch_files() the chance to run until the point awatch()
        # has set up the necessary inotify watches and suspends itself to wait
        # for events.
        await asyncio.sleep(0)
        yield change_queue
    finally:
        watch_task.cancel()
        try:
            await watch_task
        except asyncio.CancelledError:
            pass


async def test_log_file_changes() -> None:
    with TemporaryDirectory() as tmpdir:
        tmp_path = Path(tmpdir)
        test_file = tmp_path / "test.txt"

        async with log_file_changes(tmp_path) as change_queue:
            test_file.write_text("initial content")
            change = await asyncio.wait_for(change_queue.get(), timeout=1.0)
            assert change[0] == Change.added
            assert change[1] == str(test_file)


if __name__ == "__main__":
    asyncio.run(test_log_file_changes())

If you could pass in a async.Event to awatch(), then this becomes:

--- awatch-test.py	2025-09-07 04:58:23.123768732 -0400
+++ awatch-test-proposed.py	2025-09-07 04:59:15.626235470 -0400
@@ -19,10 +19,11 @@
     """
 
     change_queue = asyncio.Queue()
+    ready = asyncio.Event()
 
     async def _watch_files() -> None:
         try:
-            async for changes in awatch(path):
+            async for changes in awatch(path, ready=ready):
                 for change in changes:
                     change_queue.put_nowait(change)
         except asyncio.CancelledError:
@@ -31,10 +32,7 @@
 
     watch_task = asyncio.create_task(_watch_files())
     try:
-        # This gives _watch_files() the chance to run until the point awatch()
-        # has set up the necessary inotify watches and suspends itself to wait
-        # for events.
-        await asyncio.sleep(0)
+        await ready.wait()
         yield change_queue
     finally:
         watch_task.cancel()

Pragmatically, there's not a lot of difference between the two: my understanding while in theory it's not guaranteed that asyncio.sleep(0) will allow all scheduled coroutines to run to their next suspension point, practically speaking that's almost always the case.

But I see the advantage is that if you have trouble with events not showing up, you can just look at the API and see immediately how to fix it - it would be pretty easy for someone not to recognize this as a scheduling problem and some actual delay is needed: await asyncio.sleep(0.1) (*)

it also prevents problems if awatch is ever changed to suspend before the point where the watches are set up.

(*) The background here is that I had Claude Code write tests for some code using watchfiles, it hit this issue and added sleeps to get it to work. Since I detest tests that sleep, I dug in to try and come up with a proper fix, and spent quite a bit of time reading watchfiles and even notify code to try and understand what was going on before I realized that the actual problem. More experience with Python async would have helped me, but that's probably true of many...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions