Call timew from an extension script?

[jautz]

First of all, thank you for this versatile tool. In general, time tracking is a nuisance but Timewarrior makes it easy to automate and adapt to one's needs.

Imagine an extension ("report") script that loops over today's Timewarrior intervals. It calls another system's API for each interval. On success it tags the corresponding Timewarrior interval by executing timew tag @N exported, so if the script fails afterwards and is re-run it can skip already exported intervals.

This means that the "timew report" process is still running while the extension spawns "timew tag …" processes repeatedly.
Is this considered harmful in any way?

Quoting the Extension API docs: "This is a one-way process, the extension has no way to communicate back to Timewarrior." -- I read this like "the communication protocol (stdin) implies that the extension cannot provide feedback other that its exit code". From the docs I cannot tell whether extensions may invoke timew.

From the source code: in src/timew.cpp the dispatchCommand() is followed by database.commit() which does nothing unless the database is "dirty". From my understanding the report command itself does not modify anything, so the database shouldn't be dirty when it returns. However, having looked at the timew source for the first time today I am not sure. So I decided to ask for your opinion.

[smemsh]

If your first process is only reading, and doesn't rely on any of the read data being invariant while you write from the script (eg, by execve() or system()), then it should work. As long as there's only one writer at a time...

[lauft]

This means that the "timew report" process is still running while the extension spawns "timew tag …" processes repeatedly.
Is this considered harmful in any way?

Adding tags to intervals is an idempotent operation, i.e adding a tag repeatedly to the same interval won't change the result. After the first timew tag @4 FOO call, interval @4 has tag FOO added. Repeating that same command won't change that, there will be only one tag FOO on the interval.

This is a one-way process, the extension has no way to communicate back to Timewarrior.

The message here is that the timew process calling your script will not react on the script's output/or its actions. It runs your extension with the respective data and simply returns its exit code. When your script calls Timewarrior itself, those are independent processes.

Because those calls are independent of each other, each operation is done on the then state of the database. Because modifying commands rely on the interval id to identify the interval to operate on, you have to be careful with any command in your script that changes the order of the intervals, i.e. their assigned ids.

E.g., when timew report ... runs your script with intervals having ids 1, 2, 3, and 4, and your script deletes interval with id 2, ids 3 and 4 point to different intervals then. So any operation you intended for 3 or 4 will miss their target now.

[jautz]

Thank you both @smemsh and @lauft for your replies.

Let me try to summarize what I learned. Please correct me if I am wrong.

On calling timew from an extension

• The "timew report" reads the intervals from the given range at the time of its invocation and feeds this (static) list into the extension.
• The extension may call timew in read or write fashion, but any manipulation that changes the intervals' IDs (i.e. their order) is at least questionable because it makes the extension's input data invalid with respect to their IDs.

That said, setting tags from the extension is not a problem because it does not change the intervals' IDs and it is even idempotent.

My original concern was that the overarching "timew report" process would hold a lock on the database, so that "timew tag" invocations from the extension would fail or deadlock. It seems that this is not the case.

On concurrency in general

Allow me to strain your patience with a rather academic question on concurrency:

What if somebody ran multiple timew commands that change the database concurrently; for example from an extension that processes its input intervals in parallel threads. Rest assured, I am not going to do that, but it seems like a logical follow-up question.

[smemsh]

What if somebody ran multiple timew commands that change the database concurrently

it looks like db update is implemented by creating a new version in a temp file and rename(2) it into place (for atomicity), so probably the last writer's version of the file would win

[lauft]

I am not going to do that, but it seems like a logical follow-up question.

If you were going to do that, you would have to implement some external file locking mechanism, such that only one Timewarrior process manipulates the database at a time. Timewarrior itself does not provide that.

[jautz]

Once again, thank you both for your insights.

I marked @lauft 's first reply as "the answer" but just because it was more detailed; re-reading @smemsh 's first reply I'd say it's also spot-on. So anyway, the whole thread is worth reading :-)