Flowli¶
Flowli is a workflow engine for Python. A workflow is an async function. The
engine runs that function as a coroutine execution stack. It appends one journal
entry for each frame of that stack. Because of the journal, an execution is
traceable and it is resumable.
Flowli has no server of its own. CairnDB gives the storage, the coordination and the audit log. One bucket holds the journals, the queues, the leases and the messages. The bucket is the only shared component. You run processes. You do not run a cluster.
The idea in one function¶
@registry.workflow("invoice_approval", version="3")
async def invoice_approval(ctx: Context, invoice_id: str) -> str:
"""Approve one invoice. A large invoice waits for a person."""
invoice = await ctx.step(fetch_invoice, invoice_id)
score = await ctx.step(score_risk, invoice)
if score > 0.8:
decision = await review(ctx, "finance", invoice, timeout=timedelta(days=3))
if decision is None or decision.verdict != "approve":
return "rejected"
await ctx.send(f"invoice:{invoice_id}", {"status": "approved"})
return "approved"
This function can wait three days for a person. The worker does not wait with it. The worker writes the journal, releases the execution, and takes other work. When the decision arrives, a worker reads the journal, runs the function again from the first line, and returns the memo of each frame that ran already. The function continues at the line that waits.
What the engine gives you¶
Property |
How you get it |
|---|---|
Durable execution |
Each frame writes its outcome to the journal before the next frame starts. |
Resumption after a crash |
A worker replays the function and reads the memo of each finished frame. |
One owner at a time |
A lease gives one worker the ownership of one execution. The epoch fences the others. |
Idempotent start |
A dispatch key makes two starts converge on one execution. |
A wait that costs nothing |
A suspended execution holds no worker, no thread and no connection. |
Humans in the workflow |
A review is a task on a queue and a message on a channel. |
Work outside the engine |
A delegate task goes to an agent, to a service, or to a group of people. |
A trace you can read |
Each entry names who acted, what code acted, where it ran, and when. |
Where to start¶
Write a workflow, start it, run a worker, and read the journal. Ten minutes.
The words this project uses: execution, frame, memo, channel, lease.
The layers, the ports, the processes, and what each guarantee comes from.
Every setting: the storage, the engine, the jobs, the service, the logs.
The processes to run, the bucket to give them, and the limits to respect.
The public classes and functions, from the docstrings of the code.
The parts of the project¶
Part |
Content |
|---|---|
|
the engine: the domain, the adapters, the runtime, the CLI |
|
the HTTP service over one engine |
|
the operator interface over that service |
|
coding agents as the consumer of a delegate task |
|
the controller over the runner: plan, routing, merge, gates |
The design principles¶
One journal per execution. The journal is the trace and the memo table. No other store holds the result of a frame.
First outcome wins. The first
frame.completedentry for a frame id is the memo of that frame. A duplicate run converges.Replay from the top. To resume an execution, a worker runs the workflow function again from the start. Each frame with a memo returns the memo.
Deterministic frame ids. A frame id is a path. It does not depend on the clock, on the worker, or on the order of concurrent frames.
Ownership by lease. One worker owns one execution at a time. The lease epoch is a fence token.
Provenance on every write. Each entry, task and message carries who acted, what code acted, and where the code ran.
Humans are actors. A review is a task plus a message. The engine has no review primitive.
The writing style
These documents and the specifications use ASD-STE100 Strict: one word for one concept, short sentences, and the active voice. The glossary lists the words and the verbs. The specifications are the authority when a page here is shorter than the truth.