Coming from FastFlow and BBFlow¶
Tolquane keeps the vocabulary of the FastFlow building blocks, so the papers and the
BBFlow thesis read the same way here. What changed is the surface: a function is a
node, >> is a pipeline, a farm is one call.
| BBFlow | Tolquane |
|---|---|
defaultJob with U runJob(T) |
@tq.node def f(x) |
return null to drop |
return tq.SKIP |
runJobMulti(T, out) + sendOutTo |
def f(x, ctx) + ctx.send(y, to=i) |
runJob() manual loop |
@tq.raw def f(ctx) with ctx.inputs() |
init() / EOS() |
on_start(ctx) / on_end(ctx) on a class node |
sendOut / sendOutToAll / sendEOS |
ctx.send / ctx.broadcast / ctx.stop |
position / id |
ctx.source / ctx.index |
ff_node(job) |
the function itself |
ff_comb(a, b) |
tq.comb(a, b) |
ff_farm(n, job, EMIT, COLLECT) |
tq.farm(job, workers=n, emit=..., collect=...) |
defaultJob.uniqueJob(job, i) |
pass a class; instances are made for you |
ff_pipeline(a, b) + appendBlock |
a >> b >> c |
TYPE_1_1 ... TYPE_NxM |
inferred; tq.explain shows which |
ff_all2all.combine_farm(...) |
tq.all2all(...) |
ff_queue(blocking, bounded, size) |
tq.channel(capacity=..., batch=...) on an edge; blocking always, bounded by default |
ff_queue_TCP(INPUT/OUTPUT, id, host) |
deploy file; edges become TCP automatically |
bb_settings.* |
tq.run(..., options=tq.Options(...)) |
preloader |
not needed |
customWatch |
stats=True report, trace=True |
manual feedback via addInputChannel |
tq.feedback(...) |
ordered_farm_labeling example |
ordered=True |
Every program in BBFlow's src/tests has a counterpart here; the table is in
docs/bbflow-tests.md.
The MSOM use case is ported faithfully, grid links and all, and its parallel map
equals the sequential one exactly.
What BBFlow could not do that Tolquane does: end of stream that cannot be lost or raced, a deadlock that is reported instead of hung, ordered farms and on-demand scheduling built in, feedback loops that close by rule, batching, processes, and a distributed runtime that keeps backpressure across hosts. The design document lists the twenty bugs and traps found in the Java code and the rule that closes each one.