macula_upload behaviour (macula v11.1.0)
View SourceBehaviour for supervised content uploads (the receiver side of a push-initiated upload — PLAN_PUSH_UPLOAD.md Phase 6, the recipient macula_pusher pushes at).
advertise/5,6 registers an upload procedure. Each inbound push starts one supervised, ephemeral receiver: it reads the manifest macula_pusher passed as the stream's open-time Args (decoded via macula_manifest:from_wire/1, the same decode content sharing's own _content.get_manifest path already relies on), uses it only when its MCID, recomputed from its canonical fields, is the MCID it names (macula_manifest:verify_mcid/2), accumulates pushed chunks, and once the sender half-closes, reassembles and verifies them against the manifest with macula_manifest:verify/2 — receiver-side verification, never sender-trusted, matching content-sharing's existing "content is self-verifying by hash" model exactly; the sender's claimed manifest proves nothing on its own. Delivers {ok, Mcid, Bytes} | {error, _} to Module:handle_uploaded/2, and publishes sharing.upload_started_v1 / sharing.upload_completed_v1 mesh facts around each push's lifetime.
A correction from the plan's literal wording
The plan that named this module described it as "mirroring macula_download's shape" while, in the same sentence, also saying its public API is advertise/advertise_direct — macula_download doesn't have those; it has start_link/4,5 (a one-shot, caller-initiated fetch). What this module actually is — a long-lived, ADVERTISED provider spawning one ephemeral child per inbound push — mirrors macula_streamer's shape, not macula_download's. Built directly on top of it: this module IS a macula_streamer callback module internally (?MODULE is passed as macula_streamer:advertise/6's own Module argument, closing over the caller's Module/Args in its own state), reusing Phase 5's supervision, client_stream receive loop (handle_chunk/2), and abort-wired cancel for free — "inherits... cancel from day one, nothing to retrofit later," per the plan's own intro to this phase.
== The terminal reply, and why handle_uploaded/2 return values are ignored ==
Once the sender half-closes, macula_streamer calls this module's handle_eof/1 (Phase 6's new optional callback on that module) — the one place with access to the raw stream needed to set a terminal reply. Verification happens right there: {reply, {ok, Mcid}, NewState} on success or {reply, {error, Reason}, NewState} on failure, which macula_streamer turns into macula_stream:set_reply/2 / set_error/2 — exactly the channel macula_pusher's own macula:await_reply/1 blocks on. The Mcid itself needs no separate round trip: it's a field already present in the manifest both sides hold (macula_manifest mirrors macula-station's own algorithm BYTE-FOR-BYTE). It is trustworthy to echo back because handle_open/2 recomputed it from the manifest's canonical fields, and the actually-received bytes then verified against that manifest's size and root hash.
Module:handle_uploaded/2 — the LOCAL delivery, to whatever application registered this upload handler — fires separately, from this module's own terminate/2 (after the wire-level reply is already set, so a slow local callback can never block it). Because terminate/2 can no longer act on a {stop, _, _} return the way a live handle_info clause could, handle_uploaded/2's return value is ignored — any(), not {noreply,_} | {stop,_,_} — the same shape macula_stream_sink's own handle_close/2 already uses for the identical reason (it too only ever fires from terminate/2). This is a deliberate divergence from macula_feeder/macula_download's handle_fed/handle_downloaded callbacks, which DO get a meaningful {stop,_,_} because they fire from a live handle_info clause, not terminate/2 — "mirrors macula_feeder's shape exactly" does not hold for this specific callback's return contract, traced to WHERE each one is actually invoked from.
A malicious or buggy sender that never half-closes, or that pushes more chunks than its own manifest declared, is bounded: exceeding the declared chunk_count stops the receiver (a genuine {error, too_many_chunks}, abort-wired same as any other non-normal stop) rather than accumulating chunks without limit — the manifest is a system-boundary input from an untrusted remote peer, worth guarding explicitly, unlike macula_manifest:verify/2's own eventual size_mismatch, which would only catch this after the fact.
Direct-dial
advertise_direct/6,7 is macula_streamer:advertise_direct/6,7 verbatim, mode => client_stream folded into Opts before forwarding — see that module's own "Direct-dial" section. (Building this surfaced a real, separate bug in macula_streamer:advertise_direct/7 itself: it called the arity-5 advertise/5, silently discarding whatever modeOpts carried, so ANY direct-dial-advertised client_stream procedure — not just this module's — would have been served as server_stream instead, with no error anywhere to say so. Fixed at the source, in macula_streamer.erl itself, per this project's "fix bugs in owned libraries immediately" rule — not specific to Phase 6, but found while building it.)
Stream I/O
advertise/6 and advertise_direct/7 pass their options on to macula_streamer, stream_io and advertise_stream among them, and advertise_direct/7 also publish_advertisement (see its "Stream I/O" section). Each upload announces its sharing.upload_* facts with fact_publish, macula:publish/4 by default.
Example
-module(doc_upload).
-behaviour(macula_upload).
-export([init/1, handle_uploaded/2]).
init(Parent) -> {ok, Parent}.
handle_uploaded(Result, Parent) ->
Parent ! {uploaded, Result},
ok. {ok, _Sup} = macula_upload:advertise(Pool, Realm,
<<"bulk.ingest">>, doc_upload, self()).
Summary
Functions
Advertise Procedure on Pool/Realm as an upload target. Each push sent at it starts one supervised, ephemeral receiver, threading state through Module:init/1 and delivering the final outcome to Module:handle_uploaded/2.
As advertise/5. Opts may include announce (default true) for this module's OWN sharing.upload_* facts and fact_publish, the function it announces them with. The other options, such as auth, reuse_sup, stream_io and advertise_stream, go on to macula_streamer:advertise/6, with modeclient_stream (see "Stream I/O" above).
As advertise/5, and additionally publishes a signed procedure_advertisement DHT record naming this pool's connected station as the server, so macula_pusher:start_link_direct/5,6 can resolve and dial here directly. See macula_streamer:advertise_direct/6,7.
As advertise_direct/6, with Opts forwarded to macula_streamer:advertise_direct/7, and so to the advertisement publish, e.g. authorization, the provider authorization an org namespaced procedure needs, except fact_publish, the function this module announces its own facts with.
Stop advertising Procedure.
Callbacks
Functions
-spec advertise(macula:pool(), macula:realm(), macula:procedure(), module(), term()) -> {ok, pid()} | {error, term()}.
Advertise Procedure on Pool/Realm as an upload target. Each push sent at it starts one supervised, ephemeral receiver, threading state through Module:init/1 and delivering the final outcome to Module:handle_uploaded/2.
-spec advertise(macula:pool(), macula:realm(), macula:procedure(), module(), term(), map()) -> {ok, pid()} | {error, term()}.
As advertise/5. Opts may include announce (default true) for this module's OWN sharing.upload_* facts and fact_publish, the function it announces them with. The other options, such as auth, reuse_sup, stream_io and advertise_stream, go on to macula_streamer:advertise/6, with modeclient_stream (see "Stream I/O" above).
-spec advertise_direct(macula:pool(), macula:realm(), macula:procedure(), module(), term(), macula_node_keys:node_key()) -> {ok, pid()} | {error, term()}.
As advertise/5, and additionally publishes a signed procedure_advertisement DHT record naming this pool's connected station as the server, so macula_pusher:start_link_direct/5,6 can resolve and dial here directly. See macula_streamer:advertise_direct/6,7.
-spec advertise_direct(macula:pool(), macula:realm(), macula:procedure(), module(), term(), macula_node_keys:node_key(), map()) -> {ok, pid()} | {error, term()}.
As advertise_direct/6, with Opts forwarded to macula_streamer:advertise_direct/7, and so to the advertisement publish, e.g. authorization, the provider authorization an org namespaced procedure needs, except fact_publish, the function this module announces its own facts with.
-spec unadvertise(macula:pool(), macula:realm(), macula:procedure()) -> ok.
Stop advertising Procedure.