macula_pusher behaviour (macula v10.1.1)

View Source

Behaviour for supervised content pushes (the sender side of a push-initiated upload — PLAN_PUSH_UPLOAD.md Phase 6).

start_link/5,6 returns immediately with a pid, delivers the outcome to Module:handle_pushed/2, and publishes sharing.push_started_v1 / sharing.push_completed_v1 mesh facts around the transfer — including outcome => cancelled if the pusher is stopped before the push resolves.

Unlike macula_feeder (which puts into content-addressed storage for a downloader to discover and pull later), this actively pushes bytes AT a specific, already-known recipient advertising an upload procedure (macula_upload:advertise/5,6) — the client_stream mode STREAMING_GUIDE.md names for exactly this ("an upload, a batch submit"), wrapped with macula_feeder/macula_download's own integrity machinery: macula_manifest:create/2 chunks and hashes Bytes up front, the manifest rides the stream's open-time Args (an out-of-band channel, not an in-band header chunk — see macula_manifest's from_wire/1), and chunks are sent in order over the ONE client_stream the recipient reads from.

No multi-stream parallelism here — a deliberate correction

An earlier draft of this plan said this module "sends chunks via the Phase 3 multi-stream engine." Traced why that can't be true: Phase 3's multi-stream engine lives entirely inside macula_content_transfer, built on content-sharing's OWN dedicated content-stream bookkeeping (macula_station_link's content_stream_bufs / open_content_stream) — a wire mechanism streaming RPC's client_stream/macula_stream doesn't have and was never meant to. This plan's own scope-decision section says so explicitly: "Multi-stream parallel chunk transfer is a content-sharing-only concern. It does NOT extend to macula_streamer/macula_stream_sink." Chunks here go out sequentially over the one stream macula:call_stream/5 / macula_direct_dial:call_stream/5 opens — the same shape a hand written client_stream caller would use, just chunked and hashed for you.

The terminal reply

The recipient's own macula_upload verifies the reassembled bytes against the manifest (receiver-side, never sender-trusted — the sender's claimed manifest proves nothing on its own) and reports the outcome back over client_stream's own terminal-reply channel (macula_stream:set_reply/2 / set_error/2, surfaced here via macula_streamer's new handle_eof/1 callback — see that module's doc). This pusher blocks on macula:await_reply/1 for it, so handle_pushed/2 only ever sees {ok, Mcid} once the recipient has actually verified the bytes, never merely "the local send/2,3 calls all returned ok'" (which would only prove bytes were accepted onto the wire, not that they arrived correctly).

Real cancel

Holds the raw stream pid directly (a stream state field, alongside the lightweight open+send+await proxy worker that reports it back) so cancel/1 reaches it for a real, peer-visible macula_stream:abort/3 STREAM_ERROR — not a blunt local kill that leaves the recipient inferring cancellation from the connection simply going away. Mirrors macula_feeder's own content_transfer field / reap_content_transfer/1 pattern exactly, one layer down (a raw stream instead of a macula_content_transfer pid).

Direct-dial

A correction from the plan's literal wording, worth recording: "mirrors macula_feeder's shape exactly" holds for the WRAPPER (supervised gen_server, cancel via a held handle, mesh facts) but not for direct-dial's DIAL semantics. Content-sharing's direct-dial targets a named STATION (macula_feeder:start_link_direct/5,6 takes an explicit Station pubkey, resolved via its own signed station_endpoint record) because content storage has no notion of "advertised procedures." A push targets a specific ADVERTISED PROCEDURE instead (macula_upload:advertise_direct/6,7's procedure_advertisement), so start_link_direct/5,6 mirrors macula_stream_sink:start_link_direct/5,6's shape instead — same Procedure-based resolve, no Station parameter — and reuses macula_direct_dial:call_stream/5 as-is, which already resolves and dials as one step (unlike content-transfer's lower-level primitives, there is no separate resolve step for this module to drive itself).

Example

   -module(doc_pusher).
   -behaviour(macula_pusher).
   -export([init/1, handle_pushed/2]).
  
   init(Parent) -> {ok, Parent}.
  
   handle_pushed(Result, Parent) ->
       Parent ! {pushed, Result},
       {stop, normal, Parent}.
   {ok, Pid} = macula_pusher:start_link(doc_pusher, Pool, Realm,
       <<"bulk.ingest">>, Bytes, self()).

Summary

Functions

Cancel an in-flight push. Publishes sharing.push_completed_v1 with outcome => cancelled if the push had not resolved yet.

Start a pusher. Pushes Bytes to Procedure on (Realm) via Pool.

As start_link/5, but resolves Procedure's procedure_advertisement from the DHT and dials its provider directly instead of pushing through the pool's existing links. See the "Direct-dial" section above. Requires the recipient to have advertised via macula_upload:advertise_direct/6,7, not plain advertise/5,6.

Callbacks

handle_pushed/2

-callback handle_pushed(Result :: {ok, macula:mcid()} | {error, term()}, State :: term()) ->
                           {noreply, NewState :: term()} | {stop, Reason :: term(), NewState :: term()}.

init/1

-callback init(Args :: term()) -> {ok, State :: term()} | {stop, Reason :: term()}.

Functions

cancel(Pid)

-spec cancel(pid()) -> ok.

Cancel an in-flight push. Publishes sharing.push_completed_v1 with outcome => cancelled if the push had not resolved yet.

start_link(Module, Pool, Realm, Procedure, Bytes)

-spec start_link(module(), macula:pool(), macula:realm(), macula:procedure(), binary()) ->
                    {ok, pid()} | {error, term()}.

Start a pusher. Pushes Bytes to Procedure on (Realm) via Pool.

start_link(Module, Pool, Realm, Procedure, Bytes, Args)

-spec start_link(module(), macula:pool(), macula:realm(), macula:procedure(), binary(), term()) ->
                    {ok, pid()} | {error, term()}.

As start_link/5, with Args passed to Module:init/1.