macula_direct_dial (macula v11.1.0)
View SourceDirect-dial resolve-and-call: shared internals for macula_request/macula_response, macula_streamer/ macula_stream_sink, and macula_feeder/macula_download.
Not a public API on its own — macula_request:start_link_direct/6,7,8, macula_response:advertise_direct/6,7, macula_stream_sink:start_link_direct/5,6, macula_streamer:advertise_direct/6,7, macula_download:start_link_direct/4,5, and macula_feeder:start_link_direct/5,6 are the entry points. Factored out because RPC, streaming, and content-download all need the same shape of resolve sequence (find_records -> verify -> read the record -> build a quic:// dial URL). Streaming and RPC share the IDENTICAL discovery mechanism — a procedure_advertisement does not distinguish RPC from streaming, only the eventual dial (call_station/7 vs call_stream_station/7) does — so publish_advertisement/4,5 is reused as-is by both providers, and call/6/call_stream/6 share the same candidate resolution and "Trust model" below. Content has no publish step here at all — see "Content" further down.
Resolution
Every advertisement that passes trust filtering is a candidate, in the order the DHT returns them. A candidate whose station_endpoint can't be resolved, or whose link doesn't connect ({error, not_connected} from call_station/7 or call_stream_station/7), is passed over for the next one, but only before the request is sent: once a CALL or a stream has gone out, its outcome is returned as it is. When no candidate qualifies, or every one failed before sending, resolution asks the DHT again after a pause that doubles from 100 ms to at most 1 s, and tries a candidate that already failed again only when its advertisement or its station_endpoint record has changed. A record just published on the provider's station has not necessarily replicated to the caller's station yet, so a miss is not final until the deadline. One deadline bounds all of it: each DHT lookup, each candidate's endpoint lookup and connect wait (within a share of the time that remains, at least one second while that much remains), and the request. At the deadline the result is, in this order, the most recent candidate's failure, why the latest answered DHT lookup found nothing qualifying, the latest failed lookup's error, or {error, {unresolved, timeout}}. A failed lookup is retried like an empty pass, and one the deadline cuts off records nothing.
Trust model
Two independent checks, both mandatory, cover what the QUIC/TLS layer cannot. (1) Every candidate procedure_advertisement arrives verified under the node's crypto profile (macula:find_records/2), and is trusted only when it advertises the resolved procedure in the resolved realm and its provider authorization verifies (macula_record:verify_authorization/3, D25 item 6): a procedure with an org namespace needs an authorization for that org, and a procedure without one carries none. Otherwise any node able to sign SOME record could name a real, legitimate station as the server for a procedure it has no authority over, and the station_endpoint check below would still pass (it only proves we reached the station we were told to reach, not that whoever told us so was authorized to). (2) The resolved station_endpoint must be signed by the station itself (station_signed_endpoint/2). The actual QUIC dial trusts NEITHER the TLS certificate (pin_tls_cert => false — a production station's TLS is terminated by an unrelated PKI, e.g. Let's Encrypt, so pinning the cert key can never succeed) NOR nothing (verify => none alone would trust whoever answers): trust is enforced at the application layer instead, via the cryptographically signed CONNECT/HELLO handshake (the peer identity binding in macula_peering_conn) checked against the exact node_id the signed DHT chain above resolved.
An authorization verifies against the realm key the pool pinned for the call's realm when it started (macula:connect/2's realm_trust => #{RealmId => RealmKey}): the realm key as carried, for the org directory and the procedure delegation, the only authorization form. Without a key pinned for the realm, an advertisement for an org namespaced procedure is never trusted. A realm key never arrives with a request: realm_trust on a call, like the 10.x options verify_cert_chain and cert_chain, is refused by name, with {error, {removed_option, Key}} (see removed_option/2).
A caller checks the authorization from the advertisement alone and looks up no tombstone. A delegation its org withdraws is honoured until it expires, so the caller-side revocation bound is the delegation's maximum lifetime, six hours (macula_record's REALM_AND_ORG_MAX_LIFETIME_MS), and it lengthens if that lifetime does.
Content
put_content/4 has no resolve step at all — unlike a GET, a PUT names its OWN target: the caller already knows (or is choosing) which station to seed, so it takes Station directly and resolves only that station's own station_endpoint (resolve_station_endpoint/2,3, the same machinery call/6 uses internally for serving_station).
get_content/3 and fetch_content/4 resolve and fetch deliberately WITHOUT the authorization check above: content's threat model genuinely differs from RPC's. An RPC reply is opaque and unverifiable except by trusting whoever answered, so proving the ADVERTISER is authorized matters. Content is content-addressed, and the fetched bytes are checked against the MCID client-side regardless of which peer served them. Single-block content is re-hashed against the MCID (in macula_content_transfer). For chunked content the fetched manifest is used only if its MCID, recomputed from its canonical fields, is the one requested (macula_manifest:verify_mcid/2); each chunk is then hashed against that manifest, and the reassembled bytes are checked against its size and root hash (macula_manifest:verify/2). A rogue or unauthorized announcer can at most refuse to serve or waste a dial; it cannot make a caller accept content that does not match the MCID it asked for. The same holds for trying the next provider after a fetch that fails, so every announced provider is a candidate the way advertisements are for calls. What still matters, and is still mandatory, is (1)'s analogue for content_announcement: the signer must equal the announcer_node it claims (the check macula:find_content_providers/2 makes too), so an attacker cannot at least misattribute who is claiming to serve what.
Dial I/O
The DHT lookups, dials and transfers a call runs on come from the dial_io in its Opts, or else from macula and macula_content_transfer. A given dial_io has every function the call runs on, each at the arity its key takes, and may carry the other dial_io() functions; any other is refused with function_clause, in the caller. The option is for tests and for embedding direct dial; other callers leave it out.
Summary
Functions
As call/6 with no options.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section. Opts takes no option: realm_trust and verify_cert_chain are refused with {error, {removed_option, Key}} before anything is looked up.
As call_stream/6 with no options.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
Fetch MCID from the first of its announced providers whose fetch succeeds, within TimeoutMs, by the rules call/6 resolves by (see "Resolution" in the module doc): a provider whose fetch fails, including one whose bytes don't verify against MCID, is passed over for the next, and one that failed is tried again only when its announcement has changed. Fetch(Endpoint, Pinned, ConnectMs, RemainingMs) runs one fetch: Pinned is the dial trust override that pins the announcer, ConnectMs the time that provider gets to connect, RemainingMs what remains of the deadline; it returns {ok, Bytes} or {error, Reason}. macula_download supplies its own, so a cancel reaches whichever transfer is running.
As fetch_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
Fetch MCID from one of its providers, resolved via their signed content_announcements, and dialed directly. Same return shape as macula:get_content/2; resolve failures surface as {error, {unresolved, Reason}}, and once a provider's fetch has failed, the most recent failure is the result instead. TimeoutMs bounds the whole fetch: lookups, each provider's connect wait and the transfers. See fetch_content/4 for how providers are chosen, and the module doc's "Content" section for why it checks no provider authorization, unlike call/6. Only chunked content is discoverable this way — see macula:find_content_providers/2.
As get_content/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
As publish_advertisement/5 with no provider authorization.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, and ttl_ms. cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record and put Bytes there directly. Same return shape as macula:put_content/2; resolve failures surface as {error, {unresolved, Reason}}. TimeoutMs bounds the endpoint lookup and the connect wait (macula:put_content_station/5); the underlying block/manifest transfer has its own internal timeouts.
As put_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
Resolve MCID's provider via its signed content_announcement, asking the DHT again past a not-yet-replicated announcement for up to 10 seconds. Returns the announcement of the first provider that qualifies, {error, content_not_announced} when none has by then, or the last lookup's own error when that lookup failed. Deprecated: removed in 11.0.0. Use fetch_content/4, which also moves on to the next provider when a fetch fails.
As resolve_station_endpoint/3, within 10 seconds.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, station_endpoint_not_found (or the malformed record's reason) when a lookup answered, a failed lookup's own reason, or timeout.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
Types
-type dial_io() :: #{links => fun((macula:pool()) -> {ok, [map()]} | {error, term()}), put_record => fun((macula:pool(), map()) -> ok | {error, term()}), find_records => fun((macula:pool(), binary(), pos_integer()) -> {ok, [map()]} | {error, term()}), find_record => fun((macula:pool(), binary(), pos_integer()) -> {ok, map()} | {error, term()}), call_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), pos_integer(), map()) -> {ok, term()} | {error, term()}), call_stream_station => fun((macula:pool(), macula_client:seed(), <<_:256>>, macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}), put_content_station => fun((macula:pool(), macula_client:seed(), binary(), pos_integer(), map()) -> {ok, macula:mcid()} | {error, term()}), start_get_station => fun((macula:pool(), macula_client:seed(), macula:mcid(), pos_integer(), map()) -> {ok, pid()}), await => fun((pid(), timeout()) -> {ok, term()} | {error, term()}), cancel => fun((pid()) -> ok)}.
Functions
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000) -> {ok, term()} | {error, term()}.
As call/6 with no options.
-spec call(macula:pool(), macula:realm(), macula:procedure(), term(), 1..600000, map()) -> {ok, term()} | {error, term()}.
Resolve Procedure's provider and call it there directly. Same return shape as macula:call/5; resolve failures surface as {error, {unresolved, Reason}} so a caller can tell "nobody has advertised this via direct-dial yet" apart from a real call failure; once a candidate has failed before the CALL was sent, the most recent candidate's failure is the result instead, such as {error, not_connected}. TimeoutMs bounds resolution, each candidate's connect wait and the CALL itself (see "Resolution" in the module doc). An org namespaced procedure's authorization is checked against the realm key the pool pinned for Realm: see the module doc's "Trust model" section. Opts takes no option: realm_trust and verify_cert_chain are refused with {error, {removed_option, Key}} before anything is looked up.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map()) -> {ok, macula:stream()} | {error, term()}.
As call_stream/6 with no options.
-spec call_stream(macula:pool(), macula:realm(), macula:procedure(), term(), map(), map()) -> {ok, macula:stream()} | {error, term()}.
As call/6, but opens a stream (macula:call_stream_station/7's shape) instead of making a single-reply call, built on the exact same resolve+trust machinery — see the module doc. StreamOpts is forwarded to call_stream_station/7 alongside the resolved trust override (mode, owner, etc); its dial_timeout_ms (default 10_000, from 1 to 600_000 as a call's timeout) bounds resolution and each candidate's connect wait, and the stream itself keeps its own deadline. Opts takes no option: realm_trust and verify_cert_chain in it are refused as call/6 refuses them.
-spec fetch_content(macula:pool(), macula:mcid(), pos_integer(), fun((binary(), map(), pos_integer(), pos_integer()) -> {ok, binary()} | {error, term()})) -> {ok, binary()} | {error, term()}.
Fetch MCID from the first of its announced providers whose fetch succeeds, within TimeoutMs, by the rules call/6 resolves by (see "Resolution" in the module doc): a provider whose fetch fails, including one whose bytes don't verify against MCID, is passed over for the next, and one that failed is tried again only when its announcement has changed. Fetch(Endpoint, Pinned, ConnectMs, RemainingMs) runs one fetch: Pinned is the dial trust override that pins the announcer, ConnectMs the time that provider gets to connect, RemainingMs what remains of the deadline; it returns {ok, Bytes} or {error, Reason}. macula_download supplies its own, so a cancel reaches whichever transfer is running.
-spec fetch_content(macula:pool(), macula:mcid(), pos_integer(), fun((binary(), map(), pos_integer(), pos_integer()) -> {ok, binary()} | {error, term()}), map()) -> {ok, binary()} | {error, term()}.
As fetch_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
-spec get_content(macula:pool(), macula:mcid(), pos_integer()) -> {ok, binary()} | {error, term()}.
Fetch MCID from one of its providers, resolved via their signed content_announcements, and dialed directly. Same return shape as macula:get_content/2; resolve failures surface as {error, {unresolved, Reason}}, and once a provider's fetch has failed, the most recent failure is the result instead. TimeoutMs bounds the whole fetch: lookups, each provider's connect wait and the transfers. See fetch_content/4 for how providers are chosen, and the module doc's "Content" section for why it checks no provider authorization, unlike call/6. Only chunked content is discoverable this way — see macula:find_content_providers/2.
-spec get_content(macula:pool(), macula:mcid(), pos_integer(), map()) -> {ok, binary()} | {error, term()}.
As get_content/3, on the dial_io in Opts (see "Dial I/O" in the module doc).
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key()) -> ok | {error, term()}.
As publish_advertisement/5 with no provider authorization.
-spec publish_advertisement(macula:pool(), macula:realm(), macula:procedure(), macula_node_keys:node_key(), map()) -> ok | {error, term()}.
Publish a signed procedure_advertisement for Procedure, naming Pool's currently-connected station as the serving station. NodeIdentity signs it, and its node_id is the advertiser: it must be the node identity key Pool was started with, since a caller targets that node_id and the station knows the pool's connection by it. Opts may include authorization, the provider authorization an org namespaced procedure needs (D25 item 6), as #{org_directory => Wire, procedure_delegation => Wire}, and ttl_ms. cert_chain, a 10.x option authorization replaces, is refused with {error, {removed_option, cert_chain}} before anything is read or put.
-spec put_content(macula:pool(), <<_:256>>, binary(), pos_integer()) -> {ok, macula:mcid()} | {error, term()}.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record and put Bytes there directly. Same return shape as macula:put_content/2; resolve failures surface as {error, {unresolved, Reason}}. TimeoutMs bounds the endpoint lookup and the connect wait (macula:put_content_station/5); the underlying block/manifest transfer has its own internal timeouts.
-spec put_content(macula:pool(), <<_:256>>, binary(), pos_integer(), map()) -> {ok, macula:mcid()} | {error, term()}.
As put_content/4, on the dial_io in Opts (see "Dial I/O" in the module doc).
The first option in Opts that 11.0.0 removed from a call or an advertisement, as {removed_option, Key}, or none. On a call, the realm keys the pool pins replace verify_cert_chain and realm_trust; on an advertisement, authorization replaces cert_chain.
-spec resolve_content_provider(macula:pool(), macula:mcid()) -> {ok, map()} | {error, term()}.
Resolve MCID's provider via its signed content_announcement, asking the DHT again past a not-yet-replicated announcement for up to 10 seconds. Returns the announcement of the first provider that qualifies, {error, content_not_announced} when none has by then, or the last lookup's own error when that lookup failed. Deprecated: removed in 11.0.0. Use fetch_content/4, which also moves on to the next provider when a fetch fails.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, within 10 seconds.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer()) -> {ok, binary()} | {error, term()}.
Resolve Station's dialable quic:// URL from its own signed station_endpoint record, verifying the record's signer is exactly Station and asking again past an absent, expired or malformed record, or a failed lookup, until TimeoutMs has passed — the same discipline call/6 applies once it has a procedure's serving_station. The error is, in this order, station_endpoint_not_found (or the malformed record's reason) when a lookup answered, a failed lookup's own reason, or timeout.
-spec resolve_station_endpoint(macula:pool(), <<_:256>>, pos_integer(), map()) -> {ok, binary()} | {error, term()}.
As resolve_station_endpoint/3, on the dial_io in Opts (see "Dial I/O" in the module doc).