macula_manifest (macula v11.1.0)
View SourceFixed-size chunking, Merkle-root computation, and manifest construction for content larger than one storage block.
Mirrors macula-station's macula_manifest / macula_content_chunker / macula_content_hasher BYTE-FOR-BYTE: same MCID format, same default chunk size (256 KiB), same Merkle fold, same canonical-CBOR MCID derivation, same manifest wire shape. This is deliberate, not incidental — the SDK puts a manifest via the station's existing (unmodified) _content.put_manifest / _content.get_manifest RPCs, so the two sides must agree on the algorithm bit-for-bit. Both hash with SHA-384 through OTP crypto and use the same deterministic CBOR encoder (macula_record_cbor, SDK-owned; the station's manifest module calls it directly), so this is a faithful client-side port, not a re-derivation.
MCID format (50 bytes): <<Tag:8, Codec:8, Hash:48/binary>>. The tag names the hash; the post-quantum format has only tag 2, SHA-384 (D24). ?CODEC_RAW (16#55) addresses a single chunk (or a whole blob that fits in one chunk — see the module doc on macula:put_content/2). ?CODEC_MANIFEST (16#56) addresses a manifest describing many chunks.
Summary
Functions
The MCID a chunk at Index is stored/fetched under: tag 2, codec raw, and the chunk's SHA-384 hash. The station derives this same value independently when serving the chunk, so both sides agree on its address without exchanging it, and a fetched chunk is checked against it by macula_content_transfer:verify_block_hash/2.
Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), so a caller can upload each chunk (via _content.put_block) and then the manifest (via _content.put_manifest). Options
Read a manifest as it arrives over _content.get_manifest: the station stores + returns the map exactly as its RPC layer decoded it, with no dedicated re-encode/decode round trip on either side — so the shape depends on the general CALL-result codec, not the canonical {text,_} record shape. Robust to atom keys, to binary-string keys (mirroring macula_record:payload_field/2), and to {text, Bin} keys: the frame decoder resolves a key to an atom only when that atom already exists, so in a node that has not yet loaded this module the field names arrive as text. A name or hash algorithm sent as text is read as its binary value. A manifest without an mcid, whose chunks are not a list of maps, or that does not name sha384 as its hash algorithm, is {error, invalid_manifest}. So is one that does not describe whole content (whole/1), before any caller sizes or counts anything by it.
Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way. A manifest whose chunk size is not a positive integer is {error, invalid_manifest}, so the re-chunking always ends.
Check that Manifest describes the content Mcid names: the MCID recomputed from the manifest's canonical fields (name, size, chunk_size, chunk_count, hash_algorithm and root_hash) must equal Mcid. The manifest's own mcid field is not consulted; a sender can put anything there. A manifest missing a canonical field, whose name is not valid UTF-8 text, or whose hash algorithm is not one this module knows, does not describe Mcid either.
Types
-type algorithm() :: sha384.
-type chunk_info() :: #{index := non_neg_integer(), offset := non_neg_integer(), size := pos_integer(), hash := binary()}.
-type manifest() :: #{mcid := mcid(), version := pos_integer(), name := binary(), size := non_neg_integer(), created := non_neg_integer(), chunk_size := pos_integer(), chunk_count := non_neg_integer(), hash_algorithm := algorithm(), root_hash := binary(), chunks := [chunk_info()]}.
-type mcid() :: <<_:400>>.
Functions
-spec chunk_mcid(manifest(), non_neg_integer()) -> {ok, mcid()} | {error, invalid_index}.
The MCID a chunk at Index is stored/fetched under: tag 2, codec raw, and the chunk's SHA-384 hash. The station derives this same value independently when serving the chunk, so both sides agree on its address without exchanging it, and a fetched chunk is checked against it by macula_content_transfer:verify_block_hash/2.
Split Data into fixed-size chunks and build its manifest. Returns the manifest and the chunk bytes in order (index 0 first), so a caller can upload each chunk (via _content.put_block) and then the manifest (via _content.put_manifest). Options:
name— content name (default<<"unnamed">>)chunk_size— bytes per chunk, a positive integer (defaultdefault_chunk_size/0)hash_algorithm:sha384, the only one and the default
-spec default_chunk_size() -> pos_integer().
Read a manifest as it arrives over _content.get_manifest: the station stores + returns the map exactly as its RPC layer decoded it, with no dedicated re-encode/decode round trip on either side — so the shape depends on the general CALL-result codec, not the canonical {text,_} record shape. Robust to atom keys, to binary-string keys (mirroring macula_record:payload_field/2), and to {text, Bin} keys: the frame decoder resolves a key to an atom only when that atom already exists, so in a node that has not yet loaded this module the field names arrive as text. A name or hash algorithm sent as text is read as its binary value. A manifest without an mcid, whose chunks are not a list of maps, or that does not name sha384 as its hash algorithm, is {error, invalid_manifest}. So is one that does not describe whole content (whole/1), before any caller sizes or counts anything by it.
-spec verify(manifest(), binary()) -> ok | {error, size_mismatch | root_hash_mismatch | invalid_manifest}.
Verify reassembled Data against Manifest: size, then a fresh Merkle root over Data re-chunked the same way. A manifest whose chunk size is not a positive integer is {error, invalid_manifest}, so the re-chunking always ends.
Check that Manifest describes the content Mcid names: the MCID recomputed from the manifest's canonical fields (name, size, chunk_size, chunk_count, hash_algorithm and root_hash) must equal Mcid. The manifest's own mcid field is not consulted; a sender can put anything there. A manifest missing a canonical field, whose name is not valid UTF-8 text, or whose hash algorithm is not one this module knows, does not describe Mcid either.