diff --git a/bdasl.src.html b/bdasl.src.html index 64f60a0..174df38 100644 --- a/bdasl.src.html +++ b/bdasl.src.html @@ -1,140 +1,140 @@ -
- - -- BDASL extends DASL CIDs with a new hash type that works better - for large files but isn't available by default in browsers, and - therefore not an appropriate option in most situations. -
-- BDASL extends DASL CIDs by adding BLAKE3 support ([[blake3]]). - BLAKE3 is a powerful hashing framework that works well for - progressive verification of large streams. Unfortunately, it - isn't available in browser (and neither is streaming hashing in - general) which makes it inappriopriate for inclusion as the - primary hash function in DASL CIDs. -
-- It is recommended to avoid using BDASL CIDs in arbitrary open - environments, and rather to focus on using such CIDs in specific - cases in which participants are likely to know how to handle - them. -
-- All the parsing works the same as for DASL CIDs ([[cid]]) with - one modification. -
-
- In the steps to
- decode a CID, the
- hash type may also be equal to
- 0x1e (BLAKE3) ([[blake3]]).
-
- Section 6.4 of the [[blake3]] spec outlines - Streaming Verification, a process that allows the sender - and receiver of a CID to incrementally verify data as it is - being transferred. Applying this technique to CIDs is the core - benefit of BDASL, which is well-suited to both fetching byte - ranges within a CID, and verifying data where the cost of - faulty transmission will impact the performance of an application. - Verified streaming incurs minimal overhead on payloads of all - sizes, and scales linearly as a small percentage of the size of - the CID being verified. -
-- Streaming verification rounds to the nearest kilobyte for - verification. For more details & a reference implementation, see - [[iroh-blobs]]. -
-
- [[rfc9110]] Defines HTTP Range requests for fetching a single
- contiguous set of bytes from a larger source held by a server.
- Range requests use a Range header:
- Range: bytes={start}-{end}
- Both start and end values are optional, and when missing indicate
- the first and late byte, respectively.
-
- In this context the HTTP server is acting as a trusted gateway, - performing verificaiton on behalf of the user. The advantage of - delegating trust here is interop with standard HTTP semantics. - For trustless fetching, use verified streaming directly. -
-
- For a given normalized byte range
- (start: Option<u64>, end: Option<u64>),
- that byte range is mapped to a chunk range, which is the
- set of chunks that fully contains the set of bytes in the
- HTTP range request.
- the corresponding chunk range is calculated as:
-
- Start Chunk: -
start is a number s, use ⌈s / 1024⌉ (ceiling division)start is empty, there is no lower bound, so begin the request from byte 0.- End Chunk: -
stop is a number s, use ⌈s / 1024⌉ (ceiling division)stop is empty, no upper bound, so request to the end of the byte array- From here, construct a verified range request in accordance with - the verified streaming protocol, as chunks arrive, check if the - chunk responded intersects with either start or end chunks. -
- For a request of Range: bytes=512-1535
- two chunks will be retrieved, chunk 0 and chunk 1.
-
-
[512, 1024)chunk_data[512..1024] (512 bytes)[1024, 1536)chunk_data[0..512] (512 bytes)+ BDASL extends DASL CIDs with a new hash type that works better + for large files but isn't available by default in browsers, and + therefore not an appropriate option in most situations. +
++ BDASL extends DASL CIDs by adding BLAKE3 support ([[blake3]]). + BLAKE3 is a powerful hashing framework that works well for + progressive verification of large streams. Unfortunately, it + isn't available in browser (and neither is streaming hashing in + general) which makes it inappriopriate for inclusion as the + primary hash function in DASL CIDs. +
++ It is recommended to avoid using BDASL CIDs in arbitrary open + environments, and rather to focus on using such CIDs in specific + cases in which participants are likely to know how to handle + them. +
++ All the parsing works the same as for DASL CIDs ([[cid]]) with + one modification. +
+
+ In the steps to
+ decode a CID, the
+ hash type may also be equal to
+ 0x1e (BLAKE3) ([[blake3]]).
+
+ Section 6.4 of the [[blake3]] spec outlines + Streaming Verification, a process that allows the sender + and receiver of a CID to incrementally verify data as it is + being transferred. Applying this technique to CIDs is the core + benefit of BDASL, which is well-suited to both fetching byte + ranges within a CID, and verifying data where the cost of + faulty transmission will impact the performance of an application. + Verified streaming incurs minimal overhead on payloads of all + sizes, and scales linearly as a small percentage of the size of + the CID being verified. +
++ Streaming verification rounds to the nearest kilobyte for + verification. For more details & a reference implementation, see + [[iroh-blobs]]. +
+
+ [[rfc9110]] Defines HTTP Range requests for fetching a single
+ contiguous set of bytes from a larger source held by a server.
+ Range requests use a Range header:
+ Range: bytes={start}-{end}
+ Both start and end values are optional, and when missing indicate
+ the first and late byte, respectively.
+
+ In this context the HTTP server is acting as a trusted gateway, + performing verificaiton on behalf of the user. The advantage of + delegating trust here is interop with standard HTTP semantics. + For trustless fetching, use verified streaming directly. +
+
+ For a given normalized byte range
+ (start: Option<u64>, end: Option<u64>),
+ that byte range is mapped to a chunk range, which is the
+ set of chunks that fully contains the set of bytes in the
+ HTTP range request.
+ the corresponding chunk range is calculated as:
+
+ Start Chunk: +
start is a number s, use ⌈s / 1024⌉ (ceiling division)start is empty, there is no lower bound, so begin the request from byte 0.+ End Chunk: +
stop is a number s, use ⌈s / 1024⌉ (ceiling division)stop is empty, no upper bound, so request to the end of the byte array+ From here, construct a verified range request in accordance with + the verified streaming protocol, as chunks arrive, check if the + chunk responded intersects with either start or end chunks. +
+ For a request of Range: bytes=512-1535
+ two chunks will be retrieved, chunk 0 and chunk 1.
+
+
[512, 1024)chunk_data[512..1024] (512 bytes)[1024, 1536)chunk_data[0..512] (512 bytes)