diff --git a/bdasl.src.html b/bdasl.src.html index 64f60a0..174df38 100644 --- a/bdasl.src.html +++ b/bdasl.src.html @@ -1,140 +1,140 @@ - - - - Big DASL (BDASL) - - -
-

- BDASL extends DASL CIDs with a new hash type that works better - for large files but isn't available by default in browsers, and - therefore not an appropriate option in most situations. -

-
-
-

Introduction

-

- BDASL extends DASL CIDs by adding BLAKE3 support ([[blake3]]). - BLAKE3 is a powerful hashing framework that works well for - progressive verification of large streams. Unfortunately, it - isn't available in browser (and neither is streaming hashing in - general) which makes it inappriopriate for inclusion as the - primary hash function in DASL CIDs. -

-

- It is recommended to avoid using BDASL CIDs in arbitrary open - environments, and rather to focus on using such CIDs in specific - cases in which participants are likely to know how to handle - them. -

-
-
-

Parsing BDASL CIDs

-

- All the parsing works the same as for DASL CIDs ([[cid]]) with - one modification. -

-

- In the steps to - decode a CID, the - hash type may also be equal to - 0x1e (BLAKE3) ([[blake3]]). -

-
-
-

Streaming Verification

-

- Section 6.4 of the [[blake3]] spec outlines - Streaming Verification, a process that allows the sender - and receiver of a CID to incrementally verify data as it is - being transferred. Applying this technique to CIDs is the core - benefit of BDASL, which is well-suited to both fetching byte - ranges within a CID, and verifying data where the cost of - faulty transmission will impact the performance of an application. - Verified streaming incurs minimal overhead on payloads of all - sizes, and scales linearly as a small percentage of the size of - the CID being verified. -

-

- Streaming verification rounds to the nearest kilobyte for - verification. For more details & a reference implementation, see - [[iroh-blobs]]. -

-
-
-

Verifying HTTP Range Requests

-

- [[rfc9110]] Defines HTTP Range requests for fetching a single - contiguous set of bytes from a larger source held by a server. - Range requests use a Range header: - Range: bytes={start}-{end} - Both start and end values are optional, and when missing indicate - the first and late byte, respectively. -

-

- In this context the HTTP server is acting as a trusted gateway, - performing verificaiton on behalf of the user. The advantage of - delegating trust here is interop with standard HTTP semantics. - For trustless fetching, use verified streaming directly. -

-

Fufilling requests

-

- For a given normalized byte range - (start: Option<u64>, end: Option<u64>), - that byte range is mapped to a chunk range, which is the - set of chunks that fully contains the set of bytes in the - HTTP range request. - the corresponding chunk range is calculated as: -

-

- Start Chunk: -

-

-

- End Chunk: -

-

-

- From here, construct a verified range request in accordance with - the verified streaming protocol, as chunks arrive, check if the - chunk responded intersects with either start or end chunks. -

-

-

Example

-

- For a request of Range: bytes=512-1535 - two chunks will be retrieved, chunk 0 and chunk 1. -

-

-

-

-
- + + + + Big DASL (BDASL) + + +
+

+ BDASL extends DASL CIDs with a new hash type that works better + for large files but isn't available by default in browsers, and + therefore not an appropriate option in most situations. +

+
+
+

Introduction

+

+ BDASL extends DASL CIDs by adding BLAKE3 support ([[blake3]]). + BLAKE3 is a powerful hashing framework that works well for + progressive verification of large streams. Unfortunately, it + isn't available in browser (and neither is streaming hashing in + general) which makes it inappriopriate for inclusion as the + primary hash function in DASL CIDs. +

+

+ It is recommended to avoid using BDASL CIDs in arbitrary open + environments, and rather to focus on using such CIDs in specific + cases in which participants are likely to know how to handle + them. +

+
+
+

Parsing BDASL CIDs

+

+ All the parsing works the same as for DASL CIDs ([[cid]]) with + one modification. +

+

+ In the steps to + decode a CID, the + hash type may also be equal to + 0x1e (BLAKE3) ([[blake3]]). +

+
+
+

Streaming Verification

+

+ Section 6.4 of the [[blake3]] spec outlines + Streaming Verification, a process that allows the sender + and receiver of a CID to incrementally verify data as it is + being transferred. Applying this technique to CIDs is the core + benefit of BDASL, which is well-suited to both fetching byte + ranges within a CID, and verifying data where the cost of + faulty transmission will impact the performance of an application. + Verified streaming incurs minimal overhead on payloads of all + sizes, and scales linearly as a small percentage of the size of + the CID being verified. +

+

+ Streaming verification rounds to the nearest kilobyte for + verification. For more details & a reference implementation, see + [[iroh-blobs]]. +

+
+
+

Verifying HTTP Range Requests

+

+ [[rfc9110]] Defines HTTP Range requests for fetching a single + contiguous set of bytes from a larger source held by a server. + Range requests use a Range header: + Range: bytes={start}-{end} + Both start and end values are optional, and when missing indicate + the first and late byte, respectively. +

+

+ In this context the HTTP server is acting as a trusted gateway, + performing verificaiton on behalf of the user. The advantage of + delegating trust here is interop with standard HTTP semantics. + For trustless fetching, use verified streaming directly. +

+

Fufilling requests

+

+ For a given normalized byte range + (start: Option<u64>, end: Option<u64>), + that byte range is mapped to a chunk range, which is the + set of chunks that fully contains the set of bytes in the + HTTP range request. + the corresponding chunk range is calculated as: +

+

+ Start Chunk: +

+

+

+ End Chunk: +

+

+

+ From here, construct a verified range request in accordance with + the verified streaming protocol, as chunks arrive, check if the + chunk responded intersects with either start or end chunks. +

+

+

Example

+

+ For a request of Range: bytes=512-1535 + two chunks will be retrieved, chunk 0 and chunk 1. +

+

+

+

+
+