commit cd996d924ceeb26dcdc8af5b840207e42023b14e · anil.recoil.org/ocaml-html5rw

+530 -87

lib/dom/node.mli

··· 5 5 6 6 (** HTML5 DOM Node Types and Operations 7 7 8 - This module provides the DOM node representation used by the HTML5 parser. 9 - Nodes form a tree structure representing HTML documents. The type follows 10 - the WHATWG HTML5 specification for document structure. 8 + This module provides the DOM (Document Object Model) node representation 9 + used by the HTML5 parser. The DOM is a programming interface that 10 + represents an HTML document as a tree of nodes, where each node represents 11 + part of the document (an element, text content, comment, etc.). 12 + 13 + {2 What is the DOM?} 14 + 15 + When an HTML parser processes markup like [<p>Hello <b>world</b></p>], it 16 + doesn't store the text directly. Instead, it builds a tree structure in 17 + memory: 18 + 19 + {v 20 + Document 21 + └── html 22 + └── body 23 + └── p 24 + ├── #text "Hello " 25 + └── b 26 + └── #text "world" 27 + v} 28 + 29 + This tree is the DOM. Each box in the tree is a {i node}. Programs can 30 + traverse and modify this tree to read or change the document. 31 + 32 + @see <https://html.spec.whatwg.org/multipage/dom.html> 33 + WHATWG: The elements of HTML (DOM chapter) 11 34 12 35 {2 Node Types} 13 36 14 37 The HTML5 DOM includes several node types, all represented by the same 15 38 record type with different field usage: 16 39 17 - - {b Element nodes}: Regular HTML elements like [<div>], [<p>], [<span>] 18 - - {b Text nodes}: Text content within elements 19 - - {b Comment nodes}: HTML comments [] 20 - - {b Document nodes}: The root node representing the entire document 21 - - {b Document fragment nodes}: A lightweight container (used for templates) 22 - - {b Doctype nodes}: The [<!DOCTYPE html>] declaration 40 + - {b Element nodes}: HTML elements like [<div>], [<p>], [<a href="...">]. 41 + Elements are the building blocks of HTML documents. They can have 42 + attributes and contain other nodes. 43 + 44 + - {b Text nodes}: The actual text content within elements. For example, 45 + in [<p>Hello</p>], "Hello" is a text node that is a child of the [<p>] 46 + element. 47 + 48 + - {b Comment nodes}: HTML comments written as []. 49 + Comments are preserved in the DOM but not rendered. 50 + 51 + - {b Document nodes}: The root of the entire document tree. Every HTML 52 + document has exactly one Document node at the top. 53 + 54 + - {b Document fragment nodes}: Lightweight containers that hold a 55 + collection of nodes without a parent. Used for efficient batch DOM 56 + operations and [<template>] element contents. 57 + 58 + - {b Doctype nodes}: The [<!DOCTYPE html>] declaration at the start of 59 + HTML5 documents. This declaration tells browsers to render the page 60 + in standards mode. 61 + 62 + @see <https://html.spec.whatwg.org/multipage/dom.html#kinds-of-content> 63 + WHATWG: Kinds of content 23 64 24 65 {2 Namespaces} 25 66 26 - Elements can belong to different namespaces: 27 - - [None] or [Some "html"]: HTML namespace (default) 28 - - [Some "svg"]: SVG namespace for embedded SVG content 29 - - [Some "mathml"]: MathML namespace for mathematical notation 67 + HTML5 can embed content from other XML vocabularies. Elements belong to 68 + one of three {i namespaces}: 69 + 70 + - {b HTML namespace} ([None] or implicit): Standard HTML elements like 71 + [<div>], [<p>], [<table>]. This is the default for all elements. 72 + 73 + - {b SVG namespace} ([Some "svg"]): Scalable Vector Graphics for drawing. 74 + When the parser encounters an [<svg>] tag, all elements inside it 75 + (like [<rect>], [<circle>], [<path>]) are placed in the SVG namespace. 76 + 77 + - {b MathML namespace} ([Some "mathml"]): Mathematical Markup Language 78 + for equations. When the parser encounters a [<math>] tag, elements 79 + inside it are placed in the MathML namespace. 80 + 81 + The parser automatically switches namespaces when entering and leaving 82 + these foreign content islands. 30 83 31 - The parser automatically switches namespaces when encountering [<svg>] 32 - or [<math>] elements, as specified by the HTML5 algorithm. 84 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign> 85 + WHATWG: Parsing foreign content 33 86 34 87 {2 Tree Structure} 35 88 36 89 Nodes form a bidirectional tree: each node has a list of children and 37 - an optional parent reference. Modification functions maintain these 38 - references automatically. 90 + an optional parent reference. Modification functions in this module 91 + maintain these references automatically. 92 + 93 + The tree is always well-formed: a node can only have one parent, and 94 + circular references are not possible. 39 95 *) 40 96 41 97 (** {1 Types} *) 42 98 43 99 (** Information associated with a DOCTYPE node. 44 100 45 - In HTML5, the DOCTYPE is primarily used for quirks mode detection. 46 - Most modern HTML5 documents use [<!DOCTYPE html>] which results in 47 - all fields being [None] or the name being [Some "html"]. 101 + The {i document type declaration} (DOCTYPE) tells browsers what version 102 + of HTML the document uses. In HTML5, the standard declaration is simply: 103 + 104 + {v <!DOCTYPE html> v} 105 + 106 + This minimal DOCTYPE triggers {i standards mode} (no quirks). The DOCTYPE 107 + can optionally include a public identifier and system identifier for 108 + legacy compatibility with SGML-based tools, but these are rarely used 109 + in modern HTML5 documents. 110 + 111 + {b Historical context:} In HTML4 and XHTML, DOCTYPEs were verbose and 112 + referenced DTD files. For example: 113 + {v <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" 114 + "http://www.w3.org/TR/html4/strict.dtd"> v} 115 + 116 + HTML5 simplified this to just [<!DOCTYPE html>] because: 117 + - Browsers never actually fetched or validated against DTDs 118 + - The DOCTYPE's only real purpose is triggering standards mode 119 + - A minimal DOCTYPE achieves this goal 120 + 121 + {b Field meanings:} 122 + - [name]: The document type name, almost always ["html"] for HTML documents 123 + - [public_id]: A public identifier (legacy); [None] for HTML5 124 + - [system_id]: A system identifier/URL (legacy); [None] for HTML5 48 125 126 + @see <https://html.spec.whatwg.org/multipage/syntax.html#the-doctype> 127 + WHATWG: The DOCTYPE 49 128 @see <https://html.spec.whatwg.org/multipage/parsing.html#the-initial-insertion-mode> 50 - The WHATWG specification for DOCTYPE handling 129 + WHATWG: DOCTYPE handling during parsing 51 130 *) 52 131 type doctype_data = { 53 132 name : string option; (** The DOCTYPE name, e.g., "html" *) ··· 57 136 58 137 (** Quirks mode setting for the document. 59 138 60 - Quirks mode affects CSS layout behavior for backwards compatibility with 61 - old web content. The HTML5 parser determines quirks mode based on the 62 - DOCTYPE declaration. 139 + {i Quirks mode} is a browser rendering mode that emulates bugs and 140 + non-standard behaviors from older browsers (primarily Internet Explorer 5). 141 + Modern HTML5 documents should always render in {i standards mode} 142 + (no quirks) for consistent, predictable behavior. 143 + 144 + The HTML5 parser determines quirks mode based on the DOCTYPE declaration: 145 + 146 + - {b No_quirks} (Standards mode): The document renders according to modern 147 + HTML5 and CSS specifications. This is triggered by [<!DOCTYPE html>]. 148 + CSS box model, table layout, and other features work as specified. 63 149 64 - - [No_quirks]: Standards mode - full HTML5/CSS3 behavior 65 - - [Quirks]: Full quirks mode - emulates legacy browser behavior 66 - - [Limited_quirks]: Almost standards mode - limited quirks for specific cases 150 + - {b Quirks} (Full quirks mode): The document renders with legacy browser 151 + bugs emulated. This happens when: 152 + {ul 153 + {- DOCTYPE is missing entirely} 154 + {- DOCTYPE has certain legacy public identifiers} 155 + {- DOCTYPE has the wrong format}} 156 + 157 + In quirks mode, many CSS properties behave differently: 158 + {ul 159 + {- Tables don't inherit font properties} 160 + {- Box model uses non-standard width calculations} 161 + {- Certain CSS selectors don't work correctly}} 162 + 163 + - {b Limited_quirks} (Almost standards mode): A middle ground that applies 164 + only a few specific quirks, primarily affecting table cell vertical 165 + sizing. Triggered by XHTML DOCTYPEs and certain HTML4 DOCTYPEs. 166 + 167 + {b Recommendation:} Always use [<!DOCTYPE html>] at the start of HTML5 168 + documents to ensure {b No_quirks} mode. 67 169 68 - @see <https://quirks.spec.whatwg.org/> The Quirks Mode specification 170 + @see <https://quirks.spec.whatwg.org/> 171 + Quirks Mode Standard - detailed specification 172 + @see <https://html.spec.whatwg.org/multipage/parsing.html#the-initial-insertion-mode> 173 + WHATWG: How the parser determines quirks mode 69 174 *) 70 175 type quirks_mode = No_quirks | Quirks | Limited_quirks 71 176 ··· 73 178 74 179 All node types use the same record structure. The [name] field determines 75 180 the node type: 76 - - Element: the tag name (e.g., "div", "p") 181 + - Element: the tag name (e.g., "div", "p", "span") 77 182 - Text: "#text" 78 183 - Comment: "#comment" 79 184 - Document: "#document" 80 185 - Document fragment: "#document-fragment" 81 186 - Doctype: "!doctype" 82 187 83 - {3 Field Usage by Node Type} 188 + {3 Understanding Node Fields} 189 + 190 + Different node types use different combinations of fields: 84 191 85 192 {v 86 193 Node Type | name | namespace | attrs | data | template_content | doctype ··· 92 199 Document Fragment | "#document-frag" | No | No | No | No | No 93 200 Doctype | "!doctype" | No | No | No | No | Yes 94 201 v} 202 + 203 + {3 Element Tag Names} 204 + 205 + For element nodes, the [name] field contains the lowercase tag name. 206 + HTML5 defines many elements with specific meanings: 207 + 208 + {b Structural elements:} [html], [head], [body], [header], [footer], 209 + [main], [nav], [article], [section], [aside] 210 + 211 + {b Text content:} [p], [div], [span], [h1]-[h6], [pre], [blockquote] 212 + 213 + {b Lists:} [ul], [ol], [li], [dl], [dt], [dd] 214 + 215 + {b Tables:} [table], [tr], [td], [th], [thead], [tbody], [tfoot] 216 + 217 + {b Forms:} [form], [input], [button], [select], [textarea], [label] 218 + 219 + {b Media:} [img], [audio], [video], [canvas], [svg] 220 + 221 + @see <https://html.spec.whatwg.org/multipage/indices.html#elements-3> 222 + WHATWG: Index of HTML elements 223 + 224 + {3 Void Elements} 225 + 226 + Some elements are {i void elements} - they cannot have children and have 227 + no end tag. These include: [area], [base], [br], [col], [embed], [hr], 228 + [img], [input], [link], [meta], [source], [track], [wbr]. 229 + 230 + @see <https://html.spec.whatwg.org/multipage/syntax.html#void-elements> 231 + WHATWG: Void elements 232 + 233 + {3 The Template Element} 234 + 235 + The [<template>] element is special: its children are not rendered 236 + directly but stored in a separate document fragment accessible via 237 + the [template_content] field. Templates are used for client-side 238 + templating where content is cloned and inserted via JavaScript. 239 + 240 + @see <https://html.spec.whatwg.org/multipage/scripting.html#the-template-element> 241 + WHATWG: The template element 95 242 *) 96 243 type node = { 97 244 mutable name : string; 98 - (** Tag name for elements, or special name for other node types *) 245 + (** Tag name for elements, or special name for other node types. 246 + 247 + For elements, this is the lowercase tag name (e.g., "div", "span"). 248 + For other node types, use the constants {!document_name}, 249 + {!text_name}, {!comment_name}, etc. *) 99 250 100 251 mutable namespace : string option; 101 - (** Element namespace: [None] for HTML, [Some "svg"], [Some "mathml"] *) 252 + (** Element namespace: [None] for HTML, [Some "svg"], [Some "mathml"]. 253 + 254 + Most elements are in the HTML namespace ([None]). The SVG and MathML 255 + namespaces are only used when content appears inside [<svg>] or 256 + [<math>] elements respectively. 257 + 258 + @see <https://html.spec.whatwg.org/multipage/dom.html#elements-in-the-dom> 259 + WHATWG: Elements in the DOM *) 102 260 103 261 mutable attrs : (string * string) list; 104 - (** Element attributes as (name, value) pairs *) 262 + (** Element attributes as (name, value) pairs. 263 + 264 + Attributes provide additional information about elements. Common 265 + global attributes include: 266 + - [id]: Unique identifier for the element 267 + - [class]: Space-separated list of CSS class names 268 + - [style]: Inline CSS styles 269 + - [title]: Advisory text (shown as tooltip) 270 + - [lang]: Language of the element's content 271 + - [hidden]: Whether the element should be hidden 272 + 273 + Element-specific attributes include: 274 + - [href] on [<a>]: The link destination URL 275 + - [src] on [<img>]: The image source URL 276 + - [type] on [<input>]: The input control type 277 + - [disabled] on form controls: Whether the control is disabled 278 + 279 + In HTML5, attribute names are case-insensitive and are normalized 280 + to lowercase by the parser. 281 + 282 + @see <https://html.spec.whatwg.org/multipage/dom.html#global-attributes> 283 + WHATWG: Global attributes 284 + @see <https://html.spec.whatwg.org/multipage/indices.html#attributes-3> 285 + WHATWG: Index of attributes *) 105 286 106 287 mutable children : node list; 107 - (** Child nodes in document order *) 288 + (** Child nodes in document order. 289 + 290 + For most elements, this list contains the nested elements and text. 291 + For void elements (like [<br>], [<img>]), this is always empty. 292 + For [<template>] elements, the actual content is in 293 + [template_content], not here. *) 108 294 109 295 mutable parent : node option; 110 - (** Parent node, [None] for root nodes *) 296 + (** Parent node, [None] for root nodes. 297 + 298 + Every node except the Document node has a parent. This back-reference 299 + enables traversing up the tree. *) 111 300 112 301 mutable data : string; 113 - (** Text content for text and comment nodes *) 302 + (** Text content for text and comment nodes. 303 + 304 + For text nodes, this contains the actual text. For comment nodes, 305 + this contains the comment text (without the [] 306 + delimiters). For other node types, this field is empty. *) 114 307 115 308 mutable template_content : node option; 116 - (** Document fragment for [<template>] element contents *) 309 + (** Document fragment for [<template>] element contents. 310 + 311 + The [<template>] element holds "inert" content that is not 312 + rendered but can be cloned and inserted elsewhere. This field 313 + contains a document fragment with the template's content. 314 + 315 + For non-template elements, this is [None]. 316 + 317 + @see <https://html.spec.whatwg.org/multipage/scripting.html#the-template-element> 318 + WHATWG: The template element *) 117 319 118 320 mutable doctype : doctype_data option; 119 - (** DOCTYPE information for doctype nodes *) 321 + (** DOCTYPE information for doctype nodes. 322 + 323 + Only doctype nodes use this field; for all other nodes it is [None]. *) 120 324 } 121 325 122 326 (** {1 Node Name Constants} ··· 126 330 *) 127 331 128 332 val document_name : string 129 - (** ["#document"] - name for document nodes *) 333 + (** ["#document"] - name for document nodes. 334 + 335 + The Document node is the root of every HTML document tree. It represents 336 + the entire document and is the parent of the [<html>] element. 337 + 338 + @see <https://html.spec.whatwg.org/multipage/dom.html#document> 339 + WHATWG: The Document object *) 130 340 131 341 val document_fragment_name : string 132 - (** ["#document-fragment"] - name for document fragment nodes *) 342 + (** ["#document-fragment"] - name for document fragment nodes. 343 + 344 + Document fragments are lightweight container nodes used to hold a 345 + collection of nodes without a parent document. They are used: 346 + - To hold [<template>] element contents 347 + - As results of fragment parsing (innerHTML) 348 + - For efficient batch DOM operations 349 + 350 + @see <https://dom.spec.whatwg.org/#documentfragment> 351 + DOM Standard: DocumentFragment *) 133 352 134 353 val text_name : string 135 - (** ["#text"] - name for text nodes *) 354 + (** ["#text"] - name for text nodes. 355 + 356 + Text nodes contain the character data within elements. When the 357 + parser encounters text between tags like [<p>Hello world</p>], 358 + it creates a text node with data ["Hello world"] as a child of 359 + the [<p>] element. 360 + 361 + Adjacent text nodes are automatically merged by the parser. *) 136 362 137 363 val comment_name : string 138 - (** ["#comment"] - name for comment nodes *) 364 + (** ["#comment"] - name for comment nodes. 365 + 366 + Comment nodes represent HTML comments: []. 367 + Comments are preserved in the DOM but not rendered to users. 368 + They're useful for development notes or conditional content. *) 139 369 140 370 val doctype_name : string 141 - (** ["!doctype"] - name for doctype nodes *) 371 + (** ["!doctype"] - name for doctype nodes. 372 + 373 + The DOCTYPE node represents the [<!DOCTYPE html>] declaration. 374 + It is always the first child of the Document node (if present). 375 + 376 + @see <https://html.spec.whatwg.org/multipage/syntax.html#the-doctype> 377 + WHATWG: The DOCTYPE *) 142 378 143 379 (** {1 Constructors} 144 380 145 381 Functions to create new DOM nodes. All nodes start with no parent and 146 - no children. 382 + no children. Use {!append_child} or {!insert_before} to build a tree. 147 383 *) 148 384 149 385 val create_element : string -> ?namespace:string option -> 150 386 ?attrs:(string * string) list -> unit -> node 151 387 (** Create an element node. 152 388 153 - @param name The tag name (e.g., "div", "p", "span") 154 - @param namespace Element namespace: [None] for HTML, [Some "svg"], [Some "mathml"] 155 - @param attrs Initial attributes as (name, value) pairs 389 + Elements are the primary building blocks of HTML documents. Each 390 + element represents a component of the document with semantic meaning. 391 + 392 + @param name The tag name (e.g., "div", "p", "span"). Tag names are 393 + case-insensitive in HTML; by convention, use lowercase. 394 + @param namespace Element namespace: 395 + - [None] (default): HTML namespace for standard elements 396 + - [Some "svg"]: SVG namespace for graphics elements 397 + - [Some "mathml"]: MathML namespace for mathematical notation 398 + @param attrs Initial attributes as [(name, value)] pairs 156 399 400 + {b Examples:} 157 401 {[ 402 + (* Simple HTML element *) 158 403 let div = create_element "div" () 159 - let svg = create_element "rect" ~namespace:(Some "svg") () 160 - let link = create_element "a" ~attrs:[("href", "/")] () 404 + 405 + (* Element with attributes *) 406 + let link = create_element "a" 407 + ~attrs:[("href", "https://example.com"); ("class", "external")] 408 + () 409 + 410 + (* SVG element *) 411 + let rect = create_element "rect" 412 + ~namespace:(Some "svg") 413 + ~attrs:[("width", "100"); ("height", "50"); ("fill", "blue")] 414 + () 161 415 ]} 416 + 417 + @see <https://html.spec.whatwg.org/multipage/dom.html#elements-in-the-dom> 418 + WHATWG: Elements in the DOM 162 419 *) 163 420 164 421 val create_text : string -> node 165 422 (** Create a text node with the given content. 166 423 424 + Text nodes contain the readable content of HTML documents. They 425 + appear as children of elements and represent the characters that 426 + users see. 427 + 428 + {b Note:} Text content is stored as-is. Character references like 429 + [&] should already be decoded to their character values. 430 + 431 + {b Example:} 167 432 {[ 168 433 let text = create_text "Hello, world!" 434 + (* To put text in a paragraph: *) 435 + let p = create_element "p" () in 436 + append_child p text 169 437 ]} 170 438 *) 171 439 172 440 val create_comment : string -> node 173 441 (** Create a comment node with the given content. 174 442 175 - The content should not include the comment delimiters. 443 + Comments are human-readable notes in HTML that don't appear in 444 + the rendered output. They're written as [] in HTML. 176 445 446 + @param data The comment text (without the [] delimiters) 447 + 448 + {b Example:} 177 449 {[ 178 - let comment = create_comment " This is a comment " 179 - (* Represents:  *) 450 + let comment = create_comment " TODO: Add navigation " 451 + (* Represents:  *) 180 452 ]} 453 + 454 + @see <https://html.spec.whatwg.org/multipage/syntax.html#comments> 455 + WHATWG: HTML comments 181 456 *) 182 457 183 458 val create_document : unit -> node 184 459 (** Create an empty document node. 185 460 186 - Document nodes are the root of a complete HTML document tree. 461 + The Document node is the root of an HTML document tree. It represents 462 + the entire document and serves as the parent for the DOCTYPE (if any) 463 + and the root [<html>] element. 464 + 465 + In a complete HTML document, the structure is: 466 + {v 467 + #document 468 + ├── !doctype 469 + └── html 470 + ├── head 471 + └── body 472 + v} 473 + 474 + @see <https://html.spec.whatwg.org/multipage/dom.html#document> 475 + WHATWG: The Document object 187 476 *) 188 477 189 478 val create_document_fragment : unit -> node 190 479 (** Create an empty document fragment. 191 480 192 - Document fragments are lightweight containers used for: 193 - - Template contents 194 - - Fragment parsing results 195 - - Efficient batch DOM operations 481 + Document fragments are lightweight containers that can hold multiple 482 + nodes without being part of the main document tree. They're useful for: 483 + 484 + - {b Template contents:} The [<template>] element stores its children 485 + in a document fragment, keeping them inert until cloned 486 + 487 + - {b Fragment parsing:} When parsing HTML fragments (like innerHTML), 488 + the result is placed in a document fragment 489 + 490 + - {b Batch operations:} Build a subtree in a fragment, then insert it 491 + into the document in one operation for better performance 492 + 493 + @see <https://dom.spec.whatwg.org/#documentfragment> 494 + DOM Standard: DocumentFragment 196 495 *) 197 496 198 497 val create_doctype : ?name:string -> ?public_id:string -> 199 498 ?system_id:string -> unit -> node 200 499 (** Create a DOCTYPE node. 201 500 202 - For HTML5, use [create_doctype ~name:"html" ()] which produces 203 - [<!DOCTYPE html>]. 501 + The DOCTYPE declaration tells browsers to use standards mode for 502 + rendering. For HTML5 documents, use: 503 + 504 + {[ 505 + let doctype = create_doctype ~name:"html" () 506 + (* Represents: <!DOCTYPE html> *) 507 + ]} 508 + 509 + @param name DOCTYPE name (usually ["html"] for HTML documents) 510 + @param public_id Public identifier (legacy, rarely needed) 511 + @param system_id System identifier (legacy, rarely needed) 512 + 513 + {b Legacy example:} 514 + {[ 515 + (* HTML 4.01 Strict DOCTYPE - not recommended for new documents *) 516 + let legacy = create_doctype 517 + ~name:"HTML" 518 + ~public_id:"-//W3C//DTD HTML 4.01//EN" 519 + ~system_id:"http://www.w3.org/TR/html4/strict.dtd" 520 + () 521 + ]} 204 522 205 - @param name DOCTYPE name (usually "html") 206 - @param public_id Public identifier (legacy) 207 - @param system_id System identifier (legacy) 523 + @see <https://html.spec.whatwg.org/multipage/syntax.html#the-doctype> 524 + WHATWG: The DOCTYPE 208 525 *) 209 526 210 527 val create_template : ?namespace:string option -> 211 528 ?attrs:(string * string) list -> unit -> node 212 529 (** Create a [<template>] element with its content document fragment. 213 530 214 - Template elements have special semantics: their children are not rendered 215 - directly but stored in a separate document fragment accessible via 216 - [template_content]. 531 + The [<template>] element holds inert HTML content that is not 532 + rendered directly. The content is stored in a separate document 533 + fragment and can be: 534 + - Cloned and inserted into the document via JavaScript 535 + - Used as a stamping template for repeated content 536 + - Pre-parsed without affecting the page 537 + 538 + {b How templates work:} 539 + 540 + Unlike normal elements, a [<template>]'s children are not rendered. 541 + Instead, they're stored in the [template_content] field. This means: 542 + - Images inside won't load 543 + - Scripts inside won't execute 544 + - The content is "inert" until explicitly activated 545 + 546 + {b Example:} 547 + {[ 548 + let template = create_template () in 549 + let div = create_element "div" () in 550 + let text = create_text "Template content" in 551 + append_child div text; 552 + (* Add to template's content fragment, not children *) 553 + match template.template_content with 554 + | Some fragment -> append_child fragment div 555 + | None -> () 556 + ]} 217 557 218 558 @see <https://html.spec.whatwg.org/multipage/scripting.html#the-template-element> 219 - The HTML5 template element specification 559 + WHATWG: The template element 220 560 *) 221 561 222 562 (** {1 Node Type Predicates} 223 563 224 - Functions to test what type of node you have. 564 + Functions to test what type of node you have. Since all nodes use the 565 + same record type, these predicates check the [name] field to determine 566 + the actual node type. 225 567 *) 226 568 227 569 val is_element : node -> bool 228 570 (** [is_element node] returns [true] if the node is an element node. 229 571 230 - Elements are nodes with HTML tags like [<div>], [<p>], etc. 572 + Elements are HTML tags like [<div>], [<p>], [<a>]. They are 573 + identified by having a tag name that doesn't match any of the 574 + special node name constants. 231 575 *) 232 576 233 577 val is_text : node -> bool 234 - (** [is_text node] returns [true] if the node is a text node. *) 578 + (** [is_text node] returns [true] if the node is a text node. 579 + 580 + Text nodes contain the character content within elements. 581 + They have [name = "#text"]. *) 235 582 236 583 val is_comment : node -> bool 237 - (** [is_comment node] returns [true] if the node is a comment node. *) 584 + (** [is_comment node] returns [true] if the node is a comment node. 585 + 586 + Comment nodes represent HTML comments []. 587 + They have [name = "#comment"]. *) 238 588 239 589 val is_document : node -> bool 240 - (** [is_document node] returns [true] if the node is a document node. *) 590 + (** [is_document node] returns [true] if the node is a document node. 591 + 592 + The document node is the root of the DOM tree. 593 + It has [name = "#document"]. *) 241 594 242 595 val is_document_fragment : node -> bool 243 - (** [is_document_fragment node] returns [true] if the node is a document fragment. *) 596 + (** [is_document_fragment node] returns [true] if the node is a document fragment. 597 + 598 + Document fragments are lightweight containers. 599 + They have [name = "#document-fragment"]. *) 244 600 245 601 val is_doctype : node -> bool 246 - (** [is_doctype node] returns [true] if the node is a DOCTYPE node. *) 602 + (** [is_doctype node] returns [true] if the node is a DOCTYPE node. 603 + 604 + DOCTYPE nodes represent the [<!DOCTYPE>] declaration. 605 + They have [name = "!doctype"]. *) 247 606 248 607 val has_children : node -> bool 249 - (** [has_children node] returns [true] if the node has any children. *) 608 + (** [has_children node] returns [true] if the node has any children. 609 + 610 + Note: For [<template>] elements, this checks the direct children list, 611 + not the template content fragment. *) 250 612 251 613 (** {1 Tree Manipulation} 252 614 253 615 Functions to modify the DOM tree structure. These functions automatically 254 - maintain parent/child references. 616 + maintain parent/child references, ensuring the tree remains consistent. 255 617 *) 256 618 257 619 val append_child : node -> node -> unit 258 620 (** [append_child parent child] adds [child] as the last child of [parent]. 259 621 260 622 The child's parent reference is updated to point to [parent]. 623 + If the child already has a parent, it is first removed from that parent. 624 + 625 + {b Example:} 626 + {[ 627 + let body = create_element "body" () in 628 + let p = create_element "p" () in 629 + let text = create_text "Hello!" in 630 + append_child p text; 631 + append_child body p 632 + (* Result: 633 + body 634 + └── p 635 + └── #text "Hello!" 636 + *) 637 + ]} 261 638 *) 262 639 263 640 val insert_before : node -> node -> node -> unit 264 641 (** [insert_before parent new_child ref_child] inserts [new_child] before 265 642 [ref_child] in [parent]'s children. 266 643 267 - @raise Not_found if [ref_child] is not a child of [parent] 644 + @param parent The parent node 645 + @param new_child The node to insert 646 + @param ref_child The existing child to insert before 647 + 648 + Raises [Not_found] if [ref_child] is not a child of [parent]. 649 + 650 + {b Example:} 651 + {[ 652 + let ul = create_element "ul" () in 653 + let li1 = create_element "li" () in 654 + let li3 = create_element "li" () in 655 + append_child ul li1; 656 + append_child ul li3; 657 + let li2 = create_element "li" () in 658 + insert_before ul li2 li3 659 + (* Result: ul contains li1, li2, li3 in that order *) 660 + ]} 268 661 *) 269 662 270 663 val remove_child : node -> node -> unit 271 664 (** [remove_child parent child] removes [child] from [parent]'s children. 272 665 273 666 The child's parent reference is set to [None]. 667 + 668 + Raises [Not_found] if [child] is not a child of [parent]. 274 669 *) 275 670 276 671 val insert_text_at : node -> string -> node option -> unit 277 672 (** [insert_text_at parent text before_node] inserts text content. 278 673 279 674 If [before_node] is [None], appends at the end. If the previous sibling 280 - is a text node, the text is merged into it. Otherwise, a new text node 281 - is created. 675 + is a text node, the text is merged into it (text nodes are coalesced). 676 + Otherwise, a new text node is created. 282 677 283 678 This implements the HTML5 parser's text insertion algorithm which 284 - coalesces adjacent text nodes. 679 + ensures adjacent text nodes are always merged, matching browser behavior. 680 + 681 + @see <https://html.spec.whatwg.org/multipage/parsing.html#appropriate-place-for-inserting-a-node> 682 + WHATWG: Inserting text in the DOM 285 683 *) 286 684 287 685 (** {1 Attribute Operations} 288 686 289 - Functions to read and modify element attributes. 687 + Functions to read and modify element attributes. Attributes are 688 + name-value pairs that provide additional information about elements. 689 + 690 + In HTML5, attribute names are case-insensitive and normalized to 691 + lowercase by the parser. 692 + 693 + @see <https://html.spec.whatwg.org/multipage/dom.html#attributes> 694 + WHATWG: Attributes 290 695 *) 291 696 292 697 val get_attr : node -> string -> string option 293 - (** [get_attr node name] returns the value of attribute [name], or [None]. *) 698 + (** [get_attr node name] returns the value of attribute [name], or [None] 699 + if the attribute doesn't exist. 700 + 701 + Attribute lookup is case-sensitive on the stored (lowercase) names. 702 + *) 294 703 295 704 val set_attr : node -> string -> string -> unit 296 705 (** [set_attr node name value] sets attribute [name] to [value]. 297 706 298 707 If the attribute already exists, it is replaced. 708 + If it doesn't exist, it is added. 299 709 *) 300 710 301 711 val has_attr : node -> string -> bool ··· 310 720 (** [descendants node] returns all descendant nodes in document order. 311 721 312 722 This performs a depth-first traversal, returning children before 313 - siblings at each level. 723 + siblings at each level. The node itself is not included. 724 + 725 + {b Document order} is the order nodes appear in the HTML source: 726 + parent before children, earlier siblings before later ones. 727 + 728 + {b Example:} 729 + {[ 730 + (* For tree: div > (p > "hello", span > "world") *) 731 + descendants div 732 + (* Returns: [p; text("hello"); span; text("world")] *) 733 + ]} 314 734 *) 315 735 316 736 val ancestors : node -> node list 317 737 (** [ancestors node] returns all ancestor nodes from parent to root. 318 738 319 - The first element is the immediate parent, the last is the root. 739 + The first element is the immediate parent, the last is the root 740 + (usually the Document node). 741 + 742 + {b Example:} 743 + {[ 744 + (* For a text node inside: html > body > p > text *) 745 + ancestors text_node 746 + (* Returns: [p; body; html; #document] *) 747 + ]} 320 748 *) 321 749 322 750 val get_text_content : node -> string 323 751 (** [get_text_content node] returns the concatenated text content. 324 752 325 - For text nodes, returns the text data. For elements, recursively 326 - concatenates all descendant text content. 753 + For text nodes, returns the text data directly. 754 + For elements, recursively concatenates all descendant text content. 755 + For other node types, returns an empty string. 756 + 757 + {b Example:} 758 + {[ 759 + (* For: <p>Hello <b>world</b>!</p> *) 760 + get_text_content p_element 761 + (* Returns: "Hello world!" *) 762 + ]} 327 763 *) 328 764 329 765 (** {1 Cloning} *) ··· 333 769 334 770 @param deep If [true], recursively clone all descendants (default: [false]) 335 771 336 - The cloned node has no parent. Attribute lists are copied by reference 337 - (the list itself is new, but attribute strings are shared). 772 + The cloned node has no parent. With [deep:false], only the node itself 773 + is copied (with its attributes, but not its children). 774 + 775 + {b Example:} 776 + {[ 777 + let original = create_element "div" ~attrs:[("class", "box")] () in 778 + let shallow = clone original in 779 + let deep = clone ~deep:true original 780 + ]} 338 781 *)

+666 -107

lib/html5rw/html5rw.mli

··· 5 5 6 6 (** Html5rw - Pure OCaml HTML5 Parser 7 7 8 - This module provides a complete HTML5 parsing solution following the 9 - WHATWG specification. It uses bytesrw for streaming input/output. 8 + This library provides a complete HTML5 parsing solution that implements the 9 + {{:https://html.spec.whatwg.org/multipage/parsing.html} WHATWG HTML5 10 + parsing specification}. It can parse any HTML document - well-formed or not - 11 + and produce a DOM (Document Object Model) tree that matches browser behavior. 12 + 13 + {2 What is HTML?} 14 + 15 + HTML (HyperText Markup Language) is the standard markup language for creating 16 + web pages. An HTML document consists of nested {i elements} that describe 17 + the structure and content of the page: 18 + 19 + {v 20 + <!DOCTYPE html> 21 + <html> 22 + <head> 23 + <title>My Page</title> 24 + </head> 25 + <body> 26 + <h1>Welcome</h1> 27 + <p>Hello, <b>world</b>!</p> 28 + </body> 29 + </html> 30 + v} 31 + 32 + Each element is written with a {i start tag} (like [<p>]), content, and an 33 + {i end tag} (like [</p>]). Elements can have {i attributes} that provide 34 + additional information: [<a href="https://example.com">]. 35 + 36 + @see <https://html.spec.whatwg.org/multipage/introduction.html> 37 + WHATWG: Introduction to HTML 38 + 39 + {2 The DOM} 40 + 41 + When this parser processes HTML, it doesn't just store the text. Instead, 42 + it builds a tree structure called the DOM (Document Object Model). Each 43 + element, text fragment, and comment becomes a {i node} in this tree: 44 + 45 + {v 46 + Document 47 + └── html 48 + ├── head 49 + │ └── title 50 + │ └── #text "My Page" 51 + └── body 52 + ├── h1 53 + │ └── #text "Welcome" 54 + └── p 55 + ├── #text "Hello, " 56 + ├── b 57 + │ └── #text "world" 58 + └── #text "!" 59 + v} 60 + 61 + This tree can be traversed, searched, and modified. The {!Dom} module 62 + provides types and functions for working with DOM nodes. 63 + 64 + @see <https://html.spec.whatwg.org/multipage/dom.html> 65 + WHATWG: The elements of HTML (DOM chapter) 10 66 11 67 {2 Quick Start} 12 68 13 - Parse HTML from a reader: 69 + Parse HTML from a string: 14 70 {[ 15 71 open Bytesrw 16 72 let reader = Bytes.Reader.of_string "<p>Hello, world!</p>" in ··· 32 88 let result = Html5rw.parse reader in 33 89 let divs = Html5rw.query result "div.content" 34 90 ]} 91 + 92 + {2 Error Handling} 93 + 94 + Unlike many parsers, HTML5 parsing {b never fails}. The WHATWG specification 95 + defines error recovery rules for every possible malformed input, ensuring 96 + all HTML documents produce a valid DOM tree (just as browsers do). 97 + 98 + For example, parsing [<p>Hello<p>World] produces two paragraphs, not an 99 + error, because [<p>] implicitly closes the previous [<p>]. 100 + 101 + If you need to detect malformed HTML (e.g., for validation), enable error 102 + collection with [~collect_errors:true]. Errors are advisory - the parsing 103 + still succeeds. 104 + 105 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 106 + WHATWG: Parse errors 107 + 108 + {2 HTML vs XHTML} 109 + 110 + This parser implements {b HTML5 parsing}, not XHTML parsing. Key differences: 111 + 112 + - Tag and attribute names are case-insensitive ([<DIV>] equals [<div>]) 113 + - Some end tags are optional ([<p>Hello] is valid) 114 + - Void elements have no end tag ([<br>], not [<br/>] or [<br></br>]) 115 + - Boolean attributes need no value ([<input disabled>]) 116 + 117 + XHTML uses stricter XML rules. If you need XHTML parsing, use an XML parser. 118 + 119 + @see <https://html.spec.whatwg.org/multipage/syntax.html> 120 + WHATWG: The HTML syntax 35 121 *) 36 122 37 123 (** {1 Sub-modules} *) 38 124 39 - (** DOM types and manipulation functions *) 125 + (** DOM types and manipulation functions. 126 + 127 + This module provides the core types for representing HTML documents as 128 + DOM trees. It includes: 129 + - The {!Dom.node} type representing all kinds of DOM nodes 130 + - Functions to create, modify, and traverse nodes 131 + - Serialization functions to convert DOM back to HTML 132 + 133 + @see <https://html.spec.whatwg.org/multipage/dom.html> 134 + WHATWG: The elements of HTML *) 40 135 module Dom = Html5rw_dom 41 136 42 - (** HTML5 tokenizer *) 137 + (** HTML5 tokenizer. 138 + 139 + The tokenizer is the first stage of HTML5 parsing. It converts a stream 140 + of characters into a stream of {i tokens}: start tags, end tags, text, 141 + comments, and DOCTYPEs. 142 + 143 + Most users don't need to use the tokenizer directly - the {!parse} 144 + function handles everything. The tokenizer is exposed for advanced use 145 + cases like syntax highlighting or partial parsing. 146 + 147 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tokenization> 148 + WHATWG: Tokenization *) 43 149 module Tokenizer = Html5rw_tokenizer 44 150 45 - (** Encoding detection and decoding *) 151 + (** Encoding detection and decoding. 152 + 153 + HTML documents can use various character encodings (UTF-8, ISO-8859-1, 154 + etc.). This module implements the WHATWG encoding sniffing algorithm 155 + that browsers use to detect the encoding of a document: 156 + 157 + 1. Check for a BOM (Byte Order Mark) 158 + 2. Look for a [<meta charset>] declaration 159 + 3. Use HTTP Content-Type header hint (if available) 160 + 4. Fall back to UTF-8 161 + 162 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 163 + WHATWG: Determining the character encoding 164 + @see <https://encoding.spec.whatwg.org/> 165 + WHATWG Encoding Standard *) 46 166 module Encoding = Html5rw_encoding 47 167 48 - (** CSS selector engine *) 168 + (** CSS selector engine. 169 + 170 + This module provides CSS selector support for querying the DOM tree. 171 + CSS selectors are patterns used to select HTML elements based on their 172 + tag names, attributes, classes, IDs, and position in the document. 173 + 174 + Example selectors: 175 + - [div] - all [<div>] elements 176 + - [#header] - element with [id="header"] 177 + - [.warning] - elements with [class="warning"] 178 + - [div > p] - [<p>] elements that are direct children of [<div>] 179 + - [[href]] - elements with an [href] attribute 180 + 181 + @see <https://www.w3.org/TR/selectors-4/> 182 + W3C Selectors Level 4 specification *) 49 183 module Selector = Html5rw_selector 50 184 51 - (** HTML entity decoding *) 185 + (** HTML entity decoding. 186 + 187 + HTML uses {i character references} to represent characters that are 188 + hard to type or have special meaning: 189 + 190 + - Named references: [&] (ampersand), [<] (less than), [ ] (non-breaking space) 191 + - Decimal references: [<] (less than as decimal 60) 192 + - Hexadecimal references: [<] (less than as hex 3C) 193 + 194 + This module decodes all 2,231 named character references defined in 195 + the WHATWG specification, plus numeric references. 196 + 197 + @see <https://html.spec.whatwg.org/multipage/named-characters.html> 198 + WHATWG: Named character references *) 52 199 module Entities = Html5rw_entities 53 200 54 - (** Low-level parser access *) 201 + (** Low-level parser access. 202 + 203 + This module exposes the internals of the HTML5 parser for advanced use. 204 + Most users should use the top-level {!parse} function instead. 205 + 206 + The parser exposes: 207 + - Insertion modes for the tree construction algorithm 208 + - The tree builder state machine 209 + - Lower-level parsing functions 210 + 211 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tree-construction> 212 + WHATWG: Tree construction *) 55 213 module Parser = Html5rw_parser 56 214 57 215 (** {1 Core Types} *) 58 216 59 - (** DOM node type. See {!Dom} for manipulation functions. *) 217 + (** DOM node type. 218 + 219 + A node represents one part of an HTML document. Nodes form a tree 220 + structure with parent/child relationships. There are several kinds: 221 + 222 + - {b Element nodes}: HTML tags like [<div>], [<p>], [<a>] 223 + - {b Text nodes}: Text content within elements 224 + - {b Comment nodes}: HTML comments [] 225 + - {b Document nodes}: The root of a document tree 226 + - {b Document fragment nodes}: Lightweight containers 227 + - {b Doctype nodes}: The [<!DOCTYPE html>] declaration 228 + 229 + See {!Dom} for manipulation functions. 230 + 231 + @see <https://html.spec.whatwg.org/multipage/dom.html> 232 + WHATWG: The DOM *) 60 233 type node = Dom.node 61 234 62 - (** Doctype information *) 235 + (** DOCTYPE information. 236 + 237 + The DOCTYPE declaration ([<!DOCTYPE html>]) appears at the start of HTML 238 + documents. It tells browsers to use standards mode for rendering. 239 + 240 + In HTML5, the DOCTYPE is minimal - just [<!DOCTYPE html>] with no public 241 + or system identifiers. Legacy DOCTYPEs may have additional fields. 242 + 243 + @see <https://html.spec.whatwg.org/multipage/syntax.html#the-doctype> 244 + WHATWG: The DOCTYPE *) 63 245 type doctype_data = Dom.doctype_data = { 64 246 name : string option; 247 + (** DOCTYPE name, typically ["html"] *) 248 + 65 249 public_id : string option; 250 + (** Public identifier for legacy DOCTYPEs (e.g., XHTML, HTML4) *) 251 + 66 252 system_id : string option; 253 + (** System identifier (URL) for legacy DOCTYPEs *) 67 254 } 68 255 69 - (** Quirks mode as determined during parsing *) 256 + (** Quirks mode as determined during parsing. 257 + 258 + {i Quirks mode} controls how browsers render CSS and compute layouts. 259 + It exists for backwards compatibility with old web pages that relied 260 + on browser bugs. 261 + 262 + - {b No_quirks}: Standards mode. The document is rendered according to 263 + modern HTML5 and CSS specifications. Triggered by [<!DOCTYPE html>]. 264 + 265 + - {b Quirks}: Full quirks mode. The browser emulates bugs from older 266 + browsers (primarily IE5). Triggered by missing or malformed DOCTYPEs. 267 + Affects CSS box model, table layout, font inheritance, and more. 268 + 269 + - {b Limited_quirks}: Almost standards mode. Only a few specific quirks 270 + are applied, mainly affecting table cell vertical alignment. 271 + 272 + {b Recommendation:} Always use [<!DOCTYPE html>] to ensure standards mode. 273 + 274 + @see <https://quirks.spec.whatwg.org/> 275 + Quirks Mode Standard 276 + @see <https://html.spec.whatwg.org/multipage/parsing.html#the-initial-insertion-mode> 277 + WHATWG: How quirks mode is determined *) 70 278 type quirks_mode = Dom.quirks_mode = No_quirks | Quirks | Limited_quirks 71 279 72 - (** Character encoding detected or specified *) 280 + (** Character encoding detected or specified. 281 + 282 + HTML documents are sequences of bytes that must be decoded into characters. 283 + Different encodings interpret the same bytes differently. For example: 284 + 285 + - UTF-8: The modern standard, supporting all Unicode characters 286 + - Windows-1252: Common on older Western European web pages 287 + - ISO-8859-2: Used for Central European languages 288 + - UTF-16: Used by some Windows applications 289 + 290 + The parser detects encoding automatically when using {!parse_bytes}. 291 + The detected encoding is available via {!val-encoding}. 292 + 293 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 294 + WHATWG: Determining the character encoding 295 + @see <https://encoding.spec.whatwg.org/> 296 + WHATWG Encoding Standard *) 73 297 type encoding = Encoding.encoding = 74 298 | Utf8 299 + (** UTF-8: The dominant encoding for the web, supporting all Unicode *) 300 + 75 301 | Utf16le 302 + (** UTF-16 Little-Endian: 16-bit encoding, used by Windows *) 303 + 76 304 | Utf16be 305 + (** UTF-16 Big-Endian: 16-bit encoding, network byte order *) 306 + 77 307 | Windows_1252 308 + (** Windows-1252 (CP-1252): Western European, superset of ISO-8859-1 *) 309 + 78 310 | Iso_8859_2 311 + (** ISO-8859-2: Central European (Polish, Czech, Hungarian, etc.) *) 312 + 79 313 | Euc_jp 314 + (** EUC-JP: Extended Unix Code for Japanese *) 80 315 81 316 (** A parse error encountered during HTML5 parsing. 82 317 83 - HTML5 parsing never fails - the specification defines error recovery 84 - for all malformed input. However, conformance checkers can report 85 - these errors. Enable error collection with [~collect_errors:true]. 318 + HTML5 parsing {b never fails} - the specification defines error recovery 319 + for all malformed input. However, conformance checkers can report these 320 + errors. Enable error collection with [~collect_errors:true] if you want 321 + to detect malformed HTML. 322 + 323 + {b Common parse errors:} 324 + 325 + - ["unexpected-null-character"]: Null byte in the input 326 + - ["eof-before-tag-name"]: File ended while reading a tag 327 + - ["unexpected-character-in-attribute-name"]: Invalid attribute syntax 328 + - ["missing-doctype"]: Document started without [<!DOCTYPE>] 329 + - ["duplicate-attribute"]: Same attribute appears twice on an element 330 + 331 + The full list of parse error codes is defined in the WHATWG specification. 86 332 87 333 @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 88 - WHATWG parse error definitions 89 - *) 334 + WHATWG: Complete list of parse errors *) 90 335 type parse_error = Parser.parse_error 91 336 92 - (** Get the error code (e.g., "unexpected-null-character"). *) 337 + (** Get the error code string. 338 + 339 + Error codes are lowercase with hyphens, matching the WHATWG specification 340 + names. Examples: ["unexpected-null-character"], ["eof-in-tag"], 341 + ["missing-end-tag-name"]. 342 + 343 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 344 + WHATWG: Parse error codes *) 93 345 val error_code : parse_error -> string 94 346 95 - (** Get the line number where the error occurred (1-indexed). *) 347 + (** Get the line number where the error occurred (1-indexed). 348 + 349 + Line numbers count from 1 and increment at each newline character. *) 96 350 val error_line : parse_error -> int 97 351 98 - (** Get the column number where the error occurred (1-indexed). *) 352 + (** Get the column number where the error occurred (1-indexed). 353 + 354 + Column numbers count from 1 and reset at each newline. *) 99 355 val error_column : parse_error -> int 100 356 101 357 (** Context element for HTML fragment parsing (innerHTML). 102 358 103 - When parsing HTML fragments, you must specify what element would 104 - contain the fragment. This affects how certain elements are handled. 359 + When parsing HTML fragments (like the [innerHTML] of an element), you 360 + must specify what element would contain the fragment. This affects how 361 + the parser handles certain elements. 362 + 363 + {b Why context matters:} 364 + 365 + HTML parsing rules depend on where content appears. For example: 366 + - [<td>] is valid inside [<tr>] but not inside [<div>] 367 + - [<li>] is valid inside [<ul>] but creates implied lists elsewhere 368 + - Content inside [<table>] has special parsing rules 369 + 370 + {b Example:} 371 + {[ 372 + (* Parse as if content were inside a <ul> *) 373 + let ctx = make_fragment_context ~tag_name:"ul" () in 374 + let result = parse ~fragment_context:ctx reader 375 + (* Now <li> elements are parsed correctly *) 376 + ]} 105 377 106 378 @see <https://html.spec.whatwg.org/multipage/parsing.html#parsing-html-fragments> 107 - The fragment parsing algorithm 108 - *) 379 + WHATWG: The fragment parsing algorithm *) 109 380 type fragment_context = Parser.fragment_context 110 381 111 382 (** Create a fragment parsing context. 112 383 113 - @param tag_name Tag name of the context element (e.g., "div", "tr") 114 - @param namespace Namespace: [None] for HTML, [Some "svg"], [Some "mathml"] 384 + The context element determines how the parser interprets the fragment. 385 + Choose a context that matches where the fragment would be inserted. 115 386 387 + @param tag_name Tag name of the context element (e.g., ["div"], ["tr"], 388 + ["ul"]). This is the element that would contain the fragment. 389 + @param namespace Namespace of the context element: 390 + - [None] (default): HTML namespace 391 + - [Some "svg"]: SVG namespace 392 + - [Some "mathml"]: MathML namespace 393 + 394 + {b Examples:} 116 395 {[ 117 - (* Parse as innerHTML of a <ul> *) 118 - let ctx = Html5rw.make_fragment_context ~tag_name:"ul" () 396 + (* Parse as innerHTML of a <div> (most common case) *) 397 + let ctx = make_fragment_context ~tag_name:"div" () 398 + 399 + (* Parse as innerHTML of a <ul> - <li> elements work correctly *) 400 + let ctx = make_fragment_context ~tag_name:"ul" () 119 401 120 402 (* Parse as innerHTML of an SVG <g> element *) 121 - let ctx = Html5rw.make_fragment_context ~tag_name:"g" ~namespace:(Some "svg") () 403 + let ctx = make_fragment_context ~tag_name:"g" ~namespace:(Some "svg") () 404 + 405 + (* Parse as innerHTML of a <table> - table-specific rules apply *) 406 + let ctx = make_fragment_context ~tag_name:"table" () 122 407 ]} 123 - *) 408 + 409 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parsing-html-fragments> 410 + WHATWG: Fragment parsing algorithm *) 124 411 val make_fragment_context : tag_name:string -> ?namespace:string option -> 125 412 unit -> fragment_context 126 413 ··· 132 419 133 420 (** Result of parsing an HTML document. 134 421 135 - Contains the parsed DOM tree, any errors encountered, and the 136 - detected encoding (when parsing from bytes). 422 + This record contains everything produced by parsing: 423 + - The DOM tree (accessible via {!val-root}) 424 + - Any parse errors (accessible via {!val-errors}) 425 + - The detected encoding (accessible via {!val-encoding}) 137 426 *) 138 427 type t = { 139 428 root : node; 429 + (** Root node of the parsed document tree. 430 + 431 + For full document parsing, this is a Document node containing the 432 + DOCTYPE (if any) and [<html>] element. 433 + 434 + For fragment parsing, this is a Document Fragment containing the 435 + parsed elements. *) 436 + 140 437 errors : parse_error list; 438 + (** Parse errors encountered during parsing. 439 + 440 + This list is empty unless [~collect_errors:true] was passed to the 441 + parse function. Errors are in the order they were encountered. 442 + 443 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 444 + WHATWG: Parse errors *) 445 + 141 446 encoding : encoding option; 447 + (** Character encoding detected during parsing. 448 + 449 + This is [Some encoding] when using {!parse_bytes} with automatic 450 + encoding detection, and [None] when using {!parse} (which expects 451 + pre-decoded UTF-8 input). *) 142 452 } 143 453 144 454 (** {1 Parsing Functions} *) 145 455 146 456 (** Parse HTML from a [Bytes.Reader.t]. 147 457 148 - This is the primary parsing function. Create a reader from any source: 149 - - [Bytes.Reader.of_string s] for strings 150 - - [Bytes.Reader.of_in_channel ic] for files 151 - - [Bytes.Reader.of_bytes b] for byte buffers 458 + This is the primary parsing function. It reads bytes from the provided 459 + reader and returns a DOM tree. The input should be valid UTF-8. 152 460 461 + {b Creating readers:} 153 462 {[ 154 463 open Bytesrw 155 - let reader = Bytes.Reader.of_string "<html><body>Hello</body></html>" in 464 + 465 + (* From a string *) 466 + let reader = Bytes.Reader.of_string html_string 467 + 468 + (* From a file *) 469 + let ic = open_in "page.html" in 470 + let reader = Bytes.Reader.of_in_channel ic 471 + 472 + (* From a buffer *) 473 + let reader = Bytes.Reader.of_buffer buf 474 + ]} 475 + 476 + {b Parsing a complete document:} 477 + {[ 156 478 let result = Html5rw.parse reader 479 + let doc = Html5rw.root result 157 480 ]} 158 481 159 - @param collect_errors If true, collect parse errors (default: false) 160 - @param fragment_context Context element for fragment parsing 161 - *) 162 - val parse : ?collect_errors:bool -> ?fragment_context:fragment_context -> Bytesrw.Bytes.Reader.t -> t 482 + {b Parsing a fragment:} 483 + {[ 484 + let ctx = Html5rw.make_fragment_context ~tag_name:"div" () in 485 + let result = Html5rw.parse ~fragment_context:ctx reader 486 + ]} 487 + 488 + @param collect_errors If [true], collect parse errors. Default: [false]. 489 + Error collection has some performance overhead. 490 + @param fragment_context Context element for fragment parsing. If provided, 491 + the input is parsed as a fragment (like innerHTML) rather than 492 + a complete document. 493 + 494 + @see <https://html.spec.whatwg.org/multipage/parsing.html> 495 + WHATWG: HTML parsing algorithm *) 496 + val parse : ?collect_errors:bool -> ?fragment_context:fragment_context -> 497 + Bytesrw.Bytes.Reader.t -> t 163 498 164 499 (** Parse raw bytes with automatic encoding detection. 165 500 166 - This function implements the WHATWG encoding sniffing algorithm: 167 - 1. Check for BOM (Byte Order Mark) 168 - 2. Prescan for <meta charset> 169 - 3. Fall back to UTF-8 501 + This function is useful when you have raw bytes and don't know the 502 + character encoding. It implements the WHATWG encoding sniffing algorithm: 503 + 504 + 1. {b BOM detection}: Check for UTF-8, UTF-16LE, or UTF-16BE BOM 505 + 2. {b Prescan}: Look for [<meta charset="...">] in the first 1024 bytes 506 + 3. {b Transport hint}: Use the provided [transport_encoding] if any 507 + 4. {b Fallback}: Use UTF-8 (the modern web default) 508 + 509 + The detected encoding is stored in the result's [encoding] field. 510 + 511 + {b Example:} 512 + {[ 513 + let bytes = really_input_bytes ic (in_channel_length ic) in 514 + let result = Html5rw.parse_bytes bytes in 515 + match Html5rw.encoding result with 516 + | Some Utf8 -> print_endline "UTF-8 detected" 517 + | Some Windows_1252 -> print_endline "Windows-1252 detected" 518 + | _ -> () 519 + ]} 170 520 171 - @param collect_errors If true, collect parse errors (default: false) 172 - @param transport_encoding Encoding from HTTP Content-Type header 173 - @param fragment_context Context element for fragment parsing 174 - *) 175 - val parse_bytes : ?collect_errors:bool -> ?transport_encoding:string -> ?fragment_context:fragment_context -> bytes -> t 521 + @param collect_errors If [true], collect parse errors. Default: [false]. 522 + @param transport_encoding Encoding hint from HTTP Content-Type header. 523 + For example, if the server sends [Content-Type: text/html; charset=utf-8], 524 + pass [~transport_encoding:"utf-8"]. 525 + @param fragment_context Context element for fragment parsing. 526 + 527 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 528 + WHATWG: Determining the character encoding *) 529 + val parse_bytes : ?collect_errors:bool -> ?transport_encoding:string -> 530 + ?fragment_context:fragment_context -> bytes -> t 176 531 177 532 (** {1 Querying} *) 178 533 179 534 (** Query the DOM tree with a CSS selector. 180 535 181 - Supported selectors: 182 - - Tag: [div], [p], [span] 183 - - ID: [#myid] 184 - - Class: [.myclass] 185 - - Universal: [*] 186 - - Attribute: [[attr]], [[attr="value"]], [[attr~="value"]], [[attr|="value"]] 187 - - Pseudo-classes: [:first-child], [:last-child], [:nth-child(n)] 188 - - Combinators: descendant (space), child (>), adjacent sibling (+), general sibling (~) 536 + CSS selectors are patterns used to select elements in HTML documents. 537 + This function returns all nodes matching the selector, in document order. 538 + 539 + {b Supported selectors:} 540 + 541 + {i Type selectors:} 542 + - [div], [p], [span] - elements by tag name 543 + 544 + {i Class and ID selectors:} 545 + - [#myid] - element with [id="myid"] 546 + - [.myclass] - elements with class containing "myclass" 547 + 548 + {i Attribute selectors:} 549 + - [[attr]] - elements with the [attr] attribute 550 + - [[attr="value"]] - attribute equals value 551 + - [[attr~="value"]] - attribute contains word 552 + - [[attr|="value"]] - attribute starts with value or value- 553 + - [[attr^="value"]] - attribute starts with value 554 + - [[attr$="value"]] - attribute ends with value 555 + - [[attr*="value"]] - attribute contains value 556 + 557 + {i Pseudo-classes:} 558 + - [:first-child], [:last-child] - first/last child of parent 559 + - [:nth-child(n)] - nth child (1-indexed) 560 + - [:only-child] - only child of parent 561 + - [:empty] - elements with no children 562 + - [:not(selector)] - elements not matching selector 189 563 564 + {i Combinators:} 565 + - [A B] - B descendants of A (any depth) 566 + - [A > B] - B direct children of A 567 + - [A + B] - B immediately after A (adjacent sibling) 568 + - [A ~ B] - B after A (general sibling) 569 + 570 + {i Universal:} 571 + - [*] - all elements 572 + 573 + {b Examples:} 190 574 {[ 191 - let divs = Html5rw.query result "div.content > p" 575 + (* All paragraphs *) 576 + let ps = query result "p" 577 + 578 + (* Elements with class "warning" inside a div *) 579 + let warnings = query result "div .warning" 580 + 581 + (* Direct children of nav that are links *) 582 + let nav_links = query result "nav > a" 583 + 584 + (* Complex selector *) 585 + let items = query result "ul.menu > li:first-child a[href]" 192 586 ]} 193 587 194 - @raise Selector.Selector_error if the selector is invalid 195 - *) 588 + @raise Selector.Selector_error if the selector syntax is invalid 589 + 590 + @see <https://www.w3.org/TR/selectors-4/> 591 + W3C: Selectors Level 4 *) 196 592 val query : t -> string -> node list 197 593 198 - (** Check if a node matches a CSS selector. *) 594 + (** Check if a node matches a CSS selector. 595 + 596 + This is useful for filtering nodes or implementing custom traversals. 597 + 598 + {b Example:} 599 + {[ 600 + let is_external_link node = 601 + matches node "a[href^='http']" 602 + ]} 603 + 604 + @raise Selector.Selector_error if the selector syntax is invalid *) 199 605 val matches : node -> string -> bool 200 606 201 607 (** {1 Serialization} *) 202 608 203 609 (** Write the DOM tree to a [Bytes.Writer.t]. 204 610 611 + This serializes the DOM back to HTML. The output is valid HTML5 that 612 + can be parsed to produce an equivalent DOM tree. 613 + 614 + {b Example:} 205 615 {[ 206 616 open Bytesrw 207 617 let buf = Buffer.create 1024 in ··· 211 621 let html = Buffer.contents buf 212 622 ]} 213 623 214 - @param pretty If true, format with indentation (default: true) 215 - @param indent_size Number of spaces per indent level (default: 2) 216 - *) 217 - val to_writer : ?pretty:bool -> ?indent_size:int -> t -> Bytesrw.Bytes.Writer.t -> unit 624 + @param pretty If [true] (default), add indentation for readability. 625 + If [false], output compact HTML with no added whitespace. 626 + @param indent_size Spaces per indentation level (default: 2). 627 + Only used when [pretty] is [true]. 628 + 629 + @see <https://html.spec.whatwg.org/multipage/parsing.html#serialising-html-fragments> 630 + WHATWG: Serialising HTML fragments *) 631 + val to_writer : ?pretty:bool -> ?indent_size:int -> t -> 632 + Bytesrw.Bytes.Writer.t -> unit 218 633 219 634 (** Serialize the DOM tree to a string. 220 635 221 - Convenience function when the output fits in memory. 636 + Convenience function that serializes to a string instead of a writer. 637 + Use {!to_writer} for large documents to avoid memory allocation. 222 638 223 - @param pretty If true, format with indentation (default: true) 224 - @param indent_size Number of spaces per indent level (default: 2) 225 - *) 639 + @param pretty If [true] (default), add indentation for readability. 640 + @param indent_size Spaces per indentation level (default: 2). *) 226 641 val to_string : ?pretty:bool -> ?indent_size:int -> t -> string 227 642 228 643 (** Extract text content from the DOM tree. 229 644 230 - @param separator String to insert between text nodes (default: " ") 231 - @param strip If true, trim whitespace (default: true) 232 - *) 645 + This concatenates all text nodes in the document, producing a string 646 + with just the readable text (no HTML tags). 647 + 648 + {b Example:} 649 + {[ 650 + (* For document: <div><p>Hello</p><p>World</p></div> *) 651 + let text = to_text result 652 + (* Returns: "Hello World" *) 653 + ]} 654 + 655 + @param separator String to insert between text nodes (default: [" "]) 656 + @param strip If [true] (default), trim leading/trailing whitespace *) 233 657 val to_text : ?separator:string -> ?strip:bool -> t -> string 234 658 235 - (** Serialize to html5lib test format (for testing). *) 659 + (** Serialize to html5lib test format. 660 + 661 + This produces the tree format used by the 662 + {{:https://github.com/html5lib/html5lib-tests} html5lib-tests} suite. 663 + Mainly useful for testing the parser against the reference tests. *) 236 664 val to_test_format : t -> string 237 665 238 666 (** {1 Result Accessors} *) 239 667 240 - (** Get the root node of the parsed document. *) 668 + (** Get the root node of the parsed document. 669 + 670 + For full document parsing, this returns a Document node. The structure is: 671 + {v 672 + #document 673 + ├── !doctype (if present) 674 + └── html 675 + ├── head 676 + └── body 677 + v} 678 + 679 + For fragment parsing, this returns a Document Fragment node containing 680 + the parsed elements directly. *) 241 681 val root : t -> node 242 682 243 - (** Get parse errors (if error collection was enabled). *) 683 + (** Get parse errors (if error collection was enabled). 684 + 685 + Returns an empty list if [~collect_errors:true] was not passed to the 686 + parse function, or if the document was well-formed. 687 + 688 + Errors are returned in the order they were encountered during parsing. 689 + 690 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 691 + WHATWG: Parse errors *) 244 692 val errors : t -> parse_error list 245 693 246 - (** Get the detected encoding (if parsed from bytes). *) 694 + (** Get the detected encoding (if parsed from bytes). 695 + 696 + Returns [Some encoding] when {!parse_bytes} was used, indicating which 697 + encoding was detected or specified. Returns [None] when {!parse} was 698 + used, since it expects pre-decoded UTF-8 input. 699 + 700 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 701 + WHATWG: Determining the character encoding *) 247 702 val encoding : t -> encoding option 248 703 249 704 (** {1 DOM Utilities} 250 705 251 - Common DOM operations are available directly. For the full API, 252 - see the {!Dom} module. 706 + Common DOM operations are available directly on this module. For the 707 + full API including more advanced operations, see the {!Dom} module. 708 + 709 + @see <https://html.spec.whatwg.org/multipage/dom.html> 710 + WHATWG: The elements of HTML 253 711 *) 254 712 255 713 (** Create an element node. 256 - @param namespace None for HTML, Some "svg" or Some "mathml" for foreign content 257 - @param attrs List of (name, value) attribute pairs 258 - *) 259 - val create_element : string -> ?namespace:string option -> ?attrs:(string * string) list -> unit -> node 714 + 715 + Elements are the building blocks of HTML documents. They represent tags 716 + like [<div>], [<p>], [<a>], etc. 717 + 718 + @param name Tag name (e.g., ["div"], ["p"], ["span"]) 719 + @param namespace Element namespace: 720 + - [None] (default): HTML namespace 721 + - [Some "svg"]: SVG namespace for graphics 722 + - [Some "mathml"]: MathML namespace for math notation 723 + @param attrs Initial attributes as [(name, value)] pairs 724 + 725 + {b Example:} 726 + {[ 727 + (* Simple element *) 728 + let div = create_element "div" () 260 729 261 - (** Create a text node. *) 730 + (* Element with attributes *) 731 + let link = create_element "a" 732 + ~attrs:[("href", "/about"); ("class", "nav-link")] 733 + () 734 + ]} 735 + 736 + @see <https://html.spec.whatwg.org/multipage/dom.html#elements-in-the-dom> 737 + WHATWG: Elements in the DOM *) 738 + val create_element : string -> ?namespace:string option -> 739 + ?attrs:(string * string) list -> unit -> node 740 + 741 + (** Create a text node. 742 + 743 + Text nodes contain the readable text content of HTML documents. 744 + 745 + {b Example:} 746 + {[ 747 + let text = create_text "Hello, world!" 748 + ]} *) 262 749 val create_text : string -> node 263 750 264 - (** Create a comment node. *) 751 + (** Create a comment node. 752 + 753 + Comments are preserved in the DOM but not rendered. They're written 754 + as [] in HTML. 755 + 756 + @see <https://html.spec.whatwg.org/multipage/syntax.html#comments> 757 + WHATWG: Comments *) 265 758 val create_comment : string -> node 266 759 267 - (** Create an empty document node. *) 760 + (** Create an empty document node. 761 + 762 + The Document node is the root of an HTML document tree. 763 + 764 + @see <https://html.spec.whatwg.org/multipage/dom.html#document> 765 + WHATWG: The Document object *) 268 766 val create_document : unit -> node 269 767 270 - (** Create a document fragment node. *) 768 + (** Create a document fragment node. 769 + 770 + Document fragments are lightweight containers for holding nodes 771 + without a parent document. Used for template contents and fragment 772 + parsing results. 773 + 774 + @see <https://dom.spec.whatwg.org/#documentfragment> 775 + DOM Standard: DocumentFragment *) 271 776 val create_document_fragment : unit -> node 272 777 273 - (** Create a doctype node. *) 274 - val create_doctype : ?name:string -> ?public_id:string -> ?system_id:string -> unit -> node 778 + (** Create a doctype node. 275 779 276 - (** Append a child node to a parent. *) 780 + For HTML5 documents, use [create_doctype ~name:"html" ()]. 781 + 782 + @param name DOCTYPE name (usually ["html"]) 783 + @param public_id Public identifier (legacy) 784 + @param system_id System identifier (legacy) 785 + 786 + @see <https://html.spec.whatwg.org/multipage/syntax.html#the-doctype> 787 + WHATWG: The DOCTYPE *) 788 + val create_doctype : ?name:string -> ?public_id:string -> 789 + ?system_id:string -> unit -> node 790 + 791 + (** Append a child node to a parent. 792 + 793 + The child is added as the last child of the parent. If the child 794 + already has a parent, it is first removed from that parent. *) 277 795 val append_child : node -> node -> unit 278 796 279 - (** Insert a node before a reference node. *) 797 + (** Insert a node before a reference node. 798 + 799 + @param parent The parent node 800 + @param new_child The node to insert 801 + @param ref_child The existing child to insert before 802 + 803 + Raises [Not_found] if [ref_child] is not a child of [parent]. *) 280 804 val insert_before : node -> node -> node -> unit 281 805 282 - (** Remove a child node from its parent. *) 806 + (** Remove a child node from its parent. 807 + 808 + Raises [Not_found] if [child] is not a child of [parent]. *) 283 809 val remove_child : node -> node -> unit 284 810 285 - (** Get an attribute value. *) 811 + (** Get an attribute value. 812 + 813 + Returns [Some value] if the attribute exists, [None] otherwise. 814 + Attribute names are case-sensitive (but were lowercased during parsing). 815 + 816 + @see <https://html.spec.whatwg.org/multipage/dom.html#attributes> 817 + WHATWG: Attributes *) 286 818 val get_attr : node -> string -> string option 287 819 288 - (** Set an attribute value. *) 820 + (** Set an attribute value. 821 + 822 + If the attribute exists, it is replaced. If not, it is added. *) 289 823 val set_attr : node -> string -> string -> unit 290 824 291 825 (** Check if a node has an attribute. *) 292 826 val has_attr : node -> string -> bool 293 827 294 - (** Get all descendant nodes. *) 828 + (** Get all descendant nodes in document order. 829 + 830 + Returns all nodes below this node in the tree, in the order they 831 + appear in the HTML source (depth-first). *) 295 832 val descendants : node -> node list 296 833 297 - (** Get all ancestor nodes (from parent to root). *) 834 + (** Get all ancestor nodes from parent to root. 835 + 836 + Returns the chain of parent nodes, starting with the immediate parent 837 + and ending with the Document node. *) 298 838 val ancestors : node -> node list 299 839 300 - (** Get text content of a node and its descendants. *) 840 + (** Get text content of a node and its descendants. 841 + 842 + For text nodes, returns the text directly. For elements, recursively 843 + concatenates all descendant text content. *) 301 844 val get_text_content : node -> string 302 845 303 846 (** Clone a node. 304 - @param deep If true, also clone descendants (default: false) 305 - *) 847 + 848 + @param deep If [true], recursively clone all descendants. 849 + If [false] (default), only clone the node itself. *) 306 850 val clone : ?deep:bool -> node -> node 307 851 308 - (** {1 Node Predicates} *) 852 + (** {1 Node Predicates} 309 853 310 - (** Test if a node is an element. *) 854 + Functions to test what type of node you have. 855 + *) 856 + 857 + (** Test if a node is an element. 858 + 859 + Elements are HTML tags like [<div>], [<p>], [<a>]. *) 311 860 val is_element : node -> bool 312 861 313 - (** Test if a node is a text node. *) 862 + (** Test if a node is a text node. 863 + 864 + Text nodes contain character content within elements. *) 314 865 val is_text : node -> bool 315 866 316 - (** Test if a node is a comment node. *) 867 + (** Test if a node is a comment node. 868 + 869 + Comment nodes represent HTML comments []. *) 317 870 val is_comment : node -> bool 318 871 319 - (** Test if a node is a document node. *) 872 + (** Test if a node is a document node. 873 + 874 + The document node is the root of a complete HTML document tree. *) 320 875 val is_document : node -> bool 321 876 322 - (** Test if a node is a document fragment. *) 877 + (** Test if a node is a document fragment. 878 + 879 + Document fragments are lightweight containers for nodes. *) 323 880 val is_document_fragment : node -> bool 324 881 325 - (** Test if a node is a doctype node. *) 882 + (** Test if a node is a doctype node. 883 + 884 + Doctype nodes represent the [<!DOCTYPE>] declaration. *) 326 885 val is_doctype : node -> bool 327 886 328 887 (** Test if a node has children. *)

+431 -93

lib/parser/html5rw_parser.mli

··· 3 3 SPDX-License-Identifier: MIT 4 4 ---------------------------------------------------------------------------*) 5 5 6 - (** HTML5 Parser 6 + (** HTML5 Parser - Low-Level API 7 7 8 8 This module provides the core HTML5 parsing functionality implementing 9 - the WHATWG parsing specification. It handles tokenization, tree construction, 9 + the {{:https://html.spec.whatwg.org/multipage/parsing.html} WHATWG 10 + HTML5 parsing specification}. It handles tokenization, tree construction, 10 11 error recovery, and produces a DOM tree. 11 12 12 - For most uses, prefer the top-level {!Html5rw} module which re-exports 13 - these functions with a simpler interface. 13 + For most uses, prefer the top-level {!Html5rw} module which provides 14 + a simpler interface. This module is for advanced use cases that need 15 + access to parser internals. 16 + 17 + {2 How HTML5 Parsing Works} 18 + 19 + The HTML5 parsing algorithm is unusual compared to most parsers. It was 20 + reverse-engineered from browser behavior rather than designed from a 21 + formal grammar. This ensures the parser handles malformed HTML exactly 22 + like web browsers do. 23 + 24 + The algorithm has three main phases: 25 + 26 + {3 1. Encoding Detection} 27 + 28 + Before parsing begins, the character encoding must be determined. The 29 + WHATWG specification defines a "sniffing" algorithm: 30 + 31 + 1. Check for a BOM (Byte Order Mark) at the start 32 + 2. Look for [<meta charset="...">] in the first 1024 bytes 33 + 3. Use HTTP Content-Type header hint if available 34 + 4. Fall back to UTF-8 35 + 36 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 37 + WHATWG: Determining the character encoding 38 + 39 + {3 2. Tokenization} 40 + 41 + The tokenizer converts the input stream into a sequence of tokens. 42 + It implements a state machine with over 80 states to handle: 43 + 44 + - Data (text content) 45 + - Tags (start tags, end tags, self-closing tags) 46 + - Comments 47 + - DOCTYPEs 48 + - Character references ([&], [<], [<]) 49 + - CDATA sections (in SVG/MathML) 50 + 51 + The tokenizer has special handling for: 52 + - {b Raw text elements}: [<script>], [<style>] - no markup parsing inside 53 + - {b Escapable raw text elements}: [<textarea>], [<title>] - limited parsing 54 + - {b RCDATA}: Content where only character references are parsed 55 + 56 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tokenization> 57 + WHATWG: Tokenization 58 + 59 + {3 3. Tree Construction} 60 + 61 + The tree builder receives tokens from the tokenizer and builds the DOM 62 + tree. It uses {i insertion modes} - a state machine that determines how 63 + each token should be processed based on the current document context. 64 + 65 + {b Insertion modes} include: 66 + - [initial]: Before the DOCTYPE 67 + - [before_html]: Before the [<html>] element 68 + - [before_head]: Before the [<head>] element 69 + - [in_head]: Inside [<head>] 70 + - [in_body]: Inside [<body>] (the most complex mode) 71 + - [in_table]: Inside [<table>] (special handling) 72 + - [in_template]: Inside [<template>] 73 + - And many more... 74 + 75 + The tree builder maintains: 76 + - {b Stack of open elements}: Elements that have been opened but not closed 77 + - {b List of active formatting elements}: For handling nested formatting 78 + - {b The template insertion mode stack}: For [<template>] elements 79 + 80 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tree-construction> 81 + WHATWG: Tree construction 82 + 83 + {2 Error Recovery} 84 + 85 + A key feature of HTML5 parsing is that it {b never fails}. The specification 86 + defines error recovery for every possible malformed input. For example: 87 + 88 + - Missing end tags are implicitly closed 89 + - Misnested tags are handled via the "adoption agency algorithm" 90 + - Invalid characters are replaced with U+FFFD 91 + - Unexpected elements are either ignored or moved to valid positions 14 92 15 - {2 Parsing Algorithm} 93 + This ensures every HTML document produces a valid DOM tree. 16 94 17 - The HTML5 parsing algorithm is defined by the WHATWG specification and 18 - consists of several phases: 95 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 96 + WHATWG: Parse errors 19 97 20 - 1. {b Encoding sniffing}: Detect character encoding from BOM, meta tags, 21 - or transport layer hints 22 - 2. {b Tokenization}: Convert the input stream into a sequence of tokens 23 - (start tags, end tags, character data, comments, etc.) 24 - 3. {b Tree construction}: Build the DOM tree using a state machine with 25 - multiple insertion modes 98 + {2 The Adoption Agency Algorithm} 99 + 100 + One of the most complex parts of HTML5 parsing is handling misnested 101 + formatting elements. For example: 102 + 103 + {v <p>Hello <b>world</p> <p>more</b> text</p> v} 26 104 27 - The algorithm includes extensive error recovery to handle malformed HTML 28 - in a consistent way across browsers. 105 + Browsers don't just error out - they use the "adoption agency algorithm" 106 + to produce sensible results. This algorithm: 107 + 1. Identifies formatting elements that span across other elements 108 + 2. Reconstructs the tree to properly nest elements 109 + 3. Moves nodes between parents as needed 29 110 30 - @see <https://html.spec.whatwg.org/multipage/parsing.html> 31 - The WHATWG HTML Parsing specification 111 + @see <https://html.spec.whatwg.org/multipage/parsing.html#adoption-agency-algorithm> 112 + WHATWG: The adoption agency algorithm 32 113 *) 33 114 34 115 (** {1 Sub-modules} *) 35 116 117 + (** DOM types and manipulation. *) 36 118 module Dom = Html5rw_dom 119 + 120 + (** HTML5 tokenizer. 121 + 122 + The tokenizer implements the first stage of HTML5 parsing, converting 123 + an input byte stream into a sequence of tokens (start tags, end tags, 124 + text, comments, DOCTYPEs). 125 + 126 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tokenization> 127 + WHATWG: Tokenization *) 37 128 module Tokenizer = Html5rw_tokenizer 129 + 130 + (** Character encoding detection and conversion. 131 + 132 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 133 + WHATWG: Determining the character encoding *) 38 134 module Encoding = Html5rw_encoding 135 + 136 + (** HTML element constants and categories. 137 + 138 + This module provides lists of element names that have special handling 139 + in the HTML5 parser: 140 + 141 + - {b Void elements}: Elements that cannot have children and have no end 142 + tag ([area], [base], [br], [col], [embed], [hr], [img], [input], 143 + [link], [meta], [source], [track], [wbr]) 144 + 145 + - {b Formatting elements}: Elements tracked in the list of active 146 + formatting elements for the adoption agency algorithm ([a], [b], [big], 147 + [code], [em], [font], [i], [nobr], [s], [small], [strike], [strong], 148 + [tt], [u]) 149 + 150 + - {b Special elements}: Elements with special parsing rules that affect 151 + scope and formatting reconstruction 152 + 153 + @see <https://html.spec.whatwg.org/multipage/syntax.html#void-elements> 154 + WHATWG: Void elements 155 + @see <https://html.spec.whatwg.org/multipage/parsing.html#formatting> 156 + WHATWG: Formatting elements *) 39 157 module Constants : sig 40 158 val void_elements : string list 159 + (** Elements that cannot have children: [area], [base], [br], [col], 160 + [embed], [hr], [img], [input], [link], [meta], [source], [track], [wbr]. 161 + 162 + @see <https://html.spec.whatwg.org/multipage/syntax.html#void-elements> 163 + WHATWG: Void elements *) 164 + 41 165 val formatting_elements : string list 166 + (** Elements tracked for the adoption agency algorithm: [a], [b], [big], 167 + [code], [em], [font], [i], [nobr], [s], [small], [strike], [strong], 168 + [tt], [u]. 169 + 170 + @see <https://html.spec.whatwg.org/multipage/parsing.html#formatting> 171 + WHATWG: Formatting elements *) 172 + 42 173 val special_elements : string list 174 + (** Elements with special parsing behavior that affect scope checking. 175 + 176 + @see <https://html.spec.whatwg.org/multipage/parsing.html#special> 177 + WHATWG: Special elements *) 43 178 end 179 + 180 + (** Parser insertion modes. 181 + 182 + Insertion modes are the states of the tree construction state machine. 183 + They determine how each token from the tokenizer should be processed 184 + based on the current document context. 185 + 186 + For example, a [<td>] tag is handled differently depending on whether 187 + the parser is currently in a table context or in the body. 188 + 189 + @see <https://html.spec.whatwg.org/multipage/parsing.html#insertion-mode> 190 + WHATWG: Insertion mode *) 44 191 module Insertion_mode : sig 45 192 type t 193 + (** The insertion mode type. Values include modes like [initial], 194 + [before_html], [in_head], [in_body], [in_table], etc. *) 46 195 end 196 + 197 + (** Tree builder state. 198 + 199 + The tree builder maintains the state needed for tree construction: 200 + - Stack of open elements 201 + - List of active formatting elements 202 + - Template insertion mode stack 203 + - Current insertion mode 204 + - Foster parenting flag 205 + 206 + @see <https://html.spec.whatwg.org/multipage/parsing.html#tree-construction> 207 + WHATWG: Tree construction *) 47 208 module Tree_builder : sig 48 209 type t 210 + (** The tree builder state. *) 49 211 end 50 212 51 213 (** {1 Types} *) 52 214 53 215 (** A parse error encountered during parsing. 54 216 55 - HTML5 parsing never fails - it always produces a DOM tree. However, 56 - the specification defines many error conditions that conformance 57 - checkers should report. Error collection is optional and disabled 58 - by default for performance. 217 + HTML5 parsing {b never fails} - it always produces a DOM tree. However, 218 + the WHATWG specification defines 92 specific error conditions that 219 + conformance checkers should report. These errors indicate malformed 220 + HTML that browsers will still render (with error recovery). 221 + 222 + {b Error categories:} 223 + 224 + {i Tokenizer errors} (detected during tokenization): 225 + - [abrupt-closing-of-empty-comment]: Comment closed with [-->] without content 226 + - [abrupt-doctype-public-identifier]: DOCTYPE public ID ended unexpectedly 227 + - [eof-before-tag-name]: End of file while reading a tag name 228 + - [eof-in-tag]: End of file inside a tag 229 + - [missing-attribute-value]: Attribute has [=] but no value 230 + - [unexpected-null-character]: Null byte in the input 231 + - [unexpected-question-mark-instead-of-tag-name]: [<?] used instead of [<!] 232 + 233 + {i Tree construction errors} (detected during tree building): 234 + - [missing-doctype]: No DOCTYPE before first element 235 + - [unexpected-token-*]: Token appeared in wrong context 236 + - [foster-parenting]: Content moved outside table due to invalid position 59 237 60 - Error codes follow the WHATWG specification naming convention, 61 - e.g., "unexpected-null-character", "eof-in-tag". 238 + Enable error collection with [~collect_errors:true]. Error collection 239 + has some performance overhead, so it's disabled by default. 62 240 63 241 @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 64 - The list of HTML5 parse errors 65 - *) 242 + WHATWG: Complete list of parse errors *) 66 243 type parse_error 67 244 68 245 (** Get the error code string. 69 246 70 - Error codes are lowercase with hyphens, matching the WHATWG spec names 71 - like "unexpected-null-character" or "eof-before-tag-name". 72 - *) 247 + Error codes are lowercase with hyphens, exactly matching the WHATWG 248 + specification naming. Examples: 249 + - ["unexpected-null-character"] 250 + - ["eof-before-tag-name"] 251 + - ["missing-end-tag-name"] 252 + - ["duplicate-attribute"] 253 + - ["missing-doctype"] 254 + 255 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 256 + WHATWG: Parse error codes *) 73 257 val error_code : parse_error -> string 74 258 75 - (** Get the line number where the error occurred (1-indexed). *) 259 + (** Get the line number where the error occurred. 260 + 261 + Line numbers are 1-indexed (first line is 1). Line breaks are 262 + detected at LF (U+000A), CR (U+000D), and CR+LF sequences. *) 76 263 val error_line : parse_error -> int 77 264 78 - (** Get the column number where the error occurred (1-indexed). *) 265 + (** Get the column number where the error occurred. 266 + 267 + Column numbers are 1-indexed (first column is 1). Columns reset 268 + to 1 after each line break. Column counting uses code points, 269 + not bytes or grapheme clusters. *) 79 270 val error_column : parse_error -> int 80 271 81 272 (** Context element for HTML fragment parsing. 82 273 83 - When parsing an HTML fragment (innerHTML), you need to specify the 84 - context element that would contain the fragment. This affects how 85 - the parser handles certain elements. 274 + When parsing HTML fragments (the content that would be assigned to 275 + an element's [innerHTML]), the parser needs to know what element 276 + would contain the fragment. This affects parsing in several ways: 86 277 87 - For example, parsing [<td>] as a fragment of a [<tr>] works differently 88 - than parsing it as a fragment of a [<div>]. 278 + {b Parser state initialization:} 279 + - For [<title>] or [<textarea>]: Tokenizer starts in RCDATA state 280 + - For [<style>], [<xmp>], [<iframe>], [<noembed>], [<noframes>]: 281 + Tokenizer starts in RAWTEXT state 282 + - For [<script>]: Tokenizer starts in script data state 283 + - For [<noscript>]: Tokenizer starts in RAWTEXT state (if scripting enabled) 284 + - For [<plaintext>]: Tokenizer starts in PLAINTEXT state 285 + - Otherwise: Tokenizer starts in data state 286 + 287 + {b Insertion mode:} 288 + The initial insertion mode depends on the context element: 289 + - [<template>]: "in template" mode 290 + - [<html>]: "before head" mode 291 + - [<head>]: "in head" mode 292 + - [<body>], [<div>], etc.: "in body" mode 293 + - [<table>]: "in table" mode 294 + - And so on... 89 295 90 296 @see <https://html.spec.whatwg.org/multipage/parsing.html#parsing-html-fragments> 91 - The HTML fragment parsing algorithm 92 - *) 297 + WHATWG: The fragment parsing algorithm *) 93 298 type fragment_context 94 299 95 300 (** Create a fragment parsing context. 96 301 97 - @param tag_name The tag name of the context element (e.g., "div", "tr") 98 - @param namespace Namespace: [None] for HTML, [Some "svg"], [Some "mathml"] 302 + @param tag_name Tag name of the context element. This should be the 303 + tag name of the element that would contain the fragment. 304 + Common choices: 305 + - ["div"]: General-purpose (most common) 306 + - ["body"]: For full body content 307 + - ["tr"]: For table row content ([<td>] elements) 308 + - ["ul"], ["ol"]: For list content ([<li>] elements) 309 + - ["select"]: For [<option>] elements 99 310 311 + @param namespace Element namespace: 312 + - [None]: HTML namespace (default) 313 + - [Some "svg"]: SVG namespace 314 + - [Some "mathml"]: MathML namespace 315 + 316 + {b Examples:} 100 317 {[ 101 - (* Parse as innerHTML of a table row *) 318 + (* Parse innerHTML of a table row - <td> works correctly *) 102 319 let ctx = make_fragment_context ~tag_name:"tr" () 103 320 104 - (* Parse as innerHTML of an SVG element *) 321 + (* Parse innerHTML of an SVG group element *) 105 322 let ctx = make_fragment_context ~tag_name:"g" ~namespace:(Some "svg") () 323 + 324 + (* Parse innerHTML of a select element - <option> works correctly *) 325 + let ctx = make_fragment_context ~tag_name:"select" () 106 326 ]} 107 - *) 327 + 328 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parsing-html-fragments> 329 + WHATWG: Fragment parsing algorithm *) 108 330 val make_fragment_context : tag_name:string -> ?namespace:string option -> 109 331 unit -> fragment_context 110 332 111 333 (** Get the tag name of a fragment context. *) 112 334 val fragment_context_tag : fragment_context -> string 113 335 114 - (** Get the namespace of a fragment context. *) 336 + (** Get the namespace of a fragment context ([None] for HTML). *) 115 337 val fragment_context_namespace : fragment_context -> string option 116 338 117 339 (** Result of parsing an HTML document or fragment. 118 340 119 - Contains the parsed DOM tree, any errors encountered (if error 120 - collection was enabled), and the detected encoding (for byte input). 341 + This opaque type contains: 342 + - The DOM tree (access via {!root}) 343 + - Parse errors if collection was enabled (access via {!errors}) 344 + - Detected encoding for byte input (access via {!encoding}) 121 345 *) 122 346 type t 123 347 124 348 (** {1 Parsing Functions} *) 125 349 350 + (** Parse HTML from a byte stream reader. 351 + 352 + This function implements the complete HTML5 parsing algorithm: 353 + 354 + 1. Reads bytes from the provided reader 355 + 2. Tokenizes the input into HTML tokens 356 + 3. Constructs a DOM tree using the tree construction algorithm 357 + 4. Returns the parsed result 358 + 359 + The input should be valid UTF-8. For automatic encoding detection 360 + from raw bytes, use {!parse_bytes} instead. 361 + 362 + {b Parser behavior:} 363 + 364 + For {b full document parsing} (no fragment context), the parser: 365 + - Creates a Document node as the root 366 + - Processes any DOCTYPE declaration 367 + - Creates [<html>], [<head>], and [<body>] elements as needed 368 + - Builds the full document tree 369 + 370 + For {b fragment parsing} (with fragment context), the parser: 371 + - Creates a Document Fragment as the root 372 + - Initializes tokenizer state based on context element 373 + - Initializes insertion mode based on context element 374 + - Does not create implicit [<html>], [<head>], [<body>] 375 + 376 + @param collect_errors If [true], collect parse errors in the result. 377 + Default: [false]. Enabling error collection adds overhead. 378 + @param fragment_context Context for fragment parsing. If provided, 379 + the input is parsed as fragment content (like innerHTML). 380 + 381 + @see <https://html.spec.whatwg.org/multipage/parsing.html> 382 + WHATWG: HTML parsing *) 126 383 val parse : ?collect_errors:bool -> ?fragment_context:fragment_context -> 127 384 Bytesrw.Bytes.Reader.t -> t 128 - (** Parse HTML from a byte stream reader. 385 + 386 + (** Parse HTML bytes with automatic encoding detection. 387 + 388 + This function wraps {!parse} with encoding detection, implementing the 389 + WHATWG encoding sniffing algorithm: 390 + 391 + {b Detection order:} 392 + 1. {b BOM}: Check first 2-3 bytes for UTF-8, UTF-16LE, or UTF-16BE BOM 393 + 2. {b Prescan}: Look for [<meta charset="...">] or 394 + [<meta http-equiv="Content-Type" content="...charset=...">] 395 + in the first 1024 bytes 396 + 3. {b Transport hint}: Use [transport_encoding] if provided 397 + 4. {b Fallback}: Use UTF-8 398 + 399 + The detected encoding is stored in the result (access via {!encoding}). 129 400 130 - This is the primary parsing function. The input must be valid UTF-8 131 - (or will be converted from detected encoding when using {!parse_bytes}). 401 + {b Prescan details:} 132 402 133 - @param collect_errors If [true], collect parse errors (default: [false]) 134 - @param fragment_context Context for fragment parsing (innerHTML) 403 + The prescan algorithm parses just enough of the document to find a 404 + charset declaration. It handles: 405 + - [<meta charset="utf-8">] 406 + - [<meta http-equiv="Content-Type" content="text/html; charset=utf-8">] 407 + - Comments and other markup are skipped 408 + - Parsing stops after 1024 bytes 135 409 136 - {[ 137 - open Bytesrw 138 - let reader = Bytes.Reader.of_string "<p>Hello</p>" in 139 - let result = parse reader 140 - ]} 141 - *) 410 + @param collect_errors If [true], collect parse errors. Default: [false]. 411 + @param transport_encoding Encoding hint from HTTP Content-Type header. 412 + For example: ["utf-8"], ["iso-8859-1"], ["windows-1252"]. 413 + @param fragment_context Context for fragment parsing. 142 414 415 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 416 + WHATWG: Determining the character encoding 417 + @see <https://html.spec.whatwg.org/multipage/parsing.html#prescan-a-byte-stream-to-determine-its-encoding> 418 + WHATWG: Prescan algorithm *) 143 419 val parse_bytes : ?collect_errors:bool -> ?transport_encoding:string -> 144 420 ?fragment_context:fragment_context -> bytes -> t 145 - (** Parse HTML bytes with automatic encoding detection. 146 421 147 - Implements the WHATWG encoding sniffing algorithm: 148 - 1. Check for BOM (UTF-8, UTF-16LE, UTF-16BE) 149 - 2. Prescan for [<meta charset>] declaration 150 - 3. Use transport encoding hint if provided 151 - 4. Fall back to UTF-8 422 + (** {1 Result Accessors} *) 423 + 424 + (** Get the root node of the parsed document. 152 425 153 - @param collect_errors If [true], collect parse errors (default: [false]) 154 - @param transport_encoding Encoding from HTTP Content-Type header 155 - @param fragment_context Context for fragment parsing (innerHTML) 156 - *) 426 + For full document parsing, returns a Document node with structure: 427 + {v 428 + #document 429 + ├── !doctype (if DOCTYPE was present) 430 + └── html 431 + ├── head 432 + │ └── ... (title, meta, link, script, style) 433 + └── body 434 + └── ... (page content) 435 + v} 157 436 158 - (** {1 Result Accessors} *) 437 + For fragment parsing, returns a Document Fragment node containing 438 + the parsed elements directly (no implicit html/head/body). 159 439 440 + @see <https://html.spec.whatwg.org/multipage/dom.html#document> 441 + WHATWG: The Document object *) 160 442 val root : t -> Dom.node 161 - (** Get the root node of the parsed document. 162 443 163 - For full document parsing, this is a document node. 164 - For fragment parsing, this is a document fragment node. 165 - *) 444 + (** Get parse errors collected during parsing. 445 + 446 + Returns an empty list if error collection was not enabled 447 + ([collect_errors:false] or omitted) or if the document was well-formed. 448 + 449 + Errors are returned in the order they were encountered. 166 450 451 + {b Example:} 452 + {[ 453 + let result = parse ~collect_errors:true reader in 454 + List.iter (fun e -> 455 + Printf.printf "Line %d, col %d: %s\n" 456 + (error_line e) (error_column e) (error_code e) 457 + ) (errors result) 458 + ]} 459 + 460 + @see <https://html.spec.whatwg.org/multipage/parsing.html#parse-errors> 461 + WHATWG: Parse errors *) 167 462 val errors : t -> parse_error list 168 - (** Get parse errors (empty if error collection was disabled). *) 463 + 464 + (** Get the detected encoding. 465 + 466 + Returns [Some encoding] when {!parse_bytes} was used, indicating which 467 + encoding was detected or specified. 468 + 469 + Returns [None] when {!parse} was used (it expects pre-decoded UTF-8). 169 470 471 + @see <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding> 472 + WHATWG: Determining the character encoding *) 170 473 val encoding : t -> Encoding.encoding option 171 - (** Get the detected encoding (only set when using {!parse_bytes}). *) 172 474 173 475 (** {1 Querying} *) 174 476 175 - val query : t -> string -> Dom.node list 176 477 (** Query the DOM with a CSS selector. 177 478 479 + Returns all elements matching the selector in document order. 480 + 481 + {b Supported selectors:} 482 + 483 + See {!Html5rw_selector} for the complete list. Key selectors include: 484 + - Type: [div], [p], [a] 485 + - ID: [#myid] 486 + - Class: [.myclass] 487 + - Attribute: [[href]], [[type="text"]] 488 + - Pseudo-class: [:first-child], [:nth-child(2)] 489 + - Combinators: [div p] (descendant), [div > p] (child) 490 + 178 491 @raise Html5rw_selector.Selector_error if the selector is invalid 179 492 180 - See {!Html5rw_selector} for supported selector syntax. 181 - *) 493 + @see <https://www.w3.org/TR/selectors-4/> 494 + W3C: Selectors Level 4 *) 495 + val query : t -> string -> Dom.node list 182 496 183 497 (** {1 Serialization} *) 184 498 499 + (** Serialize the DOM tree to a byte writer. 500 + 501 + Outputs valid HTML5 that can be parsed to produce an equivalent DOM tree. 502 + The output follows the WHATWG serialization algorithm. 503 + 504 + {b Serialization rules:} 505 + - Void elements are written without end tags 506 + - Attributes are quoted with double quotes 507 + - Special characters in text/attributes are escaped 508 + - Comments preserve their content 509 + - DOCTYPE is serialized as [<!DOCTYPE html>] 510 + 511 + @param pretty If [true] (default), add indentation for readability. 512 + @param indent_size Spaces per indent level (default: 2). 513 + 514 + @see <https://html.spec.whatwg.org/multipage/parsing.html#serialising-html-fragments> 515 + WHATWG: Serialising HTML fragments *) 185 516 val to_writer : ?pretty:bool -> ?indent_size:int -> t -> 186 517 Bytesrw.Bytes.Writer.t -> unit 187 - (** Serialize the DOM tree to a byte stream writer. 188 518 189 - @param pretty If [true], format with indentation (default: [true]) 190 - @param indent_size Spaces per indent level (default: [2]) 191 - *) 519 + (** Serialize the DOM tree to a string. 192 520 521 + Convenience wrapper around {!to_writer} that returns a string. 522 + 523 + @param pretty If [true] (default), add indentation for readability. 524 + @param indent_size Spaces per indent level (default: 2). *) 193 525 val to_string : ?pretty:bool -> ?indent_size:int -> t -> string 194 - (** Serialize the DOM tree to a string. 195 526 196 - @param pretty If [true], format with indentation (default: [true]) 197 - @param indent_size Spaces per indent level (default: [2]) 198 - *) 199 - 200 - val to_text : ?separator:string -> ?strip:bool -> t -> string 201 527 (** Extract text content from the DOM tree. 202 528 203 - @param separator String between text nodes (default: [" "]) 204 - @param strip If [true], trim whitespace (default: [true]) 205 - *) 529 + Returns the concatenation of all text node content in document order, 530 + with no HTML markup. 206 531 207 - val to_test_format : t -> string 532 + @param separator String to insert between text nodes (default: [" "]) 533 + @param strip If [true] (default), trim leading/trailing whitespace *) 534 + val to_text : ?separator:string -> ?strip:bool -> t -> string 535 + 208 536 (** Serialize to html5lib test format. 209 537 210 - This format is used by the html5lib test suite and shows the tree 211 - structure with indentation and node type prefixes. 212 - *) 538 + This produces the tree representation format used by the 539 + {{:https://github.com/html5lib/html5lib-tests} html5lib-tests} suite. 540 + 541 + The format shows the tree structure with: 542 + - Indentation indicating depth (2 spaces per level) 543 + - Prefixes indicating node type: 544 + - [<!DOCTYPE ...>] for DOCTYPE 545 + - [<tagname>] for elements (with attributes on same line) 546 + - ["text"] for text nodes 547 + - [] for comments 548 + 549 + Mainly useful for testing the parser against the reference test suite. *) 550 + val to_test_format : t -> string