{"name":"std/bufio.nu","source":"// stdlib/std/bufio.nu — buffered streaming reader\n//\n// BufReader pulls a file (or stdin) through a growable buffer one chunk\n// at a time, so a multi-gigabyte input is processed without ever being\n// fully resident — the opposite of `read_file` / `csv_reader_new`, which\n// load the whole file first. Built for fast ETL: log / CSV / JSONL\n// line scanning over inputs far larger than RAM.\n//\n// Speed:\n//   * One `fread` per refill against a 64 KiB buffer — few syscalls.\n//   * `memchr` (glibc SIMD) finds the line terminator, not a byte loop.\n//   * `bufreader_read_line_into` refills the *caller's* String, so a\n//     steady-state line loop allocates nothing after warm-up.\n//\n// Safety:\n//   * BufReader owns its buffer + file handle and is a library handle\n//     (docs/MEMORY.md §7.6): every copy is the same reader, and the last\n//     owner releases the buffer and closes the file it opened.\n//     `bufreader_close` does that now (optional).\n//   * Neither read entry point hands back a pointer into the internal\n//     buffer — `read_line` returns a fresh owned String, `read_line_into`\n//     copies into a String the caller already owns. No view is\n//     invalidated by the next call, so there is no lifetime foot-gun.\n//\n// API:\n//   ( bufreader_open path )            → ! BufReader IoErr   open a file \"rb\"\n//   ( bufreader_stdin )                → BufReader           wrap fd 0\n//   ( bufreader_read_line br )         → ? String   owned line, '\\n'/'\\r\\n'\n//                                                     stripped; None at EOF\n//   ( bufreader_read_line_into br dst ) → b         refill `dst` with the\n//                                                     next line (cleared\n//                                                     first); F at EOF.\n//                                                     Reuse one `dst`\n//                                                     across the loop for\n//                                                     zero-allocation ETL.\n//   ( bufreader_read_line_raw br dst )  → b         the same, but VERBATIM:\n//                                                     the line's own\n//                                                     terminator is kept\n//                                                     ('\\n', '\\r\\n', or\n//                                                     none at a final\n//                                                     unterminated line),\n//                                                     so writing every\n//                                                     `dst` back out\n//                                                     reproduces the input\n//                                                     byte for byte. What a\n//                                                     filter that must not\n//                                                     rewrite line endings\n//                                                     needs.\n//   ( bufreader_eof br )               → b          T once fully drained\n//   ( bufreader_close br )             → v          release now (optional)\n//\n// Example — count lines in a huge file without loading it. Note the\n// loop shape: a `~` condition must be side-effect-free, so the read\n// goes in the body and a flag drives the loop (the stdlib-wide idiom).\n//   ?? ( bufreader_open `events.jsonl` ) {\n//     T br → {\n//       : String line ( string_new )\n//       : ~ i n 0\n//       : ~ b more T\n//       ~ more {\n//         ? ( bufreader_read_line_into br line ) { = n + n 1 } { = more F }\n//       }\n//       ( string_free line )\n//       ( bufreader_close br )\n//       ( nurl_print ( nurl_str_int n ) )\n//     }\n//     F e → ( nurl_eprintln `open failed` )\n//   }\n\n$ `stdlib/core/marker.nu`\n$ `stdlib/core/string.nu`\n$ `stdlib/core/errors.nu`\n$ `stdlib/core/posix.nu`  // errno_kind\n$ `stdlib/std/fs.nu`  // nurl_file_open / nurl_file_close\n$ `stdlib/core/vec.nu`\n$ `stdlib/std/arc.nu`\n\n// ── libc bridges (pure-NURL FFI) ────────────────────────────────────\n// fread is declared globally in nurlc's preamble (Phase 7, 2026-05-23);\n// the local declaration has different pointer types so we keep this\n// one for the *u pointer shape — actually wait, nope, fread is\n// globally declared as (i8*, i64, i64, i8*) → i64; the per-file\n// declaration with *u clashes with that. Drop the local one and\n// adjust the call site to use the global shape.\n& `c` @ memchr in *u hay i needle i n → s\n\n& `c` @ memmove *u dst in *u src i n → *u\n\n& `c` @ fdopen i fd s mode → *v\n\n// Default read buffer: 64 KiB. Grows (×2) only for a line longer than\n// the current buffer; never shrinks.\n: i BUFIO_CAP 65536\n\n// BufReader is a one-word handle on its state in an Arc, so it\n// round-trips through `! T E`; every copy is the same reader, and the last\n// owner releases the buffer and closes a file the reader opened\n// (BufReaderImpl's drop).\n//\n//   h       — file handle (FILE* as i64; 0 ⇒ no stream / degrade to EOF)\n//   buf     — the read buffer; its length is the capacity in use\n//   start   — index of first unconsumed byte\n//   end     — index past last valid byte\n//   eof     — 1 once the underlying stream is exhausted\n//   owns    — 1 if the reader closes the handle (it opened it)\n//   scratch — start offset of the line last returned\n: BufReaderImpl { i h ( Vec u ) buf i start i end i eof i owns i scratch }\n\n// The stream is the outside world's: closing a file the reader opened is\n// its own drop, run by its last owner (the buffer goes with it).\n% Drop BufReaderImpl {\n    @ drop BufReaderImpl r → v {\n        ? && != 0 . r owns != 0 . r h { ( nurl_file_close # *v . r h ) } {}\n    }\n}\n\n: BufReader { ( Arc BufReaderImpl ) a }\n\n// Its drop closes something the outside world sees (stdlib/core/marker.nu).\n% Resource BufReader {}\n\n@ BufReader_share BufReader h → BufReader { ^ @ BufReader { ( arc_clone [BufReaderImpl] . h a ) } }\n\n@ BufReader_drop sink BufReader h → v {}\n\n// The reader's state in place.\n@ __br BufReader h → *BufReaderImpl { ^ ( arc_ptr [BufReaderImpl] . h a ) }\n\n// ── Internal ────────────────────────────────────────────────────────\n\n// errno-kind → IoErr; order matches `nurl_errno_kind` (see fs.nu).\n@ __bufio_err_of_kind i k → IoErr {\n    ? == k 0 { ^ @ IoErr { NotFound } } {}\n    ? == k 1 { ^ @ IoErr { PermissionDenied } } {}\n    ? == k 2 { ^ @ IoErr { AlreadyExists } } {}\n    ? == k 3 { ^ @ IoErr { Interrupted } } {}\n    ? == k 4 { ^ @ IoErr { UnexpectedEof } } {}\n    ? == k 5 { ^ @ IoErr { WriteFailed } } {}\n    ? == k 6 { ^ @ IoErr { ReadFailed } } {}\n    ^ @ IoErr { Other }\n}\n\n@ __bufreader_make * v h i owns → BufReader {\n    ^ @ BufReader { ( arc_new [BufReaderImpl] @ BufReaderImpl { # i h ( vec_zeroed [u] BUFIO_CAP ) 0 0 0 owns 0 } ) }\n}\n\n// Make room and pull one more chunk from the stream into [end, cap).\n// Compacts live bytes to the front first, then grows the buffer if it\n// is completely full. Sets the EOF flag when the stream yields nothing.\n@ __bufreader_fill * BufReaderImpl r → v {\n    ? == . r h 0 { = . r eof 1 } {\n        : ~ i cap ( vec_len [u] . r buf )\n        : i start . r start\n        : ~ i end . r end\n        // Compact: shift [start, end) down to offset 0.\n        ? > start 0 {\n            : i live - end start\n            : *u b ( vec_data [u] . r buf )\n            ? > live 0 {\n                : *u _m ( memmove b # *u + # i b start live )\n            } {}\n            = end live\n            = . r start 0\n            = . r end end\n        } {}\n        // Grow when the buffer is full and compaction freed nothing.\n        ? >= end cap {\n            = cap * cap 2\n            : b _g ( vec_resize_zeroed [u] . r buf cap )\n        } {}\n        // The data pointer is read after any grow.\n        : *u b ( vec_data [u] . r buf )\n        : i want - cap end\n        : i got ( fread # s + # i b end 1 want # s . r h )\n        ? > got 0 { = . r end + end got } { = . r eof 1 }\n    }\n}\n\n// Absolute buffer index of the next '\\n' in [start, end), or -1.\n@ __bufreader_find_nl * BufReaderImpl r → i {\n    : i start . r start\n    : i n - . r end start\n    ? <= n 0 { ^ -1 } {}\n    : *u buf ( vec_data [u] . r buf )\n    : s found ( memchr # *u + # i buf start 10 n )\n    ? == 0 # i found { ^ -1 } {}\n    ^ - # i found # i buf\n}\n\n// Consume the next line. Returns its length excluding the terminator\n// (>= 0), or -1 at end of input. On success `scratch` holds the line's\n// start offset in the current buffer — valid only until the next call.\n@ __bufreader_next_line * BufReaderImpl r → i {\n    : ~ i result -2\n    ~ == result -2 {\n        : i nl ( __bufreader_find_nl r )\n        ? >= nl 0 {\n            : i start . r start\n            : ~ i len - nl start\n            : *u buf ( vec_data [u] . r buf )\n            ? > len 0 {\n                ? == # i . buf - nl 1 13 { = len - len 1 } {}\n            } {}\n            = . r scratch start\n            = . r start + nl 1\n            = result len\n        } {\n            ? == 0 . r eof {\n                ( __bufreader_fill r )\n            } {\n                // EOF: emit the trailing unterminated line, if any.\n                : i start . r start\n                : i end . r end\n                ? > - end start 0 {\n                    : ~ i len - end start\n                    : *u buf ( vec_data [u] . r buf )\n                    ? == # i . buf - end 1 13 { = len - len 1 } {}\n                    = . r scratch start\n                    = . r start end\n                    = result len\n                } { = result -1 }\n            }\n        }\n    }\n    ^ result\n}\n\n// ── Constructors ────────────────────────────────────────────────────\n\n@ bufreader_open s path → !BufReader IoErr {\n    : *v h ( nurl_file_open path `rb` )\n    ? == 0 # i h {\n        ^ @ !BufReader IoErr { F ( __bufio_err_of_kind ( errno_kind ) ) }\n    } {}\n    ^ @ !BufReader IoErr { T ( __bufreader_make h 1 ) }\n}\n\n// Wrap stdin (fd 0). If fdopen somehow fails the reader degrades to an\n// immediately-empty stream rather than faulting.\n@ bufreader_stdin → BufReader {\n    : *v h ( fdopen 0 `rb` )\n    : BufReader br ( __bufreader_make h 1 )\n    ? == 0 # i h {\n        : *BufReaderImpl r ( __br br )\n        = . r eof 1\n    } {}\n    ^ br\n}\n\n// ── Reading ─────────────────────────────────────────────────────────\n\n@ bufreader_read_line BufReader br → ?String {\n    : *BufReaderImpl r ( __br br )\n    : i len ( __bufreader_next_line r )\n    ? < len 0 { ^ @ ?String { F # String 0 } } {}\n    : *u buf ( vec_data [u] . r buf )\n    ^ @ ?String { T ( string_from_bytes # *u + # i buf . r scratch len ) }\n}\n\n// Replace `dst`'s contents with the next line (terminator stripped) and\n// return T; return F at EOF leaving `dst` cleared. Reuse a single `dst`\n// across the read loop — its buffer grows once and is then never\n// reallocated, so the hot loop performs no per-line allocation.\n@ bufreader_read_line_into BufReader br String dst → b {\n    : *BufReaderImpl r ( __br br )\n    : i len ( __bufreader_next_line r )\n    ( string_clear dst )\n    ? < len 0 { ^ F } {}\n    : *u buf ( vec_data [u] . r buf )\n    ( string_push_bytes dst # *u + # i buf . r scratch len )\n    ^ T\n}\n\n// Byte-exact twin of `bufreader_read_line_into`: `dst` receives the line\n// AND the terminator the input actually carried — `\\n`, `\\r\\n`, or\n// nothing at all for a final unterminated line. Concatenating every\n// `dst` reproduces the stream exactly, which is the difference between a\n// filter (`head`, `grep`, `sed`) that preserves a CRLF file and one that\n// silently rewrites it to LF and drops or adds a trailing newline.\n//\n// The stripped form is the right default for parsing; this one is the\n// right default for copying. Same zero-allocation loop shape — reuse one\n// `dst`.\n@ bufreader_read_line_raw BufReader br String dst → b {\n    : *BufReaderImpl r ( __br br )\n    : i len ( __bufreader_next_line r )\n    ( string_clear dst )\n    ? < len 0 { ^ F } {}\n    : *u buf ( vec_data [u] . r buf )\n    : i off . r scratch\n    // `start` is the next unconsumed byte, so the span [off, start) is\n    // the line plus whatever terminated it.\n    : i raw - . r start off\n    ( string_push_bytes dst # *u + # i buf off raw )\n    ^ T\n}\n\n// ── Inspect / cleanup ───────────────────────────────────────────────\n\n@ bufreader_eof BufReader br → b {\n    : *BufReaderImpl r ( __br br )\n    ? == 0 . r eof { ^ F } {}\n    ^ >= . r start . r end\n}\n\n// Release the buffer and close the file now (optional: the last owner\n// does it). Every copy of `br` sees an exhausted reader afterwards.\n@ bufreader_close BufReader br → v {\n    : ( Arc BufReaderImpl ) a . br a\n    ? == 0 # i . a ctl { ^ } {}\n    : *BufReaderImpl r ( __br br )\n    = . r buf ( vec_new [u] )\n    ? && != 0 . r owns != 0 . r h { ( nurl_file_close # *v . r h ) } {}\n    = . r h 0\n    = . r start 0\n    = . r end 0\n    = . r eof 1\n}\n","bytes":13408}