OXYGENHUB

Change format 3: text blobs hold the lines' bytes 0f93f49ce059

A version 2 text blob held its insert's lines as a CBOR array, so every
line carried a length header: for the Linux tree's 46 308 files of 4 KiB
or more, 62 MB more raw and 35 MB more compressed, most of what its
store still had over Git's after gc.

Version 3 stores the lines' bytes, joined, whenever splitting them by
the line rule gives the lines back, which every insert a snapshot makes
does; others stay a CBOR array and are marked (key 7), and a reader
refuses a marked array the line rule would have split back, so a change
still has one encoding. A text blob is then the file's own bytes and
shares chunks with any blob of the same content. Readers keep reading
versions 1 and 2, whose hashes do not move; the resolver and TextRef
carry the blob's form. Spec Draft 0.8r.

refs #40
Author
Andy McCall @andymccall <[email protected]>
When
· 2026-09-30 02:28 UTC
Signature
✔ signed by Andy McCall
Hash
0f93f49ce059545cb28a316def983a6221dda14dc185d411f8e1304abd8aa081
Depends on
01a47cd7f0c8 17f94b13ae54 192fad1e46a2 2c75672fb2fe 387335ff7f4d 57d8dc05c9d4 598009a75f6c 5b9c6ad45f6e 6f8aeb996558 75167672ebcd 86e1824f2d0b 982864068a59 a4fad5b085c7 da652d6392d8 ff0af1ee299d
Depended on by
5b4c51307b14 acadb5ed28a1
In bookmarks
agents-in-readme bare-hashes carried-not-removed carry-identical doctor-trust identical-sides keyless-tags main merge-remote-bookmark only-help-103 paths-for-resolutions-and-amend push-notes show-stat subrepo-pin-state switch-says
In tags
v0.1.0 v0.10.0 v0.11.0 v0.2.0 v0.3.0 v0.3.1 v0.4.0 v0.4.1 v0.4.2 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.5.0 v0.5.1 v0.6.0 v0.7.0 v0.8.0 v0.9.0
Edits
56 atoms, 0 tree operations, 9 files

Browse the tree at this change

modified crates/oxy-core/src/lines.rs +33 −0
…
64 unchanged lines
65 65
        &self.0.buf
66 66
    }
67 67
68 +
    /// Whether splitting [`Lines::bytes`] by the line rule gives these lines
69 +
    /// back: every line but the last ends in a newline and none holds
70 +
    /// another. Every insert oxy makes is so, and such lines are stored as
71 +
    /// their bytes alone (spec §7.4).
72 +
    pub fn is_split_form(&self) -> bool {
73 +
        let buf = &self.0.buf;
74 +
        let mut start = 0usize;
75 +
        for (i, &end) in self.0.ends.iter().enumerate() {
76 +
            let line = &buf[start..end as usize];
77 +
            if line.is_empty() {
78 +
                return false;
79 +
            }
80 +
            let last = i + 1 == self.0.ends.len();
81 +
            let body = if last && !line.ends_with(b"\n") {
82 +
                line
83 +
            } else if line.ends_with(b"\n") {
84 +
                &line[..line.len() - 1]
85 +
            } else {
86 +
                return false;
87 +
            };
88 +
            if body.contains(&b'\n') {
89 +
                return false;
90 +
            }
91 +
            start = end as usize;
92 +
        }
93 +
        true
94 +
    }
95 +
68 96
    /// Lines `range` as lines of their own.
69 97
    pub fn slice(&self, range: std::ops::Range<usize>) -> Lines {
70 98
        self.iter().skip(range.start).take(range.len()).collect()
…
82 unchanged lines
153 181
            assert_eq!(Lines::from(l.to_vec()), l);
154 182
        }
155 183
        let l = Lines::split(b"a\nb\nc\n");
184 +
        assert!(l.is_split_form());
185 +
        assert!(!Lines::from(vec![b"a".to_vec(), b"b\n".to_vec()]).is_split_form());
186 +
        assert!(!Lines::from(vec![b"a\nb\n".to_vec()]).is_split_form());
187 +
        assert!(!Lines::from(vec![Vec::new()]).is_split_form());
188 +
        assert!(Lines::from(vec![b"a\n".to_vec(), b"end".to_vec()]).is_split_form());
156 189
        assert_eq!(&l[1], b"b\n");
157 190
        assert_eq!(
158 191
            l.slice(1..3).to_vec(),
…
5 unchanged lines
modified crates/oxy-core/src/objects.rs +124 −38
…
15 unchanged lines
16 16
pub const FORMAT_VERSION: u64 = 1;
17 17
18 18
/// The change format writers produce (spec §6.1). Version 2 stores each
19 -
/// large insert's lines as a text blob; readers accept 1 and 2, and a
20 -
/// change keeps the version it was made in, so its hash never moves.
21 -
pub const CHANGE_VERSION: u64 = 2;
19 +
/// large insert's lines as a text blob holding a CBOR array of them;
20 +
/// version 3 holds the lines' bytes themselves when splitting them by the
21 +
/// line rule gives them back. Readers accept 1 to 3, and a change keeps
22 +
/// the version it was made in, so its hash never moves.
23 +
pub const CHANGE_VERSION: u64 = 3;
22 24
25 +
/// How a text blob holds an insert's lines (spec §7.4).
26 +
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
27 +
pub enum TextForm {
28 +
    /// A CBOR array of the lines' byte strings: version 2.
29 +
    Cbor,
30 +
    /// The lines' bytes, split again by the line rule: version 3.
31 +
    Raw,
32 +
    /// A CBOR array in a version 3 change, for lines the line rule would
33 +
    /// not split back (marked by key 7): never for lines it would.
34 +
    CborFallback,
35 +
}
36 +
23 37
/// In a version 2 change, an insert whose lines encode (as a CBOR array of
24 38
/// byte strings) to at least this many bytes is stored as a text blob
25 39
/// holding exactly that encoding; a shorter one is inline (spec §6.1).
…
434 unchanged lines
460 474
    }
461 475
462 476
    /// The edit's value; `stored` is the text blob of an insert stored as
463 -
    /// one (see [`Change::value_with`], which works them out).
464 -
    fn to_value(&self, stored: Option<ObjectRef>) -> Value {
477 +
    /// one, and its form (see [`Change::value_with`], which works them out).
478 +
    fn to_value(&self, stored: Option<(ObjectRef, TextForm)>) -> Value {
465 479
        let nid = |n: &NodeId| Value::Bytes(n.0.to_vec());
466 480
        let opt = |o: &Option<ObjectRef>| match o {
467 481
            Some(r) => Value::Bytes(r.0.to_vec()),
…
7 unchanged lines
475 489
                    before,
476 490
                    lines,
477 491
                },
478 -
                Some(r),
479 -
            ) => Value::map([
480 -
                (0, Value::Uint(0)),
481 -
                (1, nid(node)),
482 -
                (2, after.to_value()),
483 -
                (3, before.to_value()),
484 -
                (5, Value::Bytes(r.0.to_vec())),
485 -
                (6, Value::Uint(lines.len() as u64)),
486 -
            ]),
492 +
                Some((r, form)),
493 +
            ) => {
494 +
                let mut fields = vec![
495 +
                    (0, Value::Uint(0)),
496 +
                    (1, nid(node)),
497 +
                    (2, after.to_value()),
498 +
                    (3, before.to_value()),
499 +
                    (5, Value::Bytes(r.0.to_vec())),
500 +
                    (6, Value::Uint(lines.len() as u64)),
501 +
                ];
502 +
                if form == TextForm::CborFallback {
503 +
                    fields.push((7, Value::Uint(1)));
504 +
                }
505 +
                Value::map(fields)
506 +
            }
487 507
            (
488 508
                Edit::Insert {
489 509
                    node,
…
59 unchanged lines
549 569
                        if count == 0 {
550 570
                            return schema("insert with no lines");
551 571
                        }
572 +
                        let form = match (version, m.get(&7)) {
573 +
                            (2, None) => TextForm::Cbor,
574 +
                            (_, None) => TextForm::Raw,
575 +
                            (3.., Some(Value::Uint(1))) => TextForm::CborFallback,
576 +
                            _ => return schema("unknown text blob form"),
577 +
                        };
552 578
                        // The resolver checks the count (`text_lines`).
553 -
                        resolve(&text, count)?
579 +
                        resolve(&text, count, form)?
554 580
                    }
555 581
                    _ => {
556 582
                        return schema(
…
85 unchanged lines
642 668
    /// the stored texts go to `texts`, in order, when it is given. Worked
643 669
    /// out on every core, a bounded batch at a time: chunking and hashing
644 670
    /// a whole tree's text is most of encoding a change that adds it.
645 -
    fn stored_texts(&self, sink: &mut TextSink) -> Vec<Option<ObjectRef>> {
671 +
    fn stored_texts(&self, sink: &mut TextSink) -> Vec<Option<(ObjectRef, TextForm)>> {
646 672
        use rayon::prelude::*;
647 -
        let mut refs = vec![None; self.edits.len()];
673 +
        let mut refs: Vec<Option<(ObjectRef, TextForm)>> = vec![None; self.edits.len()];
648 674
        let stored: Vec<usize> = (0..self.edits.len())
649 675
            .filter(|i| self.edits[*i].stored_text(self.version))
650 676
            .collect();
…
3 unchanged lines
654 680
                .map(|i| {
655 681
                    let Edit::Insert { lines, .. } = &self.edits[*i] else {
656 682
                        unreachable!("only inserts are stored as text");
683 +
                    };
684 +
                    let form = text_form(self.version, lines);
685 +
                    let content = match form {
686 +
                        TextForm::Raw => lines.bytes().to_vec(),
687 +
                        TextForm::Cbor | TextForm::CborFallback => cbor::encode_byte_strings(lines),
657 688
                    };
658 -
                    let content = cbor::encode_byte_strings(lines);
659 689
                    StoredText {
660 690
                        blob: Blob::of(&content),
661 691
                        content,
692 +
                        form,
662 693
                    }
663 694
                })
664 695
                .collect();
665 696
            for (i, t) in batch.iter().zip(&made) {
666 -
                refs[*i] = Some(t.blob.hash());
697 +
                refs[*i] = Some((t.blob.hash(), t.form));
667 698
            }
668 699
            if !sink(made) {
669 700
                break;
…
97 unchanged lines
767 798
    /// Decode a change that stores no text blob; one that does needs
768 799
    /// [`Change::decode_with`] and something to read them from.
769 800
    pub fn decode(bytes: &[u8]) -> Result<Change, ObjectError> {
770 -
        Self::decode_with(bytes, &mut |r, _| {
801 +
        Self::decode_with(bytes, &mut |r, _, _| {
771 802
            schema(format!(
772 803
                "text blob {} needs a store to read it from",
773 804
                r.short(12)
…
9 unchanged lines
783 814
    }
784 815
785 816
    pub fn from_value(v: &Value) -> Result<Change, ObjectError> {
786 -
        Self::from_value_with(v, &mut |r, _| {
817 +
        Self::from_value_with(v, &mut |r, _, _| {
787 818
            schema(format!(
788 819
                "text blob {} needs a store to read it from",
789 820
                r.short(12)
…
50 unchanged lines
840 871
/// the insert claims; [`text_lines`] reads and checks a blob's content.
841 872
/// A resolver may also return no lines, leaving a placeholder for a reader
842 873
/// that fills it in later (history-only states, spec §11).
843 -
pub type Resolver<'a> = dyn FnMut(&ObjectRef, u32) -> Result<Lines, ObjectError> + 'a;
874 +
pub type Resolver<'a> = dyn FnMut(&ObjectRef, u32, TextForm) -> Result<Lines, ObjectError> + 'a;
844 875
845 876
/// Receives the stored texts of a change being encoded, a batch at a
846 877
/// time; returns false to stop making more.
…
3 unchanged lines
850 881
/// file's graph (spec §11): it keeps no line until then.
851 882
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
852 883
pub enum TextRef {
853 -
    /// A stored insert: its text blob and line count.
854 -
    Blob(ObjectRef, u32),
884 +
    /// A stored insert: its text blob, line count, and the blob's form.
885 +
    Blob(ObjectRef, u32, TextForm),
855 886
    /// An inline insert: the byte range of its lines' array in the
856 887
    /// encoding of `change`.
857 888
    Inline {
…
116 unchanged lines
974 1005
        fields.insert(k, c.value()?);
975 1006
    }
976 1007
    let Some((start, end, count)) = inline else {
977 -
        return Edit::from_value(&Value::Map(fields), version, &mut |text, count| {
978 -
            texts.push(TextRef::Blob(*text, count));
1008 +
        return Edit::from_value(&Value::Map(fields), version, &mut |text, count, form| {
1009 +
            texts.push(TextRef::Blob(*text, count, form));
979 1010
            Ok(Lines::default())
980 1011
        });
981 1012
    };
…
28 unchanged lines
1010 1041
pub struct StoredText {
1011 1042
    pub blob: Blob,
1012 1043
    pub content: Vec<u8>,
1044 +
    pub form: TextForm,
1013 1045
}
1014 1046
1047 +
/// The form a change of `version` stores these lines in.
1048 +
fn text_form(version: u64, lines: &Lines) -> TextForm {
1049 +
    match version {
1050 +
        ..=2 => TextForm::Cbor,
1051 +
        _ if lines.is_split_form() => TextForm::Raw,
1052 +
        _ => TextForm::CborFallback,
1053 +
    }
1054 +
}
1055 +
1015 1056
/// The lines a text blob's `content` holds, checked against the rules for
1016 1057
/// one: canonical, as many as the insert claims, and not short enough to
1017 1058
/// have been inline.
1018 -
pub fn text_lines(content: &[u8], count: u32) -> Result<Lines, ObjectError> {
1019 -
    if content.len() < TEXT_BLOB_MIN {
1059 +
pub fn text_lines(content: &[u8], count: u32, form: TextForm) -> Result<Lines, ObjectError> {
1060 +
    let lines = match form {
1061 +
        TextForm::Raw => Lines::split(content),
1062 +
        TextForm::Cbor | TextForm::CborFallback => array_lines(&cbor::decode(content)?)?,
1063 +
    };
1064 +
    // One encoding per change: the size rule and the form are both fixed
1065 +
    // by the lines.
1066 +
    if cbor::byte_strings_len(&lines) < TEXT_BLOB_MIN {
1020 1067
        return schema(format!(
1021 1068
            "a text blob of {} bytes is short enough to be inline",
1022 1069
            content.len()
1023 1070
        ));
1024 1071
    }
1025 -
    let lines = array_lines(&cbor::decode(content)?)?;
1072 +
    if form == TextForm::CborFallback && lines.is_split_form() {
1073 +
        return schema("these lines are stored as their bytes in a version 3 change");
1074 +
    }
1026 1075
    if lines.len() != count as usize {
1027 1076
        return schema(format!(
1028 1077
            "a text blob holds {} lines, the insert says {count}",
…
233 unchanged lines
1262 1311
        let c = sample_change();
1263 1312
        let mut v = c.to_value();
1264 1313
        if let Value::Map(m) = &mut v {
1265 -
            m.insert(0, Value::Uint(3));
1314 +
            m.insert(0, Value::Uint(4));
1266 1315
        }
1267 1316
        assert!(
1268 1317
            matches!(Change::from_value(&v), Err(ObjectError::Schema(s)) if s.contains("version"))
…
48 unchanged lines
1317 1366
        assert_eq!(bytes, c.encode());
1318 1367
        assert!(bytes.len() < 1024, "{}", bytes.len());
1319 1368
        assert_eq!(texts.len(), 1);
1320 -
        assert_eq!(texts[0].content, cbor::encode_byte_strings(&lines));
1369 +
        // Version 3: the lines' bytes themselves.
1370 +
        assert_eq!(texts[0].form, TextForm::Raw);
1371 +
        assert_eq!(texts[0].content, lines.bytes());
1321 1372
        assert_eq!(texts[0].blob, Blob::of(&texts[0].content));
1322 1373
        assert!(Change::decode(&bytes).is_err(), "reading it needs the text");
1323 -
        let d = Change::decode_with(&bytes, &mut |r, n| {
1374 +
        let d = Change::decode_with(&bytes, &mut |r, n, form| {
1324 1375
            assert_eq!(*r, texts[0].blob.hash());
1325 -
            text_lines(&texts[0].content, n)
1376 +
            assert_eq!(form, TextForm::Raw);
1377 +
            text_lines(&texts[0].content, n, form)
1326 1378
        })
1327 1379
        .unwrap();
1328 1380
        assert_eq!(d, c);
1329 1381
        assert_eq!(d.hash(), c.hash());
1382 +
        // Version 2 held the same lines as a CBOR array, and still reads.
1383 +
        let mut v2 = c.clone();
1384 +
        v2.version = 2;
1385 +
        let (b2, t2) = v2.encode_parts();
1386 +
        assert_eq!(t2[0].form, TextForm::Cbor);
1387 +
        assert_eq!(t2[0].content, cbor::encode_byte_strings(&lines));
1388 +
        let d2 = Change::decode_with(&b2, &mut |_, n, form| text_lines(&t2[0].content, n, form))
1389 +
            .unwrap();
1390 +
        assert_eq!(d2, v2);
1391 +
        // Lines the line rule would not split back stay a CBOR array,
1392 +
        // marked; and a marked array of lines that would is refused.
1393 +
        let odd: Lines = (0..500)
1394 +
            .map(|i| format!("no newline {i}").into_bytes())
1395 +
            .collect();
1396 +
        let c3 = with_insert(sample_change(), odd.clone());
1397 +
        let (b3, t3) = c3.encode_parts();
1398 +
        assert_eq!(t3[0].form, TextForm::CborFallback);
1399 +
        let d3 = Change::decode_with(&b3, &mut |_, n, form| text_lines(&t3[0].content, n, form))
1400 +
            .unwrap();
1401 +
        assert_eq!(d3, c3);
1402 +
        let marked = cbor::encode_byte_strings(&lines);
1403 +
        assert!(text_lines(&marked, 500, TextForm::CborFallback).is_err());
1330 1404
    }
1331 1405
1332 1406
    #[test]
…
37 unchanged lines
1370 1444
            if let Edit::Insert { lines, .. } = e {
1371 1445
                assert!(lines.is_empty(), "no line kept");
1372 1446
                *lines = match next.next().unwrap() {
1373 -
                    TextRef::Blob(r, n) => {
1447 +
                    TextRef::Blob(r, n, form) => {
1374 1448
                        assert_eq!(*r, stored[0].blob.hash());
1375 -
                        text_lines(&stored[0].content, *n).unwrap()
1449 +
                        text_lines(&stored[0].content, *n, *form).unwrap()
1376 1450
                    }
1377 1451
                    TextRef::Inline { change, start, end } => {
1378 1452
                        assert_eq!(*change, me);
…
24 unchanged lines
1403 1477
            cbor::encode(&Value::Array(
1404 1478
                lines.iter().map(|l| Value::Bytes(l.to_vec())).collect()
1405 1479
            ))
1480 +
        );
1481 +
        assert_eq!(text_lines(&content, 500, TextForm::Cbor).unwrap(), lines);
1482 +
        assert!(text_lines(&content, 499, TextForm::Cbor).is_err());
1483 +
        assert_eq!(
1484 +
            text_lines(lines.bytes(), 500, TextForm::Raw).unwrap(),
1485 +
            lines
1406 1486
        );
1407 -
        assert_eq!(text_lines(&content, 500).unwrap(), lines);
1408 -
        assert!(text_lines(&content, 499).is_err());
1487 +
        assert!(text_lines(lines.bytes(), 501, TextForm::Raw).is_err());
1409 1488
        let small = cbor::encode_byte_strings(&vec![b"x\n".to_vec()].into());
1410 -
        assert!(text_lines(&small, 1).is_err(), "short enough to be inline");
1489 +
        assert!(
1490 +
            text_lines(&small, 1, TextForm::Cbor).is_err(),
1491 +
            "short enough to be inline"
1492 +
        );
1493 +
        assert!(
1494 +
            text_lines(b"x\n", 1, TextForm::Raw).is_err(),
1495 +
            "short enough to be inline"
1496 +
        );
1411 1497
    }
1412 1498
1413 1499
    #[test]
…
80 unchanged lines
modified crates/oxy-core/src/protocol.rs +2 −2
…
628 unchanged lines
629 629
            let encoded = o::bytes(&pair[0], "change")?;
630 630
            b.hashes
631 631
                .push(crate::hash::hash_object(ObjectType::Change, encoded));
632 -
            let change = Change::decode_with(encoded, &mut |text, count| {
632 +
            let change = Change::decode_with(encoded, &mut |text, count, form| {
633 633
                let by_ref = by_ref.get_or_insert_with(|| {
634 634
                    b.objects
635 635
                        .iter()
…
15 unchanged lines
651 651
                for chunk in &blob.chunks {
652 652
                    content.extend_from_slice(get(chunk, ObjectType::Chunk)?);
653 653
                }
654 -
                crate::objects::text_lines(&content, count)
654 +
                crate::objects::text_lines(&content, count, form)
655 655
            })?;
656 656
            b.changes.push((change, sigs_from(&pair[1])?));
657 657
        }
…
470 unchanged lines
modified crates/oxy-core/src/repo/fsck.rs +1 −1
…
89 unchanged lines
90 90
        ObjectType::Blob => Blob::decode(body).map(|_| ()),
91 91
        // Structure only: the text blobs are checked with the change graph.
92 92
        ObjectType::Change => {
93 -
            Change::decode_with(body, &mut |_, _| Ok(crate::lines::Lines::default())).map(|_| ())
93 +
            Change::decode_with(body, &mut |_, _, _| Ok(crate::lines::Lines::default())).map(|_| ())
94 94
        }
95 95
        ObjectType::Checkpoint => crate::objects::Checkpoint::decode(body).map(|_| ()),
96 96
        ObjectType::Op => match Op::decode(body) {
…
391 unchanged lines
modified crates/oxy-core/src/repo/sync.rs +2 −2
…
301 unchanged lines
302 302
                let bytes = self.store.get_typed(h, ObjectType::Change)?;
303 303
                let (skel, texts) = Change::decode_skeleton(&bytes, *h)?;
304 304
                for t in texts {
305 -
                    if let crate::objects::TextRef::Blob(r, _) = t {
305 +
                    if let crate::objects::TextRef::Blob(r, _, _) = t {
306 306
                        let blob = Blob::decode(&self.store.get_typed(&r, ObjectType::Blob)?)?;
307 307
                        objects.insert(r);
308 308
                        objects.extend(blob.chunks);
…
142 unchanged lines
451 451
                }
452 452
            }
453 453
            for t in texts {
454 -
                if let crate::objects::TextRef::Blob(r, _) = t {
454 +
                if let crate::objects::TextRef::Blob(r, _, _) = t {
455 455
                    if !self.store.has(&r) {
456 456
                        return fail(format!(
457 457
                            "change {} came without its text blob {}",
…
1282 unchanged lines
modified crates/oxy-core/src/repo/tests.rs +1 −1
…
5801 unchanged lines
5802 5802
    let other_text = texts
5803 5803
        .iter()
5804 5804
        .filter_map(|t| match t {
5805 -
            crate::objects::TextRef::Blob(r, _) => Some(*r),
5805 +
            crate::objects::TextRef::Blob(r, _, _) => Some(*r),
5806 5806
            _ => None,
5807 5807
        })
5808 5808
        .find(|r| content(r).windows(12).any(|w| w == b"other line 1"))
…
367 unchanged lines
modified crates/oxy-core/src/state.rs +3 −3
…
245 unchanged lines
246 246
    let bases = refs.len();
247 247
    refs.extend(taken.iter().flat_map(|(_, d)| {
248 248
        d.texts.iter().filter_map(|t| match t {
249 -
            TextRef::Blob(r, _) => Some(*r),
249 +
            TextRef::Blob(r, _, _) => Some(*r),
250 250
            TextRef::Inline { .. } => None,
251 251
        })
252 252
    }));
…
28 unchanged lines
281 281
                if let Edit::Insert { lines, .. } = e {
282 282
                    if lines.is_empty() {
283 283
                        *lines = match want.next() {
284 -
                            Some(TextRef::Blob(_, count)) => {
284 +
                            Some(TextRef::Blob(_, count, form)) => {
285 285
                                let content = texts.next().ok_or_else(|| {
286 286
                                    ObjectError::Schema("text blobs: wrong count".into())
287 287
                                })?;
288 -
                                crate::objects::text_lines(&content, count)?
288 +
                                crate::objects::text_lines(&content, count, form)?
289 289
                            }
290 290
                            Some(TextRef::Inline { change, start, end }) => {
291 291
                                let bytes = &changes[&change];
…
1672 unchanged lines
modified crates/oxy-core/src/store.rs +4 −2
…
1053 unchanged lines
1054 1054
        let bytes = self.get_typed(r, ObjectType::Change)?;
1055 1055
        let mut parts = Vec::new();
1056 1056
        let mut failed = None;
1057 -
        let decoded = Change::decode_with(&bytes, &mut |text, count| match self.read_text(text) {
1057 +
        let decoded = Change::decode_with(&bytes, &mut |text, count, form| match self
1058 +
            .read_text(text)
1059 +
        {
1058 1060
            Ok((blob, content)) => {
1059 1061
                parts.push(*text);
1060 1062
                parts.extend(blob.chunks);
1061 -
                crate::objects::text_lines(&content, count)
1063 +
                crate::objects::text_lines(&content, count, form)
1062 1064
            }
1063 1065
            Err(e) => {
1064 1066
                let why = e.to_string();
…
821 unchanged lines
modified docs/spec.md +5 −5
…
198 unchanged lines
199 199
200 200
```
201 201
Change {
202 -
  version   [0]: 1 or 2
202 +
  version   [0]: 1, 2 or 3
203 203
  change_id [1]: bytes(16)            // random, CSPRNG; see §6.3
204 204
  deps      [2]: [ChangeHash]         // sorted, unique; see §8
205 205
  author    [3]: KeyId                // ObjectRef of an identity object
…
6 unchanged lines
212 212
}
213 213
```
214 214
215 -
Writers produce version 2. Versions 1 and 2 differ only in how an `Insert` holds its lines (§7.4): a version 1 change carries every inserted line inline, and a version 2 change stores each large insert as a **text blob** beside it, so no change grows with the size of the files it adds. Readers MUST accept both. A change keeps the version it was made in: rewriting its encoding would change its hash and invalidate its signatures.
215 +
Writers produce version 3. The versions differ only in how an `Insert` holds its lines (§7.4): a version 1 change carries every inserted line inline; versions 2 and 3 store each large insert as a **text blob** beside it, so no change grows with the size of the files it adds, and differ in what the text blob holds. Readers MUST accept all three. A change keeps the version it was made in: rewriting its encoding would change its hash and invalidate its signatures.
216 216
217 217
### 6.2 Two identifiers
218 218
…
47 unchanged lines
266 266
### 7.4 Edits
267 267
268 268
```
269 -
Edit = Insert  { node: NodeId, after: AtomRef, before: AtomRef, lines: [bytes] | text: ObjectRef, count: u32 }
269 +
Edit = Insert  { node: NodeId, after: AtomRef, before: AtomRef, lines: [bytes] | text: ObjectRef, count: u32, cbor?: 1 }
270 270
     | Delete  { node: NodeId, atoms: [AtomRange] }
271 271
     | Replace { node: NodeId, old: ObjectRef | null, new: ObjectRef }     // opaque nodes only
272 272
AtomRef   = START | END | AtomId
…
2 unchanged lines
275 275
276 276
- `Insert` introduces `len(lines)` new atoms, indexed consecutively, and edges `after → first`, `last → before`. `after` and `before` MUST be adjacent in the author's view at authoring time; the pair is the insert's **anchor**.
277 277
- Atoms introduced by the same change are referenced with the all-zero "self" hash. This is how a change can insert a line and then insert another after it without knowing its own hash.
278 -
- **Text blobs.** In a version 2 change, an insert whose lines encode as a canonical CBOR array of byte strings to at least 4096 bytes MUST be stored as `text` and `count`: `text` is the blob (§13.2) whose content is exactly that encoding, and `count` is the number of lines, at least 1. A shorter insert MUST be inline. The rule is deterministic, so a change has one encoding and one hash, and identical inserts share their text blob. A reader rejects an inline insert that should have been stored, a text blob shorter than 4096 bytes, and a text blob whose lines do not number `count`. A text blob and its chunks belong to their change: they are carried with it by every transfer (§16.3), kept with it by garbage collection (§21), and checked with it by `fsck`. A change cannot be read without them, so they are never withheld from a partial clone (§13.5).
278 +
- **Text blobs.** In a version 2 or 3 change, an insert whose lines encode as a canonical CBOR array of byte strings to at least 4096 bytes MUST be stored as `text` and `count`: `text` is a blob (§13.2) holding the lines, and `count` is the number of lines, at least 1. In version 2 the blob's content is exactly that CBOR array. In version 3 it is the lines' bytes, joined, when splitting those bytes by the line rule (every line but the last ends in a newline and holds no other; §7.3) gives the lines back, as every insert a snapshot makes does; otherwise it is the CBOR array and the insert carries `cbor: 1` (key 7). A reader rejects a version 3 insert marked `cbor` whose lines the line rule would split back. The joined form stores a file's text as its bytes, so a text blob shares chunks with any blob of the same content and carries no per-line framing (a tenth of a tree's compressed text). A shorter insert MUST be inline. The rule is deterministic, so a change has one encoding and one hash, and identical inserts share their text blob. A reader rejects an inline insert that should have been stored, a text blob shorter than 4096 bytes, and a text blob whose lines do not number `count`. A text blob and its chunks belong to their change: they are carried with it by every transfer (§16.3), kept with it by garbage collection (§21), and checked with it by `fsck`. A change cannot be read without them, so they are never withheld from a partial clone (§13.5).
279 279
- `Replace` is the only edit for `opaque` nodes. `old` is the blob currently expected; two concurrent `Replace` operations with the same `old` and different `new` conflict (§9.3).
280 280
281 281
### 7.5 Opaque nodes
…
695 unchanged lines
977 977
978 978
---
979 979
980 -
*Changelog.* Draft 0.8r (2026-09-29): `oxy gc` repacks into one pack at a higher compression level (§13.4); an import brings a deleted directory back rather than making a second node for its path, and retires a kept node's other placements (§23); graph keys at commit, lazy text in history-only loads, and a clean `status` from the manifest (§12.4); a parent checkout never moves a subrepo's bookmark backwards, and the pin is read from the nested working copy (§7.8); `bookmark create --upto/--only` (§10.1); the working-copy lock covers every command that writes the working copy; a workspace changed by two concurrent operations is an error (§12.1). Draft 0.8q (2026-09-29): change format version 2 stores large inserts as text blobs, so a change does not grow with the files it adds (§6.1, §7.4); no object may exceed 1 GiB (§13.2). Draft 0.8p (2026-09-29): Git import derives new NodeIds from the commit, making imports deterministic (§23). Draft 0.8o (2026-09-29): execution receipts, the `receipt` object and attestation (§14.6). Draft 0.8n (2026-09-29): line history, `oxy log -L` (§27). Draft 0.8m (2026-09-29): resolving replace conflicts (§9.3). Draft 0.8l (2026-09-28): deleting a reference on a remote; first serials from the clock (§16.4). Draft 0.8k (2026-09-28): the manifest records graph keys, so unchanged files are neither read nor rendered (§12.4). Draft 0.8j (2026-09-28): checkpoint payload version 2, file graphs by reference (§11).
980 +
*Changelog.* Draft 0.8r (2026-09-29): change format version 3, text blobs as the lines' bytes (§6.1, §7.4); `oxy gc` repacks into one pack at a higher compression level (§13.4); an import brings a deleted directory back rather than making a second node for its path, and retires a kept node's other placements (§23); graph keys at commit, lazy text in history-only loads, and a clean `status` from the manifest (§12.4); a parent checkout never moves a subrepo's bookmark backwards, and the pin is read from the nested working copy (§7.8); `bookmark create --upto/--only` (§10.1); the working-copy lock covers every command that writes the working copy; a workspace changed by two concurrent operations is an error (§12.1). Draft 0.8q (2026-09-29): change format version 2 stores large inserts as text blobs, so a change does not grow with the files it adds (§6.1, §7.4); no object may exceed 1 GiB (§13.2). Draft 0.8p (2026-09-29): Git import derives new NodeIds from the commit, making imports deterministic (§23). Draft 0.8o (2026-09-29): execution receipts, the `receipt` object and attestation (§14.6). Draft 0.8n (2026-09-29): line history, `oxy log -L` (§27). Draft 0.8m (2026-09-29): resolving replace conflicts (§9.3). Draft 0.8l (2026-09-28): deleting a reference on a remote; first serials from the clock (§16.4). Draft 0.8k (2026-09-28): the manifest records graph keys, so unchanged files are neither read nor rendered (§12.4). Draft 0.8j (2026-09-28): checkpoint payload version 2, file graphs by reference (§11).
981 981
982 982
Draft 0.8i (2026-09-28): portability rules for exec bits and path names (§7.7); found by verifying a real Git import commit by commit: deletions depend on the inserters of the lines right after them, and replacements on the change that set their old blob (§8.1); replace records are superseded causally, so a reverted binary keeps its blob (§9.3); merges resolve opaque conflicts to the commit's blob, and `oxy git verify` (§23). Draft 0.8h (2026-09-28): `oxy fsck` (§21.1); durability rules and the operation head as commit point (§13.6); the operation head moves by compare-and-swap and concurrent operations are merged (§12.1); `oxy doctor` (§21.2). Draft 0.8g (2026-09-27): delegates of a rule's identities may sign reference attestations (§14.5); local-path pushes bind unbound repositories like HTTP ones. Draft 0.8f (2026-09-27): challenge signatures (§14.3) for hosts. Draft 0.8e (2026-09-27): anonymous changes (zero author) are authenticated by a seal from any key, so imported and pre-identity history can be pushed under a policy (§14.4). Draft 0.8d (2026-09-27): unbound hosted repositories adopt the first pusher's id (§16.1). Draft 0.8c (2026-09-27): bare repositories (`oxy init --bare`, §12.2: no workspace, no working copy; hosting) and the reference server as an embeddable library (§15.1). Draft 0.8b (2026-09-27): the machine interface for editors (§24): `root`, `cat`, `version`, `hunks`, `discard`, and `--json` on the reporting commands.
983 983
…
61 unchanged lines