How a data page (page type 0x01) lays out its rows, and the Access
behaviours that the library got wrong until the catalog-lookup fix (deleted
slots that share a live row’s offset, and overflow rows) and the
jet3-long-rows fix (the Jet3 jump table), and the usage-map rows that list a
table’s and an index’s pages. The page layout is the same in
Jet3, Jet4 and ACE apart from the header offsets. The reference is
Jackcess TableImpl (findRowStart,
findRowEnd, positionAtRowData, deleteRow, updateRow), checked against
the Access-authored fixtures in tests/JetDatabaseWriter.Tests/Databases.
DataPageRows.TryGetSlotBound uses generated Dafny code for bounded directory
acquisition and the complete next-distinct-offset scan. The selected start must
follow the clamped directory and lie within the page. Every slot, including a
deleted slot, can bound the next row; equal offsets are skipped. Header and
directory reads are bounded before acquisition. This verifies the returned
slot interval under the supplied layout and stable page-image contract.
Complete row decoding, overflow traversal and native compatibility retain their
own evidence in the formal boundary.
ReclaimDeletedRowSpace uses DataPageKernel for complete admission, minimum
payload/next-distinct-boundary selection, directory relocation and free-space
accounting. Invalid bounds refuse before mutation. Optimized overlapping bulk
copy and optional secure clear retain explicit byte-effect contracts; generated
directory writes preserve the payload and unrelated header/suffix bytes. The
compaction boundary
separates modeled byte preservation from CLR effects, page ownership and the
containing statement’s rollback.
RowValueDecodePolicy validates nonempty fixed-size payloads stored in variable slots and calculated envelopes before the string, typed or compiled POCO decoder reads them. Strict parsing reports malformed values with the column name; lenient parsing returns empty text for string output, DBNull.Value for typed rows and an unassigned/default POCO property. A valid zero-length Text or binary value retains its normal empty representation. Ordinary calculated Null maps to a missing value. The measured native Byte error form (39 bytes, first UInt32 6, all remaining bytes zero) instead raises contextual JetLimitationException with NumericOverflow in both parsing modes, before scalar conversion. CalculatedByteErrorReadTests constructs that native layout separately from its Null and malformed-framing controls; value-preserving error reads, other native error forms and native error writes remain R4 work. A nonempty payload shorter than its scalar type requires is malformed. Calculated MEMO/OLE envelopes use the same validation after exact long-value retrieval. Write-back snapshots retain the unreadable-long-value sentinel, and storage I/O failures propagate.
Fixed and variable native Date/Time and Date/Time Extended slots,
and cached classic calculated dates use the same public failure policy when their
payload cannot be decoded. Strict string, typed, direct POCO and hybrid
POCO reads throw JetCorruptDataException with MalformedValue, the column
name and the original cause. Explicitly lenient reads return empty text,
DBNull.Value or an unassigned/default property and emit a Trace warning;
listener failures do not abort the read. Valid negative dates, range boundaries
and null slots retain their normal representations. Writer snapshots refuse an
unreadable date even for an unrelated field update, leaving the database
bytes unchanged. Storage I/O exceptions propagate unchanged. The malformed
fixtures in FixedDateDecodePolicyTests and VariableDateDecodePolicyTests
are synthetic corruption evidence.
Assigned fixed-field slices are bounded by the fixed payload area, ending before
variable data or the row trailer. An out-of-range descriptor offset or size is an
unreadable assigned value, rather than a null field. String, typed, DataTable,
direct POCO and hybrid POCO reads share contextual MalformedValue errors in
strict mode and their normal missing-value representation with a Trace warning
in lenient mode. Null bits are checked before fixed bounds. Writer snapshots
refuse these malformed slices before changing bytes. The descriptor mutations
in FixedSlotBoundsDecodePolicyTests are synthetic corruption evidence.
Assigned MEMO/OLE slots with missing or reversed variable offsets use the same
long-value failure policy as unreadable descriptors: strict reads throw a
column-named JetCorruptDataException; lenient typed reads return DBNull.Value,
string reads return null cells, and POCO properties remain unassigned/default,
with a trace warning. Calculated MEMO slots use their resolved value type.
Writer snapshots refuse these assigned slices before mutation. Null-mask and
unrepresented-column slots remain missing without diagnostics. Assigned
fixed and variable Complex slots use the scalar MalformedValue policy and
require an exact four-byte, positive reference. Strict reads refuse malformed assigned
slots; lenient typed/string/POCO and DataTable reads return their missing-value
representation with a Trace warning. Materialized and projected streaming write-back snapshots retain malformed scalar
column context and add the resolved table name and physical data-page number.
The reason includes the physical row slot, offset and size and retains the
decoder’s reason; the error code, message and cause are unchanged. Writer snapshots
refuse these slots before
an unrelated update can erase them. Valid positive references with no items,
null-mask slots and unrepresented columns remain missing without diagnostics.
The public MEMO/OLE and calculated MEMO mutations in
LongValueSlotBoundsDecodePolicyTests and the public Complex mutations in
FixedComplexDecodePolicyTests and VariableComplexDecodePolicyTests establish
synthetic corruption behavior using writer-created storage with a complete
complex catalog. Native fixtures use fixed references; these mutations add no
native acceptance evidence for variable Complex storage.
Row encoding reserves every declared fixed slot even when its value is Null. Boolean values occupy null-mask bits and add no fixed payload. Variable data starts after the complete fixed area; row-capacity preflight includes those reserved bytes before allocating long-value storage. New Null slots are zeroed; native bytes beneath Null fields can be opaque and are not evidence of a value.
FixedNullScalarTests covers declared scalar widths in Jet3/Jet4/ACE and
byte-identical refusal when reserved Null slots make a row too large.
NativeFixedNullScalarTests independently authors Byte, Integer, Long, Currency,
Single, Double, Date/Time and GUID fields using DAO3.6, then compares Jet3/Jet4
fixed-area geometry through writer insert/update, native update and compact.
Mixed variable rows begin their text after all 55 fixed bytes. Native fixed-only
rows are 58/59 bytes; the writer’s existing unused variable trailer makes the
corresponding rows 60/63 bytes, accepted by the native read/update/compact checks.
These controls use dao360.dll 10.0.26100.5074.
NativeDroppedNullSlotTests separately pins native Jet3/Jet4 source and compacted
specimens after deleting leading/middle/trailing fixed fields and a variable
field beside surviving Null Double/GUID fields. Source Value/GUID offsets are
8/20; compact changes them to 4/12. The native old row retains 44 fixed bytes,
the native post-drop row uses 36, and native compact uses 28, excluding the
one/two-byte column-count prefix. Removed interior slots remain reserved; the
old trailing Double slot need not be reserved by new rows. Frozen raw controls
pin retained removed-field values and text starts; bytes beneath native Nulls
and new-row gaps remain opaque rather than assumed zero. Public insert/update,
native read/update/compact and typed/string/POCO reads qualify these shapes.
Provenance and hashes are in
the fixture inventory.
Fresh DAO120 Jet4/ACE, native Numeric/Large Number/DateTime Extended, remaining
dropped-slot variants and broader record limits remain F8/E2 work.
Native fixed Text occupies the descriptor’s full byte width. Jet3 uses the
database code page; Jet4/ACE uses uncompressed UTF-16LE. Its leading FF FE
bytes can be U+FEFF content and must not select the variable-Text compression
codec. Typed, string, direct POCO and hybrid POCO reads use
JetFormat.DecodeFixedText, preserving that character, spaces and embedded
NULs. Short assignments are space-padded to the byte width; over-width
assignments are refused instead of silently truncated.
FixedTextFixtureTests uses DAO36-authored CP1252/CP932 Jet3 fixtures and
DAO120-authored Jet4/ACE fixtures. NativeFixedTextPreservationTests compares
native descriptors and raw slots through unrelated updates and Add/Drop/Rename.
DAO36 accepts Jet3 Add/Drop/Rename, row updates/inserts and code-page-preserving
compact of the CP1252/CP932 fixtures. DAO120 accepts Jet4/ACE schema edits and
row mutation/compact. NativeJet3SchemaRewriteTests additionally verifies
native scalar/wide indexed rewrites, preserved object/security identity and
unchanged catalog keys through native open, mutation and compact.
Writer snapshots retain original ordinary fixed and variable Text slots, including fixed-slot bytes under null values. Unrelated updates, schema projections and cascades copy unassigned slots without decoding and re-encoding them, preserving malformed UTF-16 and invalid multibyte sequences. Explicit assignments encode normally, even when their decoded values equal the originals. Slot bounds and descriptor widths remain validated before mutation.
Variable Text retains the stored compression representation, including a native
FF FE 61 00 field that reads as a but would otherwise be normalized to
61 00. NativeVariableTextPreservationTests compares both frozen native
specimens through Add/Drop/Rename, index creation and unrelated updates in every
write mode, checks explicit equal assignment and exact rollback/retry, and uses
DAO3.6 and DAO120 to read and compact the edited file. Calculated text caches
retain their separate encoding and recomputation rules.
Jet4/ACE marked text starts with FF FE. Each single 00 at a character
boundary switches between one-byte Latin-1 and two-byte UTF-16LE mode.
The Unicode-mode terminator consumes one byte, including when the next byte is
nonzero. DAO120-authored abc中def stores
FF FE 61 62 63 00 2D 4E 00 64 65 66; treating that terminator as two zeroes
corrupts the Latin-1 suffix. NativeCompressedTextDecodeTests pins native field slices,
source hashes, offsets and exact native UTF-16 units for Jet4 and ACE.
A mode-only suffix emits no character. An incomplete final Unicode unit remains
bounded by the field slice; the reader does not reproduce a native corrupt-slot
overread into the following field.
Ordinary compressed variable Text beginning with U+FEFF uses
FF FE 00 FF FE, an optional contiguous UTF-16LE wide prefix, 00, then
the one-byte Latin-1 tail. Writing is supported for 6 through 255 UTF-16 units.
The tail contains only U+0001 through U+00FF, with at least five units without
a wide prefix or four with one. Prefix units exceed U+00FF, have nonzero low
bytes and exclude surrogates. Wide characters after the Latin-1 tail remain
unsupported. The complete encoding occupies at most the uncompressed byte width.
The whole value must fit the descriptor’s uncompressed character limit; an
oversized value refuses rather than truncating the escape or its tail.
Unsupported shapes refuse during whole-row preflight, before long-value
allocation, with table and column context.
NativeCompressedTextMarkerTests pins source and compacted Jet4 specimens
with writer-created schemas and DAO3.6-authored rows. The descriptor compression
bit and complete field slices distinguish them from a DAO UnicodeCompression
property that leaves the bit unset. Fresh native authoring before and after
compact matches complete writer payloads, including U+4E2D/U+03A9 prefixes,
250 wide units plus four Latin-1 units and one wide unit plus 253 Latin-1 units.
Native writer read/update/compact checks cover all 255 nonzero Latin-1 tail
characters. The local engine was DAO3.6,
dao360.dll 10.0.26100.5074; fresh DAO120 Jet4/ACE controls remain guarded.
A native short U+FEFF-plus-a assignment loses the marker on insertion.
Short values/tails, NUL, low-byte-zero wide units, surrogates, wide units after
the Latin-1 tail and calculated Text retain separate gaps.
Ordinary Jet4/ACE Memo decoding also uses its LVAL descriptor’s storage mode.
Inline and single-row payloads may use the marked text grammar; chained payloads
are plain UTF-16LE, where leading FF FE is literal U+FEFF content. Calculated
Memo text is plain UTF-16LE inside its calculated envelope, in both inline and
external storage. Preparation, direct encoding, materialized reads and
write-back snapshots use that distinction.
DAO120 controls accept compressed single-row Latin-1 values through 2048 characters; larger compressed expansions can fail exact native reads, and compressed bytes in chained storage read as plain UTF-16. The writer conservatively compresses ordinary Latin-1 Memo only through that verified boundary, then emits plain UTF-16. This is a safe writer policy, not a universal statement of native allocation choices: independently native-authored controls include compressed 1025- and 1500-character values and plain chained values below a page’s raw payload capacity. Native storage context matters more than the marker alone.
For ordinary Memo beginning with U+FEFF, Unicode-compressed columns accept
6 through 2048 characters when every following character is U+0001 through
U+00FF. The payload is FF FE 00 FF FE 00 followed by the one-byte Latin-1
tail. It matches retained DAO120-authored Jet4/ACE payloads and survives DAO3.6
Jet4 exact reads and compact at the inline/external and length boundaries.
Native checks include updates, deletion, schema rewrites and all 255 nonzero
Latin-1 characters. The local engine was DAO3.6, dao360.dll 10.0.26100.5074;
fresh ACE DAO120 read/compact controls remain guarded.
Shorter escapes can lose the marker during native compact. Without the column compression flag, native acceptance also depends on inline storage, which a wide row can force to spill. The writer therefore refuses every other leading U+FEFF ordinary Memo of at most 2048 characters before page allocation, including uncompressed columns, short values and non-Latin-1 or NUL tails. Large plain chained values retain their existing literal U+FEFF behavior. The remaining representations and fresh ACE checks stay open in F8/E2.
Stored MEMO/OLE payload lengths are exact. Every read path uses all 30 native length bits, including typed inline decoding; the upper two bits select storage mode. The declared length is checked against MaxLongValueBytes before copying inline bytes. Resource-limit refusals propagate in strict and lenient modes. An inline payload is bounded by its column slice, not the remaining page; a single-row payload must contain its declared bytes. A chain must supply its declared length without truncation or cycling. Exact reads throw JetCorruptDataException for malformed payloads and propagate storage I/O failures. They never manufacture placeholder text or substitute an empty payload for unreadable data.
Mixed scalar/OLE typed scans use the same row-layout and column-slice validation as projected rows. The compiled hybrid plan assigns scalar auto-properties and records only bound OLE slices. Each enumerator owns its slice array; cached plans contain no mutable row state. The scan retains the data or resolved overflow page through every awaited LVAL read, and resolves all bound OLE byte arrays before yielding the target. Arrays contain the stored bytes without OLE unwrapping or content interpretation; null and empty remain distinct.
Unreadable OLE payloads use LongValueReadPolicy: strict scans throw a column-named JetCorruptDataException, while lenient scans report the existing trace diagnostic and leave that property at its default. Cancellation, storage I/O failures and resource-limit refusals propagate. Hybrid eligibility excludes custom setters (whose order and exceptions are observable), Numeric columns, primitive scalar values stored in variable slots, and the existing unsupported calculated/MEMO/complex/hyperlink bindings. These keep the projection-aware fallback. Scalar-only projections retain their direct decoder.
After the page header, a data page holds a table of 16-bit row-offset slots,
one per row index: Jet3 keeps the row count at offset 8 and the slots from
offset 10, Jet4 and ACE at 12 and 14 (DataPageLayout). Rows are written from
the end of the page downwards. Each slot is:
| Bits | Meaning |
|---|---|
0x1FFF |
The row’s start offset on the page |
0x8000 |
The row is deleted |
0x4000 |
The row is an overflow row’s header (see below) |
Constants.DataPage holds these as RowOffsetMask, DeletedRowFlag and
OverflowRowFlag.
A row has no length field. It runs from its start offset up to the next
greater offset of any slot on the page, deleted slots included, or to the end
of the page for the highest row. Equal offsets must be skipped: Access leaves
deleted slots pointing at the same offset as a live row (indexTestV1997.mdb
page 18 has row 21 live at offset 756 and rows 22-26 deleted at 756, probably
left by page compaction). Taking the neighbour of whichever equal entry a binary
search lands on gave such a row zero bytes, so every read skipped it.
DataPageRows.FindNextRowStart is an upper-bound search over the sorted
offsets, and TryGetSlotBound / UsageMap.TryGetRowBound use a strict
“greater than” scan.
A row starts with num_cols, then the fixed area, then the variable-length
values, and ends with a trailer (forward order):
| Field | Jet3 | Jet4/ACE |
|---|---|---|
| EOD: the offset where the variable data ends, stored right after it | 1 byte | 2 bytes |
| Variable-column offsets, last column first | 1 byte each | 2 bytes each |
| Jump table (Jet3 only, below) | (rowLength - 1) / 256 bytes |
none |
var_len: the number of variable columns |
1 byte | 2 bytes |
| Null mask, one bit per column | ceil(num_cols / 8) bytes |
same |
RowFieldSizes holds the per-format field sizes, and
RowDecodePlan.TryParseRowLayout / ResolveColumnSlice read the trailer for
every read path (typed, string and direct decoders, catalog scans, the
writer’s snapshots and long-value walks).
The fixed area holds each fixed column at its descriptor’s offset_F.
Access reserves every fixed column’s slot, even when the last fixed columns
are Null: every Jet3 fixture does, and Jackcess notes that Access expects it.
RowEncoder.SerializeRow ends the fixed area after the last fixed column that
holds a value (or the slot of a Null Attachment or Complex column), so a
writer row whose last fixed columns are Null is shorter than Access’s. This
library’s reader, Jackcess and mdbtools accept both, because a Null fixed
column is read from the null mask; whether Access accepts the shorter form is
unchecked.
A row never spans pages, so it is at most the page size less the data-page
header and one row-offset slot: 2,036 bytes on Jet3 and 4,080 on Jet4/ACE
(DataPageLayout.MaxRowLength). MEMO and byte-array OLE payloads over the
native 64 stored-byte inline limit go to LVAL pages and take a 12-byte
descriptor in the row. The limit includes a calculated value’s envelope;
ordinary compressed Memo is measured by its encoded bytes.
TableRowStore.EncodeRow serializes the row with a zeroed placeholder for
each such header, with no I/O. While the row is longer than a page, it
moves the largest MEMO or byte-array OLE value still inline (the first of
equal ones in column order) to a placeholder too and measures again:
moving a value shrinks the row by its payload’s length, so it moves values
until their payloads cover the excess before it serializes the row again.
Calculated columns with a Memo or OLE result take part, measured by their
wrapped payload, and compressed text by its compressed bytes. An OLE value
given as a string stays inline, as UTF-8 bytes. Access moves long values
out of a full row too; which ones it picks is unchecked.
TableRowStore.InsertRowDataLocAsync calls EncodeRow before it writes
any LVAL or data page, so a row still too long with every such value moved
throws JetLimitationException with the file unchanged. (The LVAL pages
used to be written first, and stayed behind, unreferenced, after the
throw.) Before the check existed, the writer appended an empty data page
and then threw ArgumentOutOfRangeException from the copy, or
InvalidDataException after copying a row one byte too long over the
row-offset table; DataPageInserter.WriteRowToPageAsync now checks the
free space before it touches the page. Physical capacity is not native record
acceptance. Frozen DAO3.6/DAO120 controls with Id LONG and fixed binary fields
split at 255 bytes accept 2000 value bytes on Jet3 and 4051 on Jet4/ACE, and
refuse the next byte at schema append with DAO 3047. Their accepted raw rows
occupy 2003/4056 bytes including count/assignment overhead. Other field shapes
and writer-emitted rows between native shape limits and page capacity remain
unverified. CompactNativeRowBoundaryTests preserves these complete raw rows,
fixed-only rows, dropped fixed-slot bytes and real Jet3 jump markers across
256/512 boundaries through private assembly, followed by native compact/readback.
See the hashed fixtures and producer details in Databases/RowBoundaries/.
An update rewrites a row by deleting it and inserting the new version, so
RelationshipEnforcer.PlanCascadeUpdatesAsync, which writes nothing, calls
EncodeRow at its end for every child row the cascades rewrite, with the
key changes of every relationship that reaches the row, and
TableDataWriter.UpdateRowsAsync then calls it for every new version of its
own rows. It waits for the plan because a self-relationship’s cascaded key
goes into the update’s own new rows, so each is measured as it is written:
a row whose old key would not fit beside the update’s other assignments, but
whose shorter cascaded key does, is updated. No row is deleted before then,
so the encoder’s refusals, a row too long or a value such as a Currency out
of range, leave the file unchanged.
Jet3JumpTable executes generated Dafny sizing and boundary/dummy decisions.
The size selector proves the smallest self-consistent jump count directly;
the decoder stops at the first unreached or decreasing boundary. Native dummy
bytes and the ambiguous 255-variable-column refusal retain their existing
meaning. Span adapters preserve stack-allocated offsets and avoid row copies.
These properties cover the named jump controls; full row acquisition and
native compatibility retain their separate evidence and
formal boundary.
A Jet3 offset is one byte, so a row longer than 256 bytes needs the high part
of its offsets from a jump table between the offset table and var_len. The
rule follows mdbtools mdb_crack_row3 and Jackcess
TableImpl.readJumpTableVarColOffsets:
J = (rowLength - 1) / 256 entries, jump table included, so a
row of exactly 256 bytes has none. Entry k, for the boundary
256 * (k + 1), sits k + 1 bytes before var_len: the entry for 256 is
next to var_len, the highest boundary’s entry comes first in the row.var_len. Offset i is its low byte plus 256
times the number of entries that index i has reached (Jackcess consumes
them in order while entry == i).eodPosition / 256 < J),
that entry is a dummy and readers drop it. Access 97 writes 0xFF there.Only one Access-authored row in the fixtures has a jump table: MSP_PROJECTS
in test2V1997.mdb, 290 bytes with EOD 255 and a single 0xFF dummy. No
fixture row has a real entry, so real entries are checked against the
mdbtools/Jackcess rule. Jet3JumpTable holds the rule; until the reader used
it, rows whose variable data crossed offset 256 decoded truncated, empty or
shifted, and a 256-byte row was misparsed, because the reader skipped
rowLength / 256 bytes and never read them.
RowEncoder.SerializeRow writes the same layout. It sizes the jump table as
the smallest J with (lengthWithoutJumps + J - 1) / 256 == J
(Jet3JumpTable.CountForLength): a row whose other parts take 511 bytes
gets one entry (512 bytes in all), and one of 512 bytes gets two. It stores
the low byte of the EOD and of each offset. Jet3JumpTable.Write fills the entries and writes 0xFF for a
dummy; rebuilt from its offsets, the MSP_PROJECTS trailer comes out as
Access’s bytes. Before, the encoder wrote the one-byte fields with a checked
cast, so any Jet3 row whose EOD or an offset reached 256 threw
OverflowException: a table of 200 Long columns, Text values adding up to
about 250 bytes, or a MSysObjects row whose inline LvProp blob carried four
or five CLR defaults.
Limits of the Jet3 row:
num_cols and var_len are one byte, so a Jet3 table holds at most 255
columns (Access’s field limit too). CreateTableAsync and AddColumnAsync
reject a 256th column with JetLimitationException before writing
anything, and SerializeRow rejects a row of a wider table.JetLimitationException.Tables with no variable columns keep the writer’s EOD, jump table and
var_len = 0 trailer (the jump entries name the EOD, index 0), which every
reader skips because the TDEF has no variable columns. Access writes no
trailer for such rows: nwind.mdb’s fixed-only table has 24-byte rows.
When an update makes a row too large for its page, Access moves the row’s bytes to a new slot, usually on another page owned by the same table, and leaves the original slot behind as the row’s header:
0x4000. Its first four bytes are a pointer: the
target row index (1 byte) followed by the target page number (3 bytes,
little-endian). Constants.DataPage.OverflowPointerSize is 4.0x8000, so a scan never reads it on its own.num_rows counts the row
once, at the header (NorthwindTraders.accdb’s MSysObjects declares 239 rows:
199 live rows plus 40 headers).0x4000 and ignores the target’s deleted flag.Access-authored fixtures with overflow rows: NorthwindTraders.accdb (40
MSysObjects rows, among them Orders, Employees, Products and Welcome), the
Jet3 nwind.mdb (Categories, Customers, Employees and Suppliers in
MSysObjects), overflowTestV2000/V2003 (rows 3 and 5 of Table1),
testOleV2007 and extDateTestV2019 (user rows), testIndexCodes,
testIndexProperties and queryTestV2007 (catalog rows).
DataPageRows.ComputeRowDirectory returns, in slot order, every live row plus
every header (a slot with 0x4000 and without 0x8000) flagged
RowBound.IsOverflowPointer. OwnedDataPages.TryResolveOverflowRowAsync follows a
header to the row data, up to Constants.DataPage.MaxOverflowHops (8) hops, and
requires the target to be a data page of the same table and a slot the page
has. A pointer that cannot be resolved (shorter than four bytes, past the end of
the file, into another table, or a cycle) skips the row during read-only
traversal, as an undecodable row is skipped. With DiagnosticsEnabled, the reader traces the table-definition page, header slot and offset, and the reason it could not resolve the pointer. StrictParsing governs value parsing only.
Every read path uses the directory: the TableReader scans,
RowDecoder (catalog and flat-table scans), the index seek in IndexRowReader,
and the writer’s OwnedDataPages.ForEachLiveTableRowAsync, which visits an
overflow row at its header with RowLocation.PageNumber / RowIndex naming the
header and DataPageNumber / DataRowIndex naming the moved bytes.
EnumerateLiveRowBounds stays live-only, for usage-map and LVAL pages, which
never hold overflow rows.
Owned-page discovery (ValidateOwnedDataPagesAsync) counts each header as one
row, so a table with overflow rows validates its usage map instead of falling
back to a whole-file scan.
TableRowStore.MarkRowDeletedAsync) flags the header 0xC000, as
Jackcess deleteRow does. With DeletedRowDataMode.Clear or
SecureEraseMode.DeletedRowsAndFreedPages it also zeroes the moved bytes and
the header’s pointer and every validated intermediate pointer slot, and under secure erase frees the row’s long values. Intermediate slots are cleared only after the entire chain resolves to a row of the same table; invalid pointers never authorize clearing another table’s bytes.0xC000. The writer never creates overflow rows.RowLocation.PageNumber /
RowIndex); anything that reads or patches the row’s bytes uses
DataPageNumber (TryReadColumnValuesTypedAsync, and the complex-reference
reads and writes in ComplexColumnManager when an attachment or multi-value
item is added or a row’s complex children are deleted).DataPageNumber in
JetErrorInfo.PageNumber, including projected streaming snapshots used by
complex-parent predicates. An overflow header identifies the logical row;
the destination page identifies the malformed stored value. Existing
decoder context and underlying I/O failures are retained.RelationshipChildRowLocator resolves it to the moved bytes, so cascade
deletes and updates reach overflow rows on the seek path as well as on the
snapshot path.DropTableAsync and schema rewrites free the long values of overflow rows
too (TableSchemaEditor.ReclaimTableStoragePagesAsync).A TDEF names two usage-map rows, each by a 1-byte row index and a 3-byte page
number: used_pages, the table’s data pages, and free_pages, those with room
for a row. Each real index’s descriptor names a third (used_pages, its
tree’s pages). The rows sit on a usage-map page, a data page whose rows are
maps. New writer-created Jet4 and ACE tables have one, with the owned-pages
and free-space rows at rows 0 and 1 and initial real index n at row n + 2,
69 bytes each (CatalogArtifactWriter, DataPageInserter). Jet3 uses native
133-byte rows. Additional index rows can spill onto new map pages. Column LVAL
maps have their own pointers in the native TDEF trailer.
Existing Access-authored maps can use larger rows and different index row
numbers. Index maintenance follows each descriptor’s page and row pointer,
updates that row within its actual bounds, and preserves every unrelated
row, including owned-pages, free-space and long-value maps. In
NorthwindTraders.accdb, MSysObjects (TDEF 2) uses map page 6: owned
row 0 is 381 bytes, free-space row 1 is 69 bytes, and its two indexes point
to rows 16 and 17. Rows 2 through 15 must not be repurposed as index maps.
A row’s first byte is its type:
| Type | Layout |
|---|---|
0x00 INLINE |
start_page (4 bytes), then a bitmap whose bit k is page start_page + k: 64 bytes, so 512 pages, in a 69-byte row |
0x01 REFERENCE |
4-byte pointers, 17 in a 69-byte row; pointer i names a bitmap page or is 0 |
A REFERENCE bitmap page has page type 0x05, then 01 00 00, then a bitmap
whose bit k is page i * (pageSize - 4) * 8 + k. Each pointer covers 32,736
pages of 4 KB, so 17 reach page 556,511, past the 524,288 pages of a 2 GB file.
UsageMapEditor keeps the writer’s rows complete:
DataPageInserter.MarkPageInOwnedMapAsync marks every data page the writer
appends in the owned-pages and free-space rows (MarkPageAsync). The bit is
set when the INLINE window holds the page. A row that lists no page yet
moves its window to the page (start_page 0 for a page below 512, otherwise
the page rounded down to a multiple of 8); a row that lists pages is promoted
to REFERENCE, keeping them, with a bitmap page for each window it needs.
Jackcess’s UsageMap promotes the same way, but first moves a window whose
pages, the new one included, still fit one. Before, a page outside the
window was left out, so both maps of a table past 512 pages lost pages and
the reader fell back to the whole-file owned-page pass; and when the window
started at page 0, the first page past it moved the window without moving
the bits already set, so they named other pages.WriteRowAsync):
INLINE when one window holds them, REFERENCE otherwise. A row that is
REFERENCE already stays REFERENCE and keeps its bitmap pages, so rebuilding
an index again takes no new page. Before, a tree spread over two windows
threw NotSupportedException.DropTableAsync and the schema rewrites free the bitmap pages of the table’s
REFERENCE rows with its other pages
(TableSchemaEditor.ReclaimTableStoragePagesAsync).Imported Jet4/ACE system tables use the same owned/free map maintenance as user tables. In TestV2010, a Table4 rewrite can exhaust the existing MSysObjects data page and relocate its catalog row to a reused data page. Leaving that page out of MSysObjects’ owned map makes native compact omit the table, although indexed lookup and the reader’s ownership-scan fallback find it. The policy permits catalogued local system tables and the fixed catalog root; map edits follow the stored pointers and participate in rollback.
DAO retains the free-space candidate until an attempted row fails to fit; the writer follows that rule and also clears a page with no remaining slot or room for a nonempty row. Deletion packs the removed payload without renumbering surviving slots, restores the free hint and allows reuse after reopening. This includes overflow headers, intermediate pointers and moved payloads, while retaining unrelated moved targets. Empty deleted slots can point at the page end. Insertion appends a slot and retains the 255-slot cap. The map and payload writes participate in transaction rollback. Native Jet3, Jet4 and ACE checks are described in usage-map maintenance.
The owned-pages rows of the Access-authored fixtures, read raw with PowerShell (92 files, the encrypted ones skipped; 1,689 table definitions):
0x01 and
pointer 0 naming page 87, which starts 05 01 00 00 and lists that table’s
data pages (86, 88, 89 and 90 in binIdxTestV2010.accdb); in
testRefGlobalV2000.mdb an empty table’s REFERENCE row has no pointer. Their
free-space rows are INLINE. The writer’s REFERENCE rows and bitmap pages
follow this layout.OwnedUsageMapGrowthTests.Dao_RecordCount_SeesRowsOnPagesPastInlineWindow
checks DAO read and compact of a writer-created ACE table whose owned-pages
map grows to REFERENCE. Successful native runs are recorded in the
validation matrix. Native system,
index, free-space and column LVAL checks and the shared-storage policy are
described in usage-map maintenance.
Writer snapshots require complete row traversal and use a write-back decode
plan. The declared directory must fit its page, and every live slot must start
past the directory, inside the page, without sharing another live slot’s
offset. Deleted slots may alias offsets, and deleted overflow targets remain
valid. An unresolved overflow pointer, a live row too short for its
column-count field, or a layout that cannot be decoded raises a contextual
“MalformedValue” corruption error before row updates or schema rewrites. The
snapshot cannot omit that row and reinsert only the readable rows. Ordinary
public table reads retain their own decode-fault policy. Catalog identity scans
require complete MSysObjects row directories and resolved overflow chains
regardless of StrictParsing; traversal failures become CorruptCatalog with
table, page and reason context before any table list or system lookup is cached.