Status: All shipped phases listed in §4.2; outstanding work tracked in the same table.
Empirical appendix: format-probe-appendix-complex.md — annotated hex dumps of MSysComplexColumns, every MSysComplexType_* template table, an attachment-bearing parent table (Documents), and the hidden flat tables from ComplexFields.accdb. Regenerate via dotnet run --project tools/JetDatabaseWriter.FormatProbe -- complex.
Validation requirement: see index-and-relationship-format-notes.md §8.
⚠️ Reverse-engineered notes. mdbtools documents complex columns only superficially. The authoritative open-source reference is Jackcess (Java, Apache-2.0) — specifically
com.healthmarketscience.jackcess.impl.complex.*. Field names and offsets in this document are derived from Jackcess source and the existing reader code in this repo (ComplexColumnReader,AttachmentWrapper).
SystemCatalogBootstrapper creates the core MSysACEs, MSysQueries and MSysRelationships tables, installs native permissions and patches the system-table header pointers. Fresh database creation runs that service before ComplexColumnManager creates the ACE-only complex catalog and type templates. Both execute inside the creation write scope and use the same pager and catalog-artifact writer. Complex-value storage does not own general database security or core catalog bootstrap.
Access 2007 introduced three “complex column” kinds. All three are stored the same way: a 4-byte per-row complex reference in the parent row, pointing into a hidden child (“flat”) table that holds the actual values. All three kinds share the column-type byte 0x12 (COMPLEX_TYPE) — confirmed both by Jackcess DataType.java and by our format-probe across the entire test corpus (no on-disk fixture in tests/JetDatabaseWriter.Tests/Databases/ carries 0x11 on a complex column). The writer emits 0x12 for attachment and multi-value parent descriptors. Type 0x11 is native Jet4 BIGBINARY, a fixed 3992-byte payload used by MSysAccessObjects.Data; it is decoded as binary bytes and never enters complex-column processing. This matches Jackcess’s native datatype definition and the bundled Access-authored Jet4 fixtures.
| Kind | Storage in child table |
|---|---|
| Attachment | One row per attached file. Columns: FileURL, FileName, FileType, FileFlags, FileTimeStamp, FileData. |
| Multi-value | One row per value. Columns: a single value column whose type matches the user-declared element type. |
| Version history | One row per historical edit. Columns: the version’s text (a Memo column named after the append-only Memo column) and Modified_<GUID> (Date/Time). Only meaningful on memo columns flagged “Append Only” in Access. |
The discrimination between attachment / multi-value / version-history is not done by the column-type byte. It is done by the linked flat table’s schema and/or the value of MSysComplexColumns.ComplexTypeObjectID (which points at one of the MSysComplexType_* template tables — see appendix).
The reader implements all three kinds. Writer support has shipped through phase C10 — see §4.2.
Phase labels record implemented library behavior. Native validation of remaining descriptor and schema variants remain open. The validation matrix distinguishes Microsoft fixtures and successful DAO checks from library round trips.
Inside the parent table’s TDEF column descriptor block (25 bytes per col on Jet4 / ACE), a complex column has:
| Field | Value |
|---|---|
col_type |
0x12 (COMPLEX_TYPE) for all three kinds — attachment, multi-value, and version-history. |
col_len |
4 |
bitmask |
0x07 (always — per mdbtools “always have the flag byte set to exactly 0x07”) |
misc |
The 4-byte ComplexID (called “complexid” in mdbtools), stored at the misc offset. This is the value used to match the parent descriptor to its MSysComplexColumns row. |
Because bitmask = 0x07, the column is treated as a fixed-length 4-byte column for row-layout purposes. The 4-byte payload in each data row is the per-row complex reference value that joins this parent row to child rows in the flat table’s FK column. The MSysComplexColumns.ConceptualTableID catalog column is separate: it identifies the parent table object/TDEF page for the complex-column definition.
Access gives every complex column its own index on that reference: unique, Required, and named <column>_<32 hex> (truncated so the name fits in 64 characters), for example Attachments_CFC7F98A63064F2DBDB6822D653AB763 in ComplexFields.accdb, attach-data_071D71EDD53D45A1A9089929F06857D9 and VersionHistory_F5F8918F-0A3F-4D_6E54CCBB170741DD8FD837271ED8B90C in complexDataTest*.accdb, and ProductCategoryImage_5C9A6A17CF9D4E1CA64DB2DECECEBB16 in NorthwindTraders.accdb. Its keys use the Long Integer layout (7F + big-endian int with the sign bit flipped; 7F 80 00 00 01 for reference 1), as Jackcess IndexData maps COMPLEX_TYPE to its integer descriptor. IndexKeyEncoder encodes complex-column keys that way, so updates, deletes and inserts on Access tables with complex columns keep these indexes current (ComplexColumnIndexFixtureTests). Users cannot declare an index on a complex column; the writer emits the native unique, Required reference index automatically.
Assigned fixed and variable parent references must contain exactly four bytes
and a positive ID. Strict public reads report MalformedValue with column
context; lenient reads return their missing-value representation and a trace
warning. Writer snapshots refuse invalid assigned references before mutation.
Valid positive references without child items, null-mask slots and absent
columns remain missing without diagnostics. The fixed and variable decode-policy
tests use synthetic corruption; native reference allocation evidence follows.
Per-row references come from the TDEF complex AutoNumber. ACE TDEF headers hold a second counter at offset 28 (TDefHeaderLayout.ComplexAutoNumber; Jackcess JetFormat.OFFSET_NEXT_COMPLEX_AUTO_NUMBER, mdbtools ct_autonum; Jet3 and Jet4 have none): the last per-row complex reference handed out. Access gives each row one reference, shared by all its complex columns, even a column with no items; it does not reuse the references of deleted rows. In the fixtures:
| Fixture / table | TDEF@28 | Live slots |
|---|---|---|
ComplexFields.accdb Documents |
2 | [1], [2] |
complexDataTestV2007.accdb Table1 |
7 | [1,1,1] … [4,4,4] (rows deleted in Access) |
complexDataTestV2010.accdb Table1 |
8 | [1,1,1] … [4,4,4] |
NorthwindTraders.accdb ProductCategories |
17 | 1 … 16 |
Byte 24 is 0x01 in all of them; bytes 32–39 hold values whose meaning is unknown, and the writer does not touch them.
The writer does the same. ConstraintRegistry.ApplyAsync gives each inserted row one reference shared by its complex columns. A supplied reference must be positive and consistent across the row; otherwise the call is refused. The next automatically assigned reference follows the persisted TDEF@28 counter. ComplexReferenceSeedReader validates existing parent slots and flat-table foreign keys before allocation: missing or invalid references, duplicate references within a parent column, missing required metadata, and a reference above the counter are contextual corruption errors. It does not reconstruct a stale counter. Statement/savepoint rollback restores registry allocations and removes entries loaded after its snapshot; MutationExecutor raises TDEF@28 once for each prepared table application that requests complex-reference counter maintenance.
AddAttachmentAsync and AddMultiValueItemAsync require a valid existing parent reference covered by TDEF@28 before appending an item. They never fill missing slots or rebuild parent identities. Schema rewrites preserve surviving complex slots, validate existing identities and allocate references only for new complex columns. The rewrite carries TDEF@28 to the new TDEF, as it carries the ordinary AutoNumber counter at offset 20. UpdateRowsAsync refuses assignments to complex columns, preserving their references and items. A valid reference with no items still reads as DBNull.
MSysComplexColumns catalog tableVerified against ComplexFields.accdb (appendix). Actual schema is 5 columns (column names and order below are probe-confirmed):
| Column (verified) | Type | Meaning |
|---|---|---|
ColumnName |
Text(510) |
Name of the parent column (e.g. "Attachments"). |
ComplexID |
LongInteger (fixed_off=12), AutoNumber (flags 0x17) |
Per-database ID for this complex column. Matches the 4-byte value the parent TDEF stores in the column descriptor’s misc+misc_ext slot (see §2.1). |
ComplexTypeObjectID |
LongInteger (fixed_off=0) |
MSysObjects.Id of the type-template table — one of MSysComplexType_Long, MSysComplexType_Text, MSysComplexType_Attachment, etc. The template’s schema dictates the kind (attachment vs multi-value of a given inner type). |
ConceptualTableID |
LongInteger (fixed_off=8) |
Parent table object/TDEF page for the complex-column definition. Every schema rewrite preserves this identity by publishing isolated rebuilt storage onto the original TDEF page; newly added complex descriptors are rebound from the temporary parent at publication. |
FlatTableID |
LongInteger (fixed_off=4) |
MSysObjects.Id of the hidden child (“flat”) table. |
The rows above are in the catalog’s logical column order; the fixed_off values give each column’s offset in the physical fixed-width row area, which Access orders independently (ComplexTypeObjectID=0, FlatTableID=4, ConceptualTableID=8, ComplexID=12).
ComplexID is an AutoNumber column. The MSysComplexColumns TDEF counter at offset 20 holds the last ID handed out, including dropped columns: 1 in ComplexFields.accdb, 7 in both complexDataTest*.accdb (largest remaining ID 4) and 3 in NorthwindTraders.accdb. GetNextComplexIdAsync validates the required descriptor and stored IDs against that counter and refuses inconsistent metadata instead of recovering an ID from existing rows. A new column takes counter + 1, and InsertMSysComplexColumnsRowAsync raises the counter after insertion. Transaction rollback discards that raise with the other page writes. Native-fixture and corruption tests cover allocation, dropping columns, rollback and refusal without mutation.
Complex-column renames and parent-identity rebinding prepare typed catalog replacements before writing. Unassigned values retain their CLR types, empty strings, Nulls and fixed Text bytes. Each replacement batch clears the old row payloads and updates the system indexes once through the incremental system-table path; an unsupported index mutation refuses the operation instead of rebuilding the catalog.
There is no ParentTable / ParentColumn column in MSysComplexColumns.
Parent complex descriptors require positive, unique IDs. The reader joins them
to ComplexID and verifies both the parent ConceptualTableID and column name.
Missing required catalog tables, columns or mapping rows raise contextual
CorruptCatalog errors in strict and lenient modes. Descriptor-only ID fallback
still requires one matching parent/column mapping with an unambiguous flat-table
identity; a flat-table name suffix never establishes ownership.
A nonzero ComplexTypeObjectID must resolve to a recognized template name on a
real local table with a valid TDEF and matching payload schema. Attachment
payloads require all six native field names with their corresponding types;
scalar templates require their canonical Value field and element type. Matching
unknown scalar discriminators in the template and flat payload remain visible as
their own hexadecimal type name, as in the Jackcess
unsupportedFieldsTestV2007 fixture’s 0x48 fields. They are not reinterpreted
from the template’s name or treated as catalog corruption. Template and payload
types must still agree.
Version-history payload names can contain native GUIDs, so template/flat agreement
does not require those names to be identical. Flat tables must carry their complex
catalog flags and a valid parent-reference field. Schema renames can preserve that
internal field’s old name. A zero template ID retains flat-schema classification.
Strict reads reject malformed optional descriptors. Lenient reads omit their cells without reintroducing invalid or shared flat tables through fallback, while preserving valid sibling columns. Required parent identities remain corruption errors in every mode. Real I/O failures propagate and retries do not use cached partial results. Synthetic catalog mutations establish these refusal behaviors; the existing Access-authored complex fixtures establish supported valid layouts.
AccessReaderOptions.MaxComplexDiscoveryEntries bounds metadata discovery and
aggregate fallback scans across one table read, with a default of 65,536 entries.
The caller can raise this positive limit for a larger catalog. A refusal is
JetLimitationException with ComplexDiscoveryBudgetExceeded, including in
lenient mode; it is not classified as corrupt input. Linked readers inherit the
limit. Broader aggregate parser and schema-allocation budgets remain in S1.
MSysComplexColumns is now created by the ACCDB full-catalog scaffold on every fresh ACCDB built via CreateDatabaseAsync (Phase C1, 2026-04-25). That same scaffold also creates the core MSysACEs, MSysQueries, and MSysRelationships tables expected by DAO compatibility paths. The catalog rows carry Flags = 0x80000000 so the tables are excluded from ListTablesAsync. ACE only — Jet3/Jet4 .mdb scaffolds skip these tables.
Verified naming and flag conventions (appendix):
f_<32-hex-uppercase>_<userColumnName>, e.g. f_A3DF50CFC033433899AF0AC1A4CF4171_Attachments. The 32 hex characters are a GUID without dashes. Access truncates the whole name to 64 characters: in complexDataTestV2007.accdb the column VersionHistory_F5F8918F-0A3F-4DA9-AE71-184EE5012880 has the flat table f_0BFE159C44334BFB968A820EA89C2D7E_VersionHistory_F5F8918F-0A3F-. That fixture’s _<userColumnName> and <parentTable>_<userColumnName> columns are 52 and 58 characters, so it does not show how Access shortens those. ComplexColumnManager.BuildFlatTableName truncates the whole generated name to the Access object-name limit.MSysObjects.Flags: 0x800A0000 (probe-confirmed in both NorthwindTraders.accdb and ComplexFields.accdb).MSysComplexType_<TypeName> (Long, Text, Attachment, UnsignedByte, Short, IEEESingle, IEEEDouble, GUID, Decimal). Flags = 0x80030000.MSysObjects (Type = 1), but the system-flag bit (0x80000000) hides it from Access UI. Its TDEF table-type byte is the ordinary user-table 0x4E, as in the Microsoft fixtures. Marking the TDEF as system-table 0x53 makes DAO expose the child recordset as read-only even when catalog permissions and indexes are present. Generated attachment and multi-value children therefore retain 0x4E. ComplexColumnsFlatIndexesTests.ComplexFlatTables_UseNativeUserTableType and the guarded ComplexSchemaDaoTests pin this distinction.The flat table has all the value-bearing columns of the complex type, plus two extra columns:
ComplexColumnReader.CreateRowResolverAsync). Both Access and this writer name it _<userColumnName>; the reader (ComplexColumnReader.FindForeignKeyIndex) takes the LongInteger column named exactly _<userColumnName>; failing that, a _-prefixed LongInteger column that is not an AutoNumber and does not end with _<userColumnName>; then any _-prefixed LongInteger column; else the first LongInteger column. The other LongInteger column, <parentTable>_<userColumnName>, is the flat table’s own autonumber key and is not a join key.The flat table also requires:
index-and-relationship-format-notes.md.Generated flat-table text fields omit AllowZeroLength, matching the native
child-field property shape. DAO120 child recordsets accept empty items when
that property is absent even though their property getter reports False;
persisting an explicit False changes assignment behavior. Required, defaults,
AutoNumber and index constraints remain independently enforced through ordinary
constraint hydration. NativeAllowZeroLengthTests checks native-created and
generated children through item replacement, native mutation and compact.
MSysComplexColumns.Type = 4, on-disk col_type = 0x12)Per Jackcess AttachmentColumnInfoImpl:
| Flat-table column | Jet type | Notes |
|---|---|---|
| (autonumber PK) | LongInteger, autoincrement |
Required by complex-column protocol. |
| (FK) | LongInteger |
Holds the per-row complex reference value from the parent row. |
FileURL |
Memo |
May be empty/null. |
FileName |
Text (max 255) |
Display filename. |
FileType |
Text (max 255) |
File extension without leading ., preserving native or caller casing. Drives the COMPRESSED_FORMATS skip-list (§3.2). |
FileFlags |
LongInteger |
Reserved by Access; Jackcess emits null. |
FileTimeStamp |
DateTime |
When the file was attached. Jackcess uses the system clock. |
FileData |
Ole |
Wrapper-encoded payload. NOT raw bytes. See §3. |
If the flat table on disk has columns whose names don’t match these exactly, Jackcess assigns by type/order: the LONG column is FileFlags; the SHORT_DATE_TIME column is FileTimeStamp; the OLE column is FileData; the MEMO column is FileUrl; the first TEXT column is FileName; the second TEXT column is FileType. We should write canonical names, but the reader path should be tolerant.
MSysComplexColumns.Type = 3, on-disk col_type = 0x12)| Flat-table column | Jet type | Notes |
|---|---|---|
| (autonumber PK) | LongInteger, autoincrement |
|
| (FK) | LongInteger |
|
value |
varies | Whatever element type the user declared (text, long, etc.). |
The shipped declaration surface is the ColumnDefinition.IsMultiValue init-only property (see §4.2 C2 row); the inner value column inherits the ColumnDefinition’s declared type and length, and, for decimal items, its NumericPrecision and NumericScale (Access’s Decimal(18,0) when none is declared).
The DAO-created ComplexDescriptorVariants.accdb fixture pins all eight native
primitive multi-value kinds. Ordinary parent references use flags/extra flags
07/00; append-only history references use 07/20. Flat foreign keys use
02/08, and scalar AutoNumbers use 06/04. Multi-value Value descriptors
have extra flags zero. Default Decimal uses a variable descriptor (flags=02),
precision 18, scale 0 and declared length 9, with 17-byte stored values. Explicit
Decimal(10,2) output retains declared length 17 and passes native read, insert
and compact checks; that acceptance does not assert byte-identical declaration.
All surviving columns, including a late-added Text multi-value column, share
references 1 and 2 in the two live parent rows. The persisted counter remains 3
after deleting the highest parent. Library Add/Rename/Drop preserves these
references, counter and append-only properties. Native mutation/compact may
advance the counter beyond live references. Calculated descriptor preservation
is checked separately against calcFieldTestV2010 storage/result type variants.
MSysComplexColumns.Type = 2, on-disk col_type = 0x12)Verified against complexDataTest*.accdb Table1.VersionHistory_F5F8918F-0A3F-4DA9-AE71-184EE5012880, whose flat table holds five versions of the append-only Memo column, all modified on 2011-09-12:
| Flat-table column | Jet type | Notes |
|---|---|---|
_VersionHistory_<GUID> (FK) |
LongInteger |
The parent row’s per-row complex reference. |
<memo column name> |
Memo |
Historical text snapshot, named after the append-only Memo column. |
Modified_<GUID> |
DateTime |
When Access recorded this version. |
<table>_VersionHistory_<GUID> (PK) |
LongInteger, autoincrement |
Only meaningful on memo columns marked “Append Only” in Access. The reader selects the one nonnull Memo payload in each sparse history row and the validated Date/Time timestamp column. GetMultiValueItemsAsync returns each version as a MultiValueItem with Modified set, and the row cell uses the 'V' kind of ComplexCellValue, which carries the timestamps (ComplexCellValue.ReadMultiValueItems reads both 'M' and 'V' cells). Versions keep the flat table’s order; Jackcess sorts them newest first. Ordinary inserts and assigned updates on native append-only Memo columns append versions in the same statement as the parent write. Each history item exposes its native flat-table Id and the source MemoColumnName. A shared history table has one Memo payload column per source field and one sparse row per assignment. An explicit same-value update still records a version; an omitted Memo that remains Null on insert records none. Explicit Null and empty strings record the native U+0000 payload. Timestamps use local wall-clock time. Native all-null rows left by dropping a Memo payload are omitted from public histories. Direct history item edits and general DML on hidden history tables are refused.
The reader decodes this in AttachmentWrapper.TryDecode, from the stored FileData bytes (the flat table’s OLE column is read as its stored bytes, as every read API returns OLE values). The writer encodes it in AttachmentWrapper.Encode.
Verified against all 24 Access-authored attachments in the fixtures: 2 .txt in ComplexFields.accdb Documents.Attachments, 3 .txt in each of complexDataTestV2007.accdb / complexDataTestV2010.accdb Table1.attach-data, and 16 .jpg in NorthwindTraders.accdb ProductCategories.ProductCategoryImage. The layout below is what all of them hold, and it matches Jackcess AttachmentColumnInfoImpl.encodeData / decodeData.
+0 uint32 LE typeFlag 0x00 = stored raw, 0x01 = zlib-compressed
+4 uint32 LE dataLen length of the UNCOMPRESSED content stream (header + file), for both flags
+8 ---- body the content stream, as is (typeFlag 0) or as a zlib stream (typeFlag 1)
The content stream (after inflating, for typeFlag 1):
+0 uint32 LE headerLen length of this header, INCLUDING the 4 bytes of headerLen itself: 12 + 2 * extChars
+4 uint32 LE flag always 1 (Jackcess CONTENT_HEADER_FLAG); meaning unknown
+8 uint32 LE extChars length of the extension in CHARACTERS, including its NUL ("txt" -> 4)
+12 bytes fileExtension UTF-16LE, NUL-terminated (2 * extChars bytes)
+headerLen bytes payload the file bytes: dataLen - headerLen of them
So a .txt file has headerLen 20 and dataLen = 20 + the file length. A compressed body is a full zlib stream: the header 78 5E (deflate, 32 KB window, FLEVEL 1, the level Jackcess’s Deflater(3) also announces), the deflate blocks, and the big-endian Adler-32 of the content stream. DAO120 writes an empty extension as just its NUL (extChars 1, headerLen 14), including for empty payloads. It preserves the source extension case in both FileType and the content header. AttachmentWrapperDaoTests reproduces native extensionless, empty, raw, compressed, uppercase and longer-extension variants, compares their on-disk wrappers with writer copies, and extracts all payloads through DAO before and after compact. Captured native wrappers also have ordinary regression coverage without DAO.
AttachmentWrapper.Encode writes this layout with the extension case preserved, matching DAO. It frames the zlib stream by hand (Infrastructure/Adler32.cs), because the netstandard2.1 build has no ZLibStream, and compresses with DeflateStream at CompressionLevel.Optimal. Its deflate blocks differ from Access’s, which no .NET compressor reproduces (they also differ between the zlib builds of .NET 8 and .NET 10), so tests never pin compressed bytes. Re-encoding each Access attachment gives Access’s wrapper byte for byte for the raw .jpg files, and the same typeFlag, dataLen, zlib header, Adler-32 and inflated content for the compressed .txt files (ComplexColumnsAttachmentFormatTests.AccessAuthoredFileData_ReEncodesToSameWrapper). AddAttachment_WriterBytes_DecodeWithJackcessSemantics decodes the writer’s stored bytes the way Jackcess decodeData does (zlib, then exactly dataLen - headerLen file bytes), including a 20 KB value on a chained LVAL.
AttachmentWrapper.TryDecode reads the extension from bytes [12, headerLen) up to its NUL. A typeFlag-1 body must carry a native zlib header, inflate to exactly dataLen bytes and match its Adler-32 checksum. Inflation stops at the declared length plus one byte to detect excess output. Raw-deflate payloads from unreleased library builds are rejected; they are not a Microsoft compatibility requirement.
MaxAttachmentContentBytes limits one declared uncompressed content length, including the extension header. MaxAttachmentContentBytesPerOperation reserves each declared length before decoding and defaults to 256 MiB across one root operation, including multi-table reads, queries, Include and linked sources. MaxAttachmentDecodeWorkBytesPerOperation defaults to 2 GiB and conservatively reserves each stored wrapper length plus four times its declared content length for input processing, inflation, checksum, extension decoding and copying. Independent operations have independent budgets; positive caller overrides permit larger trusted reads. Budget refusals propagate in strict and lenient modes. Compressed-input refills, output reads, extension processing and payload copies check cancellation in chunks no larger than 8 KiB; Adler-32 checks every 5,552 bytes. Streaming queries retain their budget across yields and release their operation lease on completion, cancellation or disposal.
Access stores a small set of formats raw and deflates everything else, including formats that are compressed already, such as gz and mp3. The writer uses Jackcess 4.0.12’s list, which its author took from Access (“Use the file formats which ms access stores uncompressed when writing an attachment … and deflate everything else”):
private static final Set<String> COMPRESSED_FORMATS = new HashSet<>(
Arrays.asList("jpg", "jpeg", "gif", "png", "zip", "cab", "docx",
"xlsx", "xlsb", "pptx"));
The match is case-insensitive. The fixtures confirm only .jpg (raw) and .txt (compressed); the rest of the list is Jackcess’s observation, not Microsoft documentation. Memo Unicode compression is a separate storage rule and does not apply to attachment compression. DAO120 controls include compressed single-row Memo values longer than 1024 characters and plain chained values; see Memo text storage.
The reader (see §1) implements Attachment / Complex column-type recognition, the MSysComplexColumns join, and attachment payload decode (AttachmentWrapper.TryDecode, §3). Schema metadata is exposed via IAccessReader.GetComplexColumnsAsync; typed item enumeration is exposed via GetAttachmentsAsync / GetMultiValueItemsAsync (see §4.2 C4).
ComplexColumnReader classifies a column by its MSysComplexType_* template name (ComplexTypeObjectID). A column with no template (ComplexTypeObjectID 0, which files written by builds of this library before C10 hold) is classified from its flat table’s schema instead (ClassifyFlatTable): a FileData column means an attachment; leaving out the foreign key and the AutoNumber key, one value column means multi-value, and a Memo plus a Date/Time column a version history. GetComplexColumnsAsync, the row cells and the type names all use that classification. GetColumnMetadataAsync reports each complex column’s TypeName, keyed by ComplexID so a rename keeps it (ReadColumnTypeNamesAsync): "Attachment", "Version History", or "Multi-value " plus the display name of the flat table’s value column type ("Multi-value Text" for complexDataTest’s multi-value-data, "Multi-value Long Integer" for a writer column of int), falling back to the type the template declares when the flat table cannot be read, and "Complex" when the column cannot be resolved at all. Before, every complex column that joined to MSysComplexColumns reported "Attachment". ComplexColumnsInfoTests covers this.
Row reads (Rows, ReadTableAsync, Rows<T>, index seeks and Query<T>) replace each complex column’s 4-byte reference with a ComplexCellValue cell (byte[], since complex columns report ClrType = byte[]). ComplexColumnReader.CreateRowResolverAsync validates selected metadata before iteration. A bounded scalar item-ID/FK/location inventory checks flat-table-global item identities and groups row locations without decoding unrelated payloads. Each consumed parent resolves only its own attachment, multi-value or history payloads; retained payload cells belong to the current parent. Usable FK-leading index candidates must agree with the scalar inventory before payload admission; unindexed sources use the grouped locations instead of rescanning for every parent. MaxComplexItemIdentityEntriesPerOperation bounds retained item metadata across the operation and its nested or authorized linked reads. Attachment content/decode budgets remain cumulative across consumed parents. A row whose reference has no flat rows reads as DBNull. ComplexCellValue.ReadAttachments / ReadMultiValueItems decode a cell into the same records GetAttachmentsAsync / GetMultiValueItemsAsync return, because both paths share ComplexColumnReader’s flat-table decode. RowsAsStrings and ReadTableAsStringsAsync return the same cell as a data:application/octet-stream;base64, URI, or an empty string. A version-history cell (kind 'V') carries each version’s text and its Modified_<GUID> timestamp, as GetMultiValueItemsAsync returns them in MultiValueItem.Modified (§2.4.3).
| Phase | Scope | Status |
|---|---|---|
| C1 | Empty-DB scaffold: add ACCDB full-catalog system tables, including MSysComplexColumns. |
✅ Shipped. ACCDB full-catalog only. Helper: ComplexColumnManager.ScaffoldSystemTablesAsync creates core MSysACEs / MSysQueries / MSysRelationships, MSysComplexColumns, and later the complex type-template tables. Catalog rows carry MSysObjects.Flags = 0x80000000 and are excluded from ListTablesAsync. Tests: ComplexColumnsWriterTests plus fresh ACCDB DAO compact coverage. |
| C2 | ColumnDefinition.IsAttachment / IsMultiValue declaration surface + ColumnInfo.Misc round-trip in TDEF emission (the 0x07 bitmask, the 4-byte misc ComplexID slot, col_len = 4). |
✅ Shipped. Public init-only props on ColumnDefinition. Descriptor offset _colMiscOff = 11 (Jet4/ACE). Tests: ComplexColumnsWriterTests. |
| C3 | CreateTableAsync emits the hidden flat child table, allocates a fresh ComplexID, writes the MSysComplexColumns row, and patches the parent descriptor. |
✅ Shipped. Helper: CatalogArtifactWriter.CreateTableAsync (formerly AccessWriter.CreateTableInternalAsync). Flat table named f_<32-hex>_<userColumnName>, MSysObjects.Flags = 0x800A0000. Per-flat PK/FK indexes are emitted by C7 (§4.3); ComplexTypeObjectID is populated by C10 (§4.6). |
| C4 | Per-kind row APIs (AddAttachmentAsync, AddMultiValueItemAsync) and attachment payload encode (§3). |
✅ Shipped. Helpers: AccessWriter.AddAttachmentAsync / AddMultiValueItemAsync, AttachmentWrapper. Reader-side: AccessReader.GetAttachmentsAsync / GetMultiValueItemsAsync. Inline-OLE 256-byte cap was lifted by C8 — oversized payloads now ride on LVAL data pages. |
| C5 | Cascade flat-table rows on parent delete. | ✅ Shipped. PlanComplexChildDeletesAsync finds matching and cascaded flat rows before mutation, deduplicating each physical row across groups. AddComplexChildDeletesAsync adds their deletions to the shared MutationPlan; MutationExecutor applies them in planned table order and finalizes each flat table’s count and indexes, retaining rebuild when original key hints are unavailable. Statement/savepoint rollback restores partial application on cancellation or failure. Cost: one O(P) flat-table scan per (parent table × complex column). Evidence: ComplexColumnsCascadeDeleteTests, ComplexColumnDeleteIndexIntegrityTests, MutationPipelineTests, UpdateDeleteCancellationTests. |
| C6 | DropTableAsync cascade for hidden flat tables and MSysComplexColumns rows. |
✅ Shipped. Helper: DropComplexChildrenForTableAsync, called from DropTableAsync. Removes MSysComplexColumns and MSysObjects catalog rows for each child flat table. It reclaims each flat table’s TDEF, data, reachable index and usage-map pages and releases its live rows’ long values through the same storage reclaimer as an ordinary table drop. Dropping a complex column does the same for that column’s flat table. |
| C7 | Per-flat-table indexes Access expects: autoincrement scalar PK column, primary key on the scalar, normal index on the FK back-reference, and (attachment only) a composite secondary index on (FK, FileName). Lifts the C3 “no PK / no autoincrement / no FK back-reference index” caveat. | ✅ Shipped. Helper: BuildFlatTableSchema (returns (ColumnDefinition[], IndexDefinition[])) wired through EmitComplexColumnArtifactsAsync and reused by WriteComplexItemCoreAsync via ApplyConstraintsAsync so the autoincrement scalar PK is seeded per insert. See §4.3. |
| C8 | LVAL chain emission for oversized MEMO / OLE / Attachment payloads. Uses the native 64-byte inline payload limit. | ✅ Shipped. Helpers: LongValueEncoder.PrepareLongValues / WriteLongValuesAsync plus shared LongValueDescriptor / LongValueStore descriptor, page-buffer, and chain helpers. Sizing uses TableRowStore.EncodeRow in shared preflight and in InsertRowDataLocAsync: any Ole / Memo value whose encoded payload exceeds the inline cap is replaced with a pending PreEncodedLongValue (a zeroed 12-byte header), and the row is serialized, which checks its values and its size, before any page is written; while the row is too long for a page, the largest inline Ole / Memo values are replaced the same way (LongValueEncoder.CollectInlineLongValues). Only then is each payload staged onto Access-style LVAL pages (single-page bitmask 0x40; chained bitmask 0x00 with one row per page, walked in reverse so each predecessor row carries its successor’s lval_dp pointer) and the placeholder replaced with a sentinel carrying the finished header, which the encoders splice through verbatim. The current writer emission limit is MaxPayloadBytes = 16 MiB - 1; native lengths occupy 30 bits. LVAL pages are emitted as page-type 0x01 with the LVAL signature and descriptor token Access uses. See §4.4. |
| C9 | Schema evolution on parent tables that already contain complex columns: lift the NotSupportedException thrown by BuildColumnDefinitionFromInfo so AddColumnAsync / DropColumnAsync / RenameColumnAsync can run against tables with attachment / multi-value columns. |
✅ Implemented (2026-04-25; DAO compact-hardened 2026-05-20 and 2026-05-23). BuildColumnDefinitionFromInfo now returns IsAttachment / IsMultiValue ColumnDefinitions with the original ComplexId preserved from ColumnInfo.Misc. RewriteTableAsync preserves the parent rows’ per-row complex references and keeps flat children + MSysComplexColumns rows attached. Every rewrite publishes the temp table root and its continuation chain onto the original TDEF page, so surviving MSysComplexColumns.ConceptualTableID values keep naming the original parent identity. Surgical post-rewrite cleanup runs for dropped/renamed complex columns. The rebuilt TDEF re-emits Complex (0x12) with the correct misc field, and HydrateConstraintsFromTableDef skips the Flags flag-bit interpretation for complex columns (the on-disk Flags = 0x07 is a magic marker, not real IsNullable / IsAutoIncrement bits). 7 round-trip tests in ComplexColumnsSchemaEvolutionTests; DAO compact coverage in DaoCompact_ComplexColumnsWithLvalPayload_SurviveCompactAndRepair and fresh ACCDB complex storage tests. See §4.5. |
| C10 | Scaffold the nine MSysComplexType_* template tables (UnsignedByte, Short, Long, IEEESingle, IEEEDouble, GUID, Decimal, Text, Attachment) and populate MSysComplexColumns.ComplexTypeObjectID with the matching template id instead of the placeholder 0. |
✅ Implemented (2026-04-25). Helpers on ComplexColumnManager create the type templates immediately after MSysComplexColumns, and the static ResolveComplexTypeTemplateName(ColumnDefinition) lookup is wired into EmitComplexColumnArtifactsAsync. ACE only (Jet3 / Jet4 .mdb skip the templates because complex columns are an Access 2007+ feature). Each template carries MSysObjects.Flags = 0x80030000 (system + the 0x30000 marker Access uses for type-template tables) so they are excluded from ListTablesAsync. 6 round-trip tests in ComplexColumnsWriterTests. See §4.6. |
AttachmentRecord.Id and MultiValueItem.Id carry the stored positive scalar
AutoNumber, including in complex row cells. Individual ReplaceAttachmentAsync,
RemoveAttachmentAsync, ReplaceMultiValueItemAsync and RemoveMultiValueItemAsync
scope that ID to one verified parent reference. Replacement retains the native ID,
reserved fields and unassigned payload metadata; deletion retains the AutoNumber
high-water mark. Both maintain row counts and every child index within the caller’s
statement/savepoint. Malformed or duplicate identities are contextual corruption;
lenient reads omit invalid and ambiguous items.
Append-only Memo history uses sparse native rows: exactly one assigned Memo
payload, a shared parent reference, an item ID and a local-wall-clock timestamp.
The shared template and flat table can have several Memo payload columns.
MemoColumnName identifies the source on each returned version. Rename changes
the paired template/flat payload names. Nonfinal drop removes that descriptor,
name and ten-byte LVAL map record, preserving column/variable-slot gaps and native
dormant rows; final Memo
drop removes the parent history reference and its private artifacts. Shared or
unverified templates are protected from edits and reclamation.
Generated flat-table and column names are truncated to 64 characters without splitting a UTF-16 surrogate pair. If the generated scalar primary-key name collides with the foreign-key name after truncation, it receives a _1 suffix within the same limit.
The flat-child schema emitted by BuildFlatTableSchema (called from EmitComplexColumnArtifactsAsync during CreateTableAsync):
_<userColumnName> (LONG, FK back-reference), FileData (OLE), FileFlags (LONG), FileName (TEXT 255), FileTimeStamp (DATETIME), FileType (TEXT 255), FileURL (MEMO), <parentTable>_<userColumnName> (LONG, autoincrement scalar PK). Three indexes ship: MSysComplexPKIndex (PK on the scalar), _<userColumnName> (normal index on the FK), and IdxFKPrimaryScalar (composite normal index on (_<userColumnName>, FileName)). The Access-specific descriptor flags/extra flags/misc values for these hidden flat-table columns are emitted through writer-owned ColumnDefinition descriptor overrides._<userColumnName> (LONG, FK back-reference), Value (CLR type from the user ColumnDefinition; a decimal value column takes its declared NumericPrecision and NumericScale, which TDefPageBuilder.ValidateColumnForFormat checks before the parent table is written), <parentTable>_<userColumnName> (LONG, autoincrement scalar PK). Three indexes: PK on the scalar, a normal index on the FK back-reference, and unique IdxFKPrimaryScalar on (FK, Value). The composite index prevents duplicate values within one parent while allowing the same value under different parents. Complex-item insertion checks every flat-table unique index before writing, including native indexes on reopened files.String multi-value items use Text with MaxLength 1-255; 0 selects Text(255). Negative lengths and lengths above 255 are refused during schema preflight. Inserting text longer than the stored column limit is refused before allocating a parent reference or inserting a flat row.
WriteComplexItemCoreAsync resolves the flat-table name from the catalog (helper ResolveFlatTableNameAsync) and calls ConstraintRegistry.ApplyAsync before InsertRowDataAsync, so the autoincrement scalar PK is seeded from the larger of the flat table’s TDEF AutoNumber counter and its existing rows; after the insert, AutoNumberMaintainer.UpdateHighWaterAsync raises that counter, so IDs freed by deleting the top flat rows are not reused in a later session. The constraint registry is hydrated from the persisted FLAG_AUTO_LONG bit when the writer instance did not declare the table itself, so reopening a native table still drives the autoincrement correctly.
C7 caveats:
AddAttachmentAsync / AddMultiValueItemAsync insert the child row and rebuild the emitted flat-table indexes, including the attachment (FK, FileName) composite index. A parent-row delete removes its hidden child rows, adjusts each flat table’s row count and rebuilds its indexes before returning. Complex table and column drops maintain the MSysComplexColumns and MSysObjects row counts and indexes; renames and parent-table transplants rebuild the rewritten complex catalog indexes. Under SecureEraseMode.DeletedRowsAndFreedPages, deleting flat or catalog rows also scrubs and frees their MEMO and OLE long values.ComplexTypeObjectID names the matching MSysComplexType_* template (§4.6). A missing or invalid template is refused before writing complex artifacts.ComplexColumns*Tests suite. DAO CompactDatabase coverage now exercises writer-authored attachment rows on the ComplexFields fixture in ComplexFixture_WriterAttachmentRowsSurviveCompactAndRepair (with ComplexFixtureCatalog_SurvivesCompactAndRepair pinning the compacted complex catalog), plus a Northwind-hosted writer-created attachment and multi-value table with flat-table indexes, schema rewrite, and a chained-LVAL attachment payload in DaoCompact_ComplexColumnsWithLvalPayload_SurviveCompactAndRepair; see writer-disk-format-validation-matrix.md.Phase C8 lifts the inline cap that limited C4 attachment payloads to ~256 bytes. The long-value pass (LongValueEncoder.PrepareLongValues, then WriteLongValuesAsync once the row is known to fit a page) runs in InsertRowDataLocAsync and fires for Ole / Memo columns whose encoded payload exceeds the in-row inline cap (Constants.LongValue.MaxInlineBytes = 64, measured after text encoding and any calculated envelope). Smaller values keep the inline path (WrapInlineLongValue, bitmask 0x80) while the row fits a data page; when it does not, the largest of them, the first of equal ones in column order, move to LVAL pages until it does (TableRowStore.EncodeRow).
12-byte LVAL header layout (matches LongValueDescriptor and AccessReader.ReadLongValueAsync):
+--------+--------+--------+--------+
| memo_len (24 LE) | bitmask| bytes 0..3
+--------+--------+--------+--------+
| lval_dp (32 LE) | bytes 4..7 ((page<<8) | row_index)
+--------+--------+--------+--------+
| LVAL token (32 LE) | bytes 8..11 (Jet4/ACE: copied to bytes 8..11 of every LVAL page in the value; Jet3: 0)
+--------+--------+--------+--------+
bitmask values produced by C8:
0x80 — inline (small payloads; bytes 4..11 zero, payload follows the header in the row body).0x40 — single LVAL page. lval_dp points at one row on a freshly-appended LVAL data page; the row body is the payload (no next-pointer prefix).0x00 — chained LVAL pages. lval_dp points at the first chained row, whose first 4 bytes are the next-pointer (LE (page<<8)|row), followed by that page’s chunk of the payload. The terminal row’s next-pointer is 0.LVAL page layout (one row per page, written by LongValueStore.BuildSinglePageBuffer / BuildChainedPageBuffer from the per-format LvalPageLayout):
page_type = 0x01 (ordinary data page with an LVAL marker in bytes 4..7; 0x05 is the usage-map page type in this codebase, not the writer’s LVAL page form).LVAL.DataPageLayout): num_rows = 1 at offset 12 and the row offset at 14 on Jet4/ACE, at 8 and 10 on Jet3.row start − 12.Allocation order for the chained form is reverse: EncodeAsLvalChainAsync appends the last chunk’s page first (next-pointer = 0), then walks backwards so each newly-appended page can carry its successor’s lval_dp as its row-prefix next-pointer. The header’s lval_dp ends up pointing at whatever page was appended last (the highest page number, holding the first chunk). Access 97 chains run in ascending page order instead; readers follow the pointers either way.
Chunking math:
pgSize − LvalPageLayout.MinRowStart: 4096 − 20 = 4076 bytes on Jet4/ACE, 2048 − 12 = 2036 on Jet3.C8 caveats:
Constants.LongValue.MaxPayloadBytes = (1 << 24) − 1 (just under 16 MiB) per single MEMO / OLE / Attachment value is a writer implementation limit. The native length field has 30 bits, and readers can opt into a larger budget. Larger payloads throw JetLimitationException.SecureEraseMode.DeletedRowsAndFreedPages additionally zeroes deleted payloads and freed pages. Table drops and schema rewrites use the same release path. A chain hop that is not a live row of an LVAL page ends the walk. See usage-map maintenance.MSysObjects.LvProp / LvModule / LvExtra) use the same caps. MSysObjects rows are inserted through TableRowStore, whose long-value pass moves an LvProp blob over 64 bytes to LVAL pages like any other OLE value (Jet3LongValueTests.CreateTable_LvPropOver256Bytes_PropertiesRoundTrip).tests/JetDatabaseWriter.Tests/ComplexColumns/ComplexColumnsLvalChainTests.cs (single-page form, chained form, deflate-compressed text payload). Automated DAO CompactDatabase coverage now includes writer-authored attachment payload bytes on the ComplexFields fixture in ComplexFixture_WriterAttachmentRowsSurviveCompactAndRepair and a Northwind-hosted writer-created attachment table with a chained-LVAL .jpg payload in DaoCompact_ComplexColumnsWithLvalPayload_SurviveCompactAndRepair; see writer-disk-format-validation-matrix.md.Phase C9 lifts the NotSupportedException previously thrown by BuildColumnDefinitionFromInfo so that AddColumnAsync / DropColumnAsync / RenameColumnAsync work on parent tables that already carry attachment / multi-value columns. The on-disk byte-format of the parent TDEF and the hidden flat children is unchanged — C9 is a control-flow change in the rewrite path that preserves the existing artifacts when possible and surgically removes / renames them when the user mutation explicitly targets a complex column.
How surviving complex columns ride through RewriteTableAsync:
BuildColumnDefinitionFromInfo now returns a ColumnDefinition flagged with IsAttachment / IsMultiValue and the original ComplexId recovered from ColumnInfo.Misc. The default-projection and rename-column projection forward IsAttachment / IsMultiValue / MultiValueElementType / ComplexId so the rebuilt TDEF re-emits Complex (Flags = 0x07) with the correct misc field.PrepareComplexColumnAllocationsAsync only allocates fresh ComplexId values for columns whose ComplexId == 0; preserved columns bypass it entirely, so no new flat child table is emitted.MSysComplexColumns.ConceptualTableID values stay unchanged, including during complex-column drops and renames. Only newly added descriptors are rebound from the temporary parent to the original parent. Dropped children are reclaimed surgically after publication; renamed artifacts retain their flat-table IDs.HydrateConstraintsFromTableDef now skips the Flags flag-bit interpretation for Complex columns. The on-disk Flags = 0x07 is a magic marker (per mdbtools docs) — interpreting bit 0x04 as FLAG_AUTO_LONG would mark a byte[] column as auto-increment and fail the IsIntegralType check during the temp-table RegisterConstraints pass.Surgical post-rewrite cleanup runs from the rewrite path itself:
DropSingleComplexChildAsync(columnName, complexId) — invoked once per dropped complex column (matched by ComplexId between existingDefs and newDefs). Deletes the matching MSysComplexColumns row (matched by ColumnName + ComplexID), adjusts the MSysComplexColumns TDEF row count, and drops the hidden flat-table catalog row in MSysObjects. The normal table-storage reclaimer releases the child’s TDEF, data, reachable index, usage-map and live LVAL storage; secure erase and transaction rollback apply. A candidate must be a catalogued local complex flat table, and no surviving complex catalog row may reference it. Idempotent; tolerates missing rows.RenameComplexColumnArtifactsAsync(oldName, newName, complexId) — invoked once per renamed complex column (matched by ComplexId, with names in existingDefs and newDefs that differ ordinally, so a case-only rename counts). Mark-deletes the matching MSysComplexColumns row, then re-inserts it (updateTDefRowCount: false) with ColumnName rewritten to the new name. The hidden flat-table’s catalog name (f_<hex>_<oldName>) is left unchanged — readers resolve the flat name via FlatTableID → MSysObjects.Name, and the cosmetic suffix carries no semantic meaning.C9 caveats:
PrepareComplexColumnAllocationsAsync allocates fresh IDs for the appended ColumnDefinition (its ComplexId == 0), and EmitComplexColumnArtifactsAsync runs at the end of CreateTableAsync (called by the rewrite for the temp table) to emit the new flat child + MSysComplexColumns row. The pre-existing complex columns continue to ride through unchanged.AddAttachmentAsync / AddMultiValueItemAsync after rename still work because the FK back-reference column on the flat table (_<userColumnName>) keeps its original name; the reader resolves the flat table via MSysComplexColumns.FlatTableID and GetAttachmentsAsync returns every flat row tagged with its FK back-reference; row reads join that FK to the parent’s complex slot, which the rebuild preserves (next bullet). The WriteComplexItemCoreAsync parent-row predicate matches on the user’s PK columns, not on the renamed complex column.TableSnapshotReader.ReadTableSnapshotAsync) keeps ComplexIdRef values for complex columns, and RowEncoder writes those values back to the rebuilt parent row so flat-table FK joins continue to resolve after AddColumnAsync / DropColumnAsync / RenameColumnAsync.UpdateRowsAsync and cascade updates) decodes OLE cells as their stored bytes, as every read API does (OleObjectValue unwraps them on request). A MEMO / OLE value whose LVAL data cannot be read becomes an UnreadableLongValue, and a rewrite that would store it throws JetCorruptDataException with UnreadableLongValue before any page changes, instead of storing placeholder text such as (memo on LVAL page). See tests/JetDatabaseWriter.Tests/Writer/LongValueWriteBackTests.cs.ComplexColumnsSchemaEvolutionTests covers library Add/Drop/Rename of complex columns and scalar siblings. ComplexSchemaDaoTests.NativeComplexTable_CompleteRewritePreservesValuesIndexesCountersAndAppendOnly applies the complete sequence to Microsoft-authored NorthwindTraders.ProductCategories, ComplexFields.Documents and both complexDataTest.Table1 variants. It compares surviving scalar/attachment/multi-value/history content, reference-index flags/columns, persisted complex counters and AppendOnly properties, generates and drops new complex columns, and retains every unassigned catalog field and application blob. ComplexFields.Documents and both complexDataTest.Table1 variants perform DAO attachment/multi-value writes and compact/readback. Northwind performs native metadata/read/extraction and compact: its unchanged ProductCategories table already refuses parent updates with DAO error 3892 because its data macro calls the application-only GetAuditFieldsUserName. ComplexSchemaMacroDaoTests pins that independent original-fixture behavior; schema rewriting preserves the macro bytes rather than discarding them. Native attachment mutation on that fixture remains dependent on its Access application. The pre-native inserted reference follows the persisted seed even when the seed exceeds all live references; native mutations may advance that counter further. NativeSchemaDaoTests.DaoAuthoredWideAttachment_RewriteSurvivesNativeMutationAndCompact separately covers Add/Drop/Rename of a DAO-authored 202-column multi-page TDEF. Native acceptance of those schema edits does not establish append-only Memo history writes or individual complex-item replacement/removal.MSysComplexType_* template tablesPhase C10 lifts the C3 caveat that wrote MSysComplexColumns.ComplexTypeObjectID = 0. Every fresh ACCDB built by CreateDatabaseAsync now also emits the nine type-template tables Access expects, and the C3 row-emit path resolves the matching template id by name and persists it on the catalog row.
Templates emitted (all ACE-only, all carry MSysObjects.Flags = 0x80030000):
| Template name | Schema | Used when |
|---|---|---|
MSysComplexType_UnsignedByte |
Value: BYTE |
MultiValueElementType = typeof(byte) |
MSysComplexType_Short |
Value: INT |
MultiValueElementType = typeof(short) |
MSysComplexType_Long |
Value: LONG |
MultiValueElementType = typeof(int) |
MSysComplexType_IEEESingle |
Value: FLOAT |
MultiValueElementType = typeof(float) |
MSysComplexType_IEEEDouble |
Value: DOUBLE |
MultiValueElementType = typeof(double) |
MSysComplexType_GUID |
Value: GUID |
MultiValueElementType = typeof(Guid) |
MSysComplexType_Decimal |
Value: NUMERIC |
MultiValueElementType = typeof(decimal) |
MSysComplexType_Text |
Value: TEXT(255) |
MultiValueElementType = typeof(string) |
MSysComplexType_Attachment |
FileData OLE, FileFlags LONG, FileName TEXT(255), FileTimeStamp DATETIME, FileType TEXT(255), FileURL MEMO |
IsAttachment = true |
Schemas come from format-probe-appendix-complex.md §MSysComplexType_* against ComplexFields.accdb.
Helpers now live on ComplexColumnManager and are called from AccessWriter:
CreateMSysComplexTypeTemplatesAsync — emits the nine tables in declaration order (TDEF page + catalog row, no indexes, no rows). Skipped for Jet3 / Jet4 .mdb; every fresh ACCDB receives these templates.ResolveComplexTypeTemplateName(ColumnDefinition) (static) — returns the canonical template name for a complex column declaration, or null if the element type has no matching template.ReadRequiredComplexTypeTemplateAsync resolves the catalog ID (= TDEF page), checks file bounds, reads the definition and verifies the native column schema. Allocation validates every required template before creating the parent; emission uses the same check before creating child artifacts.C10 caveats:
MSysComplexType_* definition. Missing, out-of-file, unreadable or malformed templates produce a contextual corruption error before any parent or child mutation. Genuine filesystem errors propagate.Value column use type 0x10, flags 0x02, extra flags 0, misc bytes 12 00 00 00 (precision 18, scale 0), and col_len = 9. Each stored Decimal value still occupies 17 bytes. Descriptor width and row storage width are separate; schema rewrites preserve the original descriptor.ComplexColumnsWriterTests (template scaffolding, hidden-from-ListTablesAsync, attachment template column list, Jet4 skip, attachment + multi-value ComplexTypeObjectID non-zero). Writer-authored complex payloads on the ComplexFields fixture have representative DAO CompactDatabase coverage in ComplexFixture_WriterAttachmentRowsSurviveCompactAndRepair, with ComplexFixtureCatalog_SurvivesCompactAndRepair covering the compacted complex catalog; the Northwind-hosted compact test remains the Access-authored-base coverage for schema evolution and chained-LVAL attachment payloads. See writer-disk-format-validation-matrix.md.General DAO validation rules live in dao-validation-strategy.md, and cross-feature coverage lives in writer-disk-format-validation-matrix.md. Same as the index doc, with one addition specific to attachments:
ComplexFields.accdb) → re-emit → re-read → byte-compare attachment payloads (post-decode)..jpg payload must be stored with typeFlag=0x00 (raw); a .txt payload must be stored with typeFlag=0x01 (a zlib stream, §3.1). Open in Access and save the attachment back to disk via the GUI (or DAO Field2.SaveToFile) — verify the saved file is byte-identical to the input. Native DAO extraction and compaction now exercise compressed attachment payloads; the validation matrix records the tested revisions. A passing compression case does not establish every wrapper variant.tests/JetDatabaseWriter.Tests/Databases/ComplexFields.accdb. This is the existing read-side fixture; the writer tests should round-trip it.0x07) and complexid in misc field. (Access uses 0x12 / COMPLEX_TYPE for all complex columns; native Jet4 0x11 is BIGBINARY — see §1.)AttachmentColumnInfoImpl.java — wrapper-header encoder/decoder, COMPRESSED_FORMATS skip-listComplexColumnInfoImpl.java — flat-table protocol (PK + FK columns, diffFlatColumns)ComplexDataType.java — type-discriminator integer valuessrc/JetDatabaseWriter/ComplexColumns/ComplexColumnReader.cs (CreateRowResolverAsync, bounded identity inventory and per-parent payload decode), src/JetDatabaseWriter/ComplexColumns/Models/AttachmentWrapper.cs, src/JetDatabaseWriter/Models/ComplexCellValue.csindex-and-relationship-format-notes.mdEvery emitted single-row LVAL payload ends at the physical page boundary, including short single-page values and the last chunk of a chain. The row directory stores the actual start; free space excludes the header and row slot. DAO characterization of a 372-byte compressed attachment wrapper on an ACE page established start 3724 and free space 3708. With identical payload bytes and token, placing that row at offset 20 caused extraction to fail and compacting to lose its content; end packing preserved all 6144 payload bytes before and after compacting. Compression framing was unchanged. Complex-item parent predicates now use strict projected typed rows and native comparison semantics. Null and empty values remain distinct, fixed Binary values use their declared zero padding, and complete Memo comparison keys avoid truncation to index widths. Schema edits initialize references only for newly added complex columns; existing slots are preserved. Inconsistent references and counters are refused before mutation.