PhysicalRollback owns preparation, commit decisions, restoration and cleanup for statement transactions, explicit transactions and physical shrinking. File-backed operations use one PersistentRollbackJournal containing a hash manifest and selective original raw page images for both live rollback and process-crash recovery. Original page bodies are captured once before overwrite or truncation. Non-file operations use raw before-images through the same owner, with temporary-file spill for large shrink tails. Stored encrypted bytes remain encrypted in either backend.
Up to seven explicit levels share one page journal and page budget, matching DAO3.6 on Jet3 and DAO120 on Jet4/ACE. Each inner level captures a journal savepoint and the writer’s allocation and constraint state; rollback invalidates catalog caches. Inner commit merges its prior images into its parent without writing or flushing the database; inner rollback restores that level. An outer rollback also discards committed inner work. Lifecycle operations resolve the innermost level first, and writer disposal rolls back the complete stack. Native Execute (with flags 0 or 128) and Recordset writes succeed through level five and refuse levels six and seven with DAO error 3003 on all three formats. Writer mutations enforce that same boundary before changing journal or cached state; the sixth and seventh explicit scopes may open but must end before further mutation.
Only the outermost commit enters the persistent journal and physical replay protocol below. Nesting adds no persistent identity or recovery protocol to caller streams: they still require caller-enforced exclusive access and retain in-process undo only.
<database>.jdw-journal.preparing under exclusive database coordination. Acquire raw page digests once per exclusive store lifetime, then reuse the ledger only while every successful aligned overwrite, contiguous complete-page append and truncation is tracked. Persist a SHA-256 digest for each original page, including an incomplete final page when present. An uncertain or untracked effect invalidates the ledger and forces fresh acquisition; standalone journal users acquire their own manifest. Record the original length, full-path identity and manifest checksum in the header, and flush the completed header and manifest to disk. Close it, promote it by a non-overwriting adjacent rename to .jdw-journal, then acknowledge its parent-directory fence on the net11.0 library asset on Windows and reopen the published journal before permitting writes. An existing final journal is never overwritten. Preparation is recovery evidence: readers refuse it, and writer open validates its complete manifest, records and current database bytes before replay or cleanup. Partial, invalid or conflicting preparation is retained and refused. New preparation never blindly overwrites surviving evidence.A failure while recording the commit decision has an uncertain outcome. The writer faults and leaves the database and journal available for recovery rather than starting rollback behind a potentially durable commit record. Cleanup failure after a completed durable decision does not turn a successful commit into a rollback. Each completed journal attempts cleanup once; disposing it again cannot delete a later transaction’s journal. Journal creation failure faults the writer; live preparation failures attempt to remove their unused sidecar without masking the original error.
The existing pager and store gates serialize journal creation, raw intent recording and physical writes. Existing cooperative commit/page locks remain in place. File statement commits request durable flushes even when UseTransactionalWrites is false.
Writer open recovers before reading database headers or checking passwords. The JDWUNDO3 journal has a fixed 128-byte header with a random generation at bytes 88 through 95, a 32-byte SHA-256 digest for each original page and fixed-size page/control records. Header, manifest and records have independent checksums. Original-image records encode the aligned page offset as Int64.MinValue + offset; nonnegative offsets identify after-images, and -1/-2/-3 identify terminal decisions, truncation and recovery-start records. Only 2 KiB and 4 KiB pages are accepted; database length is capped at 2 GiB and journal length at 16 GiB. Page buffers and offset metadata are bounded by those limits.
Recovery validates the header, complete manifest, every complete record, full-path identity and current database before writing any recovery bytes. Every original-image body must match its manifest digest independently of its record checksum. An overwrite intent requires an earlier original image for an existing page; truncation requires original-image coverage of the entire removed original tail. For an undecided transaction, the current length must be between the smallest authorized retained length (the original length if no truncation was recorded) and the largest authorized write end. Before recovery starts, a shortened file must match an exact recorded truncation target; intermediate restored lengths are accepted only after the recovery-start record. Untouched original pages must match their manifest digest. Each byte of a captured page must equal the original byte or a recorded intended version for that page; appended pages also permit their original zero bytes. This permits torn writes and interrupted partial undo without accepting unrelated changes. After a validated recovery-start record, zero bytes are also permitted within the authorized removed tail so recovery can restart after partial re-extension; the retained prefix still requires original or recorded bytes. An incomplete final record is ignored because it could not authorize a database write; a complete record with an invalid checksum is refused.
A valid final commit or rollback decision must match the complete current database hash; recovery then keeps those bytes and removes the stale journal. Otherwise a fully validated undecided journal records recovery-start authorization when needed, restores the captured original pages and original length, flushes the database, records its terminal hash and removes the journal. Repeating recovery after an interruption is safe because restored bytes still match their original images and authorized tail gaps can be restored again. Before appending a recovery or terminal record, an incomplete trailing record is removed so records remain aligned. Invalid final journals and conflicting content are retained and refused without replay. A complete final or preparation-only journal can be recovered even if no database write has started. A truncated manifest is refused and retained. When both names survive, the preparation header, manifest and complete records must be a byte-identical prefix of the final journal, including generation, before joint cleanup; conflicting generations or bytes authorize no cleanup.
Strict path writers (UseLockFile and RespectExistingLockFile enabled) retain the ordinary machine/user slot and add a second standard 64-byte slot containing JetDatabaseWriter and a random 31-character generation. A fixed 200-byte checksummed JDWLOCK1 receipt at <database>.jdw-owner binds the canonical database path and exact 128 lock bytes. Both contents are flushed before acquisition returns. Every file-backed writer open validates an existing receipt, including when ordinary lock-file creation is disabled. Recovery requests exclusive receipt and lock handles and removes only exact matching generations. It closes the lock handle before deleting its name while retaining the exclusive receipt and database handles, then closes the receipt before deleting its name; this ordering permits Windows cleanup and preserves cooperating-process exclusion on Unix. Live holders, changed bytes, foreign or malformed receipts, wrong paths and populated legacy locks without receipts are retained and refused. An exclusively available all-zero vacant legacy lock can be removed after validated journal recovery. Cleanup precedes journal deletion so a failed cleanup retains terminal evidence. Reader opens reject a receipt before appending any ordinary lock slot. A kill before receipt completion can leave conservatively refused evidence; receipt content flushing does not establish namespace durability. Recovery and lock cleanup remain distinct from password validation.
Encryption maintenance records a fixed 256-byte Prepared header at the deterministic <database>.jdw-replacement name. The checksummed header binds the canonical full path, a generation GUID, and exact old/new lengths and SHA-256 hashes. Both files are capped at 2 GiB and copied/hashed with 64 KiB buffers. Immutable adjacent .jdw-replace-<generation>.old and .new.tmp copies survive publication; a separate .publish.tmp copy permits retry. Peak space can include three additional database-sized files, besides a caller-supplied staging file.
Prepared with no complete valid decision selects old bytes. A complete valid 96-byte Committed record selects new bytes. An incomplete final decision selects old; a complete invalid record refuses. Recovery validates every surviving immutable copy, canonical image and publication copy before mutation. A partial publication copy must be a prefix of a complete authorized image. Missing selected content, foreign generations, conflicting bytes, malformed or directory evidence, and coexisting transaction journals refuse without deleting evidence. Random legacy candidates without a valid intent are never guessed.
AccessWriter.OpenAsync(path) runs replacement recovery before requiring the destination to exist. Readers, encryption detection, caller FileStream opens and database creation refuse pending intents. Recovery retains exclusive intent and generation-bound ownership receipt/native-lock handles. The source/canonical lease is retained through publication and cleanup: Windows permits deletion/rename but denies other reads and writes; Unix uses the BCL exclusive cooperative lease. An absent native lock is recreated only from a validated receipt; when neither ownership name exists, recovery creates and holds a fresh strict generation. Existing unrelated or malformed ownership is retained and refused. These are cooperating-process, trusted-directory rules, not protection against arbitrary namespace edits or path aliases.
Cancellation is checked while hashing and comparing prefixes, before publication or cleanup. Once the final validation succeeds, replay and cleanup complete without cancellation. Selected database bytes and ownership evidence remain held while owned copies, receipt and intent are removed. Repeated recovery can resume after interrupted publication, decisions and cleanup. A kill before the complete Prepared header or during fresh ownership acquisition can leave insufficient authority: such evidence is retained and refused even if complete database copies survive. The protocol does not claim automatic recovery across every bootstrap interruption.
Existing Windows destinations use managed File.Replace metadata semantics. Missing-destination restoration preserves validated database bytes under private maintenance-file creation permissions; there is no separate ACL, alternate-stream or timestamp recovery record. The net11.0 library asset on Windows checks directory acknowledgements after prepared evidence publication, selected destination flushing and ordered cleanup; standard/Linux/macOS have no adapter. These checks do not qualify physical power interruption or complete bootstrap/decision/cleanup storage ordering.
Path writers request FileShare.None. Path readers restrict sharing to FileShare.Read (or the caller’s stricter choice) and reject final journals, preparation journals, ownership receipts and replacement intents before parsing. File or directory evidence at these recovery names is refused. Caller-supplied file readers acquire a second read-only lease; conflicting supplied handles are refused. These lifetime sharing rules prevent a pre-existing reader from mixing cached pages with interrupted writes by a cooperating writer. Windows enforces sharing; Unix and arbitrary caller streams require cooperating access. A caller-supplied writer stream must exclude all other readers and writers throughout its lifetime.
Identity combines the full database path (case-normalized on Windows) with validation of all raw contents. It is not an operating-system file ID: byte-identical replacement at the same path is indistinguishable, and path aliases are not a recovery discovery mechanism. Preserve the original path, the database and its journal together; do not move, replace or independently edit them while recovery is pending. Checksums detect damage, not deliberate replacement by an attacker who can rewrite both journal data and checksums. Recovery assumes exclusive access to trusted local journal files.
Microsoft Access has no knowledge of this sidecar. After a crash, recover through AccessWriter before letting Access or another program open the file.
The first preparation acquires raw page digests; later preparations under the same exclusive store reuse tracked digests. Every preparation writes only 32 bytes per original page plus the 128-byte header: about 0.8% of a 4 KiB-page database, or 1.6% with 2 KiB pages. Original page bodies are written once and only for overwritten or removed original pages. Each physical write also adds an after-image record; original images and intents are flushed before their database mutation. Commit folds the image framing and ordered page digests without rereading unchanged payloads. Failed or untracked physical effects invalidate reuse. Grouping mutations amortizes manifest serialization and final digest folding. Live rollback retains only offset metadata alongside the journal and restores pages through the physical store so page locks and write-failure handling still apply. Ordinary records reserve space for recovery-start and terminal records within the journal size bound. Recovery can rescan intents for changed pages. These costs do not imply that total transaction I/O is proportional only to changed pages.
The process-crash protocol does not establish power-loss atomicity: net11.0-Windows publication/retirement fences now require checked directory acknowledgements, but no storage configuration has a physical qualification receipt; standard/Linux/macOS still lack a directory adapter. On Linux, a file fsync alone does not guarantee its directory entry; a separate directory flush is required (Linux fsync documentation). Windows file flushing pushes buffered data toward the device, but filesystem and storage ordering still require separate verification (Microsoft file flushing documentation).
Path-based database creation initializes a unique adjacent private staging file, disposes its initialization writer, flushes the completed file to disk, and publishes it with a non-overwriting move. Failures before publication clean up only owned staging artifacts. The net11.0 library asset on Windows also fences the published name. A later fence or open failure retains the completed file. A process crash before publication may leave an unused staging file. Encryption replacement uses the distinct generation-bound intent protocol below. These namespace operations still have the power-loss boundary described above. Non-file streams retain in-process undo; no explicit persistent journal-stream API is provided.
FileRollbackTests verifies that file spills allocate no duplicate page undo log and that live rollback restores original ciphertext, preserves untouched pages, removes torn appended bytes and recovers untouched truncated tails. SelectiveRollbackJournalTests checks missing, late and incomplete original images, checksum-valid original-image corruption, untouched-page conflicts, truncation coverage and partial-page restoration. PersistentRollbackJournalTests proves compact preparation and one original image per changed page, and exercises torn writes, partial final records, multiple page versions mixed with partial undo, interrupted recovery replay followed by repeated reopen, authorized truncation and zero-filled tail restoration, corrupt headers/manifests/records, checksum-valid wrong paths and offsets, invalid lengths, stale commit decisions, incomplete final manifests, abandoned staging and Windows path casing. TransactionCrashRecoveryTests kills a child process without disposal during statement spills and final commit writes, then compares recovered files byte for byte. It also kills after successful commit and compares against an independently committed image. ShrinkCrashRecoveryTests kills during secure tail erasure, after truncation and after successful shrink, comparing plain and encrypted databases with exact original or committed images. ReplacementCrashRecoveryTests kills real child processes at preparation, publication, committed-decision and cleanup boundaries, then repeatedly interrupts recovery and verifies exact selected old/new bytes and retained evidence. CreationCrashRecoveryTests kills during partial initialization, after the staging file is flushed and after publication, checking absent or byte-exact complete destinations for plain and encrypted creation and preserving competing files. RecoveryLockFileTests retains a completed recovery journal while lockfile cleanup is refused, then verifies exact-byte recovery on retry; invalid evidence preserves the lockfile. Completed-journal disposal regressions verify that cleanup cannot delete a successor journal. These checks cover process termination, not power interruption or Access-native recovery.
RecoveryReplayCrashTests interrupts actual child processes at raw replay,
partial-page restoration, truncated-tail re-extension, database flush, terminal
decision and cleanup boundaries across plain Jet3/Jet4/ACE and four native
encrypted fixtures. Repeated interruptions and repeated opens must preserve
exact original or committed raw bytes. Conflicting content and live lockfile
holders retain their evidence. Generation-bound receipts retain ownership evidence across repeated deaths of writers opened after recovery and journal cleanup. Exact original bytes and successful repeated reopen are checked across plain Jet3/Jet4/ACE, native Jet3/Jet4 RC4 and ACE Standard/Agile encryption.
ReplacementNamespaceModelTests selects surviving names from real captured replacement states. ReplacementIntentTests verifies deterministic discovery, missing-destination recovery, old/new decision selection, bounded parsing and cancellation, retained live leases, unchanged-evidence refusal and repeated cleanup. Legacy copies without an intent remain unselected. The missing-destination model includes the documented ReplaceFileW error 1176 with a null backup path; complete recorded generations now authorize process recovery through writer open.
Actual transaction journals exercise torn writes, bad manifests, conflicting bytes and interrupted cleanup. A model losing the final journal name while retaining a complete .preparing copy requires exact-byte recovery through writer open and refusal by readers. Partial or conflicting preparations refuse without modifying surviving evidence. None of these models proves persistence when all bootstrap recovery names are lost.
A bounded managed probe on Windows 10.0.26300 ARM64, NTFS and .NET 10.0.13
established the acquisition/last-error behavior now used by DirectoryPersistence.
Focused adapter controls on Windows ARM64/.NET 10.0.14 also acknowledged writable
parent handles and refused a read-only control with error 5 even though the BCL
returned without an exception. The source-audited
flush helper
and handle acquisition
explain the immediate cleared/captured last-error check and unnamed backup option.
These are API acknowledgement and ordered-protocol controls, not physical
power-cut qualification. The adapter exists only in the net11.0 library asset on Windows;
unsupported assets/platforms return no acknowledgement and emit no acknowledged
fence hook. Namespace loss, device/cache ordering and complete C# refinement
remain independent E4/F6/I15/V7 obligations.