OpenMcdf: Uncatchable infinite loop in DirectoryTree.TryGetDirectoryEntry on crafted CFB directory cycle

QFinch assessment

The BST name-lookup loop in DirectoryTree.TryGetDirectoryEntry (OpenMcdf/DirectoryTree.cs:35-46) walks directory entries by repeatedly calling directories.TryGetSibling(child, siblingType, validateColor). A crafted CFB file with cyclic Left/Right sibling links among directory entries - constructed so the per-step BST-order check in TryGetSibling (DirectoryEntries.cs:84-85) is satisfied at every step - drives this while (child is not null) loop forever. There is no cycle detection in TryGetDirectoryEntry.

Potential impact

A denial of service affecting any application that opens untrusted CFB files with OpenMcdf. A small crafted input with a cyclic directory tree reaches the unprotected BST name-lookup in DirectoryTree.TryGetDirectoryEntry, hit by any caller of OpenStorage / TryOpenStorage / OpenStream / TryOpenStream - including the very common pattern of iterating EnumerateEntries() and calling OpenStream(entry.Name) per Stream entry. The cycles bypass the per-step BST-order check in TryGetSibling, so no exception is thrown and try/catch cannot protect callers. The affected thread is unrecoverable without killing the process. Downstream CFB consumers (e.g. .msg-file parsers) inherit transitively.

Technical context

The recent Brent's-algorithm commit (24f445a) protects DirectoryTreeEnumerator and works correctly for both attached repros - pure EnumerateEntries() throws FileFormatException: Directory tree contains a loop cleanly. The unprotected code path is the lookup-by-name loop, which is reached from multiple public APIs: - RootStorage.OpenStorage(name) / TryOpenStorage(name) - RootStorage.OpenStream(name) / TryOpenStream(name) The second one matters most: typical consumers iterate EnumerateEntries() and call OpenStream(entry.Name) per Stream entry. With Brent's algorithm catching the enumeration cycle but not the per-entry lookup, callers can still hang as soon as they touch a streamed entry.