The executable: sections, functions, vtables, libraries

The layout of ShadowOfWar.exe: where code, vtables and globals live, how many functions there really are and why the exception table is not the list of them, vtables with and without run-time type information, the C runtime's initializer tables, the linked libraries, how the build's whole-program optimisation shows in the code, the engine's allocators and the most-called runtime primitives.

ShadowOfWar.exe is a 64-bit image of 117,732,784 bytes. It has no relocation table (IMAGE_FILE_RELOCS_STRIPPED, no DYNAMIC_BASE), so it always loads at 0x140000000 and every code and static address is the same in every run; only heap addresses move. Control Flow Guard is off (load configuration of 148 bytes, no guard flags), so the image carries no compiler-made list of address-taken functions.

Sections

The section names do not describe their contents. Executability comes from each section's characteristics, not from its name:

Section RVA range Access Contents
.arch 0x0001000 - 0x1D87C00 read, execute the engine's code (about 30 MB)
.sdata 0x1D88000 - 0x1D91400 read, execute more code
.xtext 0x1D92000 - 0x25AC000 read read-only data: the vtables, strings, the GameDB schema metadata
.idata 0x25AC000 - 0x2AA5000 read, write the writable globals: singletons, managers, static objects, startup tables
.pdata 0x2AA5000 - 0x2C85000 read the exception directory: 163,575 unwind entries
.bss a 0x2C89000 - 0x729714D read, write, execute 70 MB of code (below)
.text 0x7299000 - 0x7299014 read, write 20 bytes, not executable

Several further small sections hold the import and export directories, resources and other data.

.bss a holds engine methods that are reached only through vtables: of 2,134 vtables with recovered starts, 1,847 have at least one method there, and 12,832 of 71,422 vtable slots point into it. None of those methods is the target of a direct call (0 of 400 sampled; the static call graph has 4,371 call sources in the section and no call targets), so a function there with no callers is not evidence of a wrong address. The section also holds a large amount of code that is not part of the engine's call graph: inside it, only data references (vtable slots, pointer tables) identify engine functions, and jump targets do not.

Functions

The exception directory is not the list of functions:

The function list is therefore 243,839 functions: 137,282 with their own unwind entry and the rest leaves found as call targets, callbacks whose address is only ever loaded (lea), and methods referenced from vtables and pointer tables. 99.1% of the leaves start the way real functions start (16-byte aligned, or after int3 / nop padding, or directly after another function's body), against 97.0% for functions with an unwind entry.

The unwind data is complete enough to walk any thread's stack frame by frame from its registers, which is how the frame timings in Threads and the frame were measured. A return address found by scanning a stack is not proof of a live frame: locals keep return addresses of earlier, deeper calls.

Startup tables

33,354 qword slots in the writable globals are zero in the file and hold code pointers once the game has started (21,343 distinct targets): registration tables that startup code fills with lea rdx, [function] followed by a call to a register function (only 1,044 slots are written by a direct mov). Across two launches the slots and their targets agreed with a Jaccard index of 0.9997 over 33,347 slots; the few differences are run-time state (a stored return address and a captured call stack), not tables.

The C runtime's initializer tables

The C runtime entry 0x8C3E5C runs _initterm_e(0x1DA8778, 0x1DA8798) (C initializers) and _initterm(0x1D93950, 0x1DA8760): 10,690 C++ global constructors, in link order. The first global each constructor writes increases along the table in 8,043 of 9,483 steps (84.8%): data is laid out per object file. Code is not: across 18,206 functions, the correlation between a function's address and the address of its private globals is 0.105 (Spearman), the signature of whole-program optimisation reordering code. Only local runs survive (functions bracketed by one source file's strings belong to that file in 451 of 454 cases).

649 of the constructors register developer-console entries. Each is the same 0x7C-byte body: thread-safe initialisation of the console registry, the static object 0x2A58BC0 (vt_0x1EC6488, a critical section and a sorted entry list, guard 0x2A58BB0, built by 0x816CF4), Register(entry) (registry slot 12), then atexit(unregister). The registry's Execute is 0x15D0338, Find 0x7CBBA4 and Register 0x77F54C.

Vtables and classes

Vtables are read-only data in .xtext. 1,546 classes carry MSVC run-time type information (a complete-object locator before the vtable). About 4,100 more do not: 5,007 runs of two or more function pointers in .xtext are loaded by a lea and have no type information; for 4,097 of them (81.8%) the constructor pattern holds, the run's address being stored at [object+0], against 1,399 of 1,746 (80.1%) for vtables with type information. CResourceBundle (0x1E6E678, 18 slots, constructors 0x3DB434 and 0x3DB6EC) is one of the classes without it.

Vtables without type information pack back to back with no separator, so a table's end is the next table's start, and a start needs evidence: a code reference, a data pointer or a run-time pointer to it. A string-guessed start inside another table cuts that table short: the NPC class vt_0x1FB3640 has at least 129 slots (its per-frame AI update is slot 119, and slots 121, 122 and 128 are in use). Multiple inheritance shows as tables side by side: an inventory item's primary vtable 0x1FDABF8 has one slot and its secondary vtable, stored at +0x18, is 0x1FDABD0 with five.

The C runtime's initializer table is not a vtable even though it is a long run of code pointers in .xtext.

Live census. Walking the allocators of a running game (Minas Ithil) and testing each allocation's first qword against every known vtable finds 2.51 million objects of 2,518 classes, about two thirds of them of classes without type information. The most instantiated class is vt_0x1DBC010 (176,706 objects at a stride of 0x38). Consecutive instances repeat at a fixed stride, which bounds the object's size from above: inventory items repeat every 0x118 bytes, a CResourceBundle (0x178 bytes) every 0x180 and an NPC (0x3180 bytes by its class descriptor) every 0x3188.

A class's runtime implementation of a GameDB record type can be recognised from what its methods read: a class whose methods read fields of one category (at least three reads, at least 60% of them) implements that record type. 195 classes are named this way, 184 of them without type information, among them the AI planner steps (see AI).

Whole-program optimisation

The build uses link-time code generation, and it shows in the calling convention. Values stay in volatile registers across calls to functions the compiler proved do not touch them: computing each function's written registers as a fixed point over the call graph, 61,578 of 133,050 callees preserve at least one volatile general-purpose register, and between 74,179 and 83,251 preserve a given xmm register (0x1C0B634 keeps the screen width in xmm1 across the screen-height accessor). Analysis that assumes every call clobbers rax, rcx, rdx, r8-r11 loses these values.

The linker also folds identical functions. Many script bindings share one implementation, so a function address claimed by many bindings is not an identity: the engine assert 0x819104 (3,226 call sites), the 5-byte operator delete thunk 0x8C4370 (4,839 callers) and the record-name getter 0x18BDB0 (2,203) each sit under many binding names. Binding stubs are packed back to back with no padding (a ret at 0x1B75A4B, the next stub's mov r11, rsp at 0x1B75A4C).

Libraries

Identified from library strings and source paths, the namespaces of type information, imports, known constants (CRC32, FNV, SHA, MD5, AES tables) and contiguity (libraries link as contiguous object files; a gap bracketed by one library belongs to it, with 622 of 626 held-out anchors confirmed), 13,886 functions belong to linked libraries:

Library Functions
Havok (physics, AI navigation, cloth) 8,090
MSVC standard library 2,793
OpenSSL 1,897
Hydra (the publisher's online-services client) 619
Steamworks 327
HavokScript 61
Bink 45
Wwise 34
zlib 17

Exports. The executable exports 286 named symbols with their full C++ signatures: 263 belong to Wwise (the AK::SoundEngine API and its Query, MemoryMgr, StreamMgr, DynamicSequence, Monitor, MotionEngine, MusicEngine and DynamicDialogue namespaces, and globals such as g_pRegistryMgr and g_pRTPCMgr; see Audio and video), the rest are Scaleform's system and matrix functions, two memory pool ids and an assert hook. With the run-time type information they are the binary's only exact symbol names.

The online layer also links the Apache Avro C library (avro_record_set, src\Core\avFloatType.cpp). Small routines were identified by running them in an emulator against reference implementations: 37 functions behave as strcpy, 13 as memcpy, 5 as strlen, plus CRC32, FNV-1a and the engine's string hash.

StringHash 0x423E70 (the P919 name hash, h = h * 919 + LUT[c] with the table at 0x1DB50D0) takes its seed in r8 and never clears its accumulator, so callers chain hashes; a caller that leaves a value in r8 gets a different hash.

Memory

The engine's allocators

The engine allocates through Doug Lea's malloc 2.8.x, built with mspaces, two copies linked in: one with its parameters at 0x277CDD8 and its global state at 0x277CE10, the other with its parameters at 0x277D1E8 (create_mspace 0x1176058, sys_alloc 0x11744F8; segments from VirtualAlloc; malloc_state +0x358 footprint, +0x370 flags, +0x3A0 the segment list {base, size, next, flags}; chunk head = size | 1 previous-in-use | 2 in-use, user memory at chunk + 16, segments closed by a 0xB fencepost). In one running game a single state of the second copy owned 123 segments summing exactly to its 2,695 MB footprint, 78% of the process's heap memory: 1,086,315 chunks, 2,663 MB in use.

Primitive Callers What it is
GetDefaultHeap 0x602D20 690 (+3,033 through the thunk 0x114CC98) returns [0x26FEF88]; the first use builds the default heap in the static object 0x277CB40 ("Failed to create default memory heap!"); callers pass (heap, pointer) to a heap-type dispatch 0x1B08C4
ArenaAlloc 0x22A040 / ArenaFree 0x22A4E4 406 / 514 the arena table [0x2A86798], entries of 0x78 bytes: critical section +0x30, base +0x58, kind +0x60, used +0x64, size +0x68, linked arena +0x6C, owner thread +0x70 (a call from the owner thread goes to the linked arena); 16 arenas in use in a running game, two of 20 MB
BumpAlloc 0x24A4BC 363 a linear arena: align up, assert on overflow, advance +0x08
RefCountRelease 0xC51540 / 0x2158A4 546 / 370 lock xadd [obj+8], -1; the last reference calls vtable slot 0 (the deleting destructor) with flag 1
OperatorDeleteSized 0x8C4370 4,839 every scalar deleting destructor calls it as (this, sizeof)

Havok keeps its own heap inside one 512 MB engine allocation. Its root [base+0x2A86808] is an hkFreeListMemorySystem; +0x18 is the hkFreeListAllocator, whose +0x48 is an hkLargeBlockAllocator (dlmalloc-shaped: top +0x28, top size +0x30) with an embedded FixedMemoryBlockServer at +0x138 = {start, end, limit, break, ..., page list +0x40}: an sbrk over the fixed buffer. Small allocations come from 16 hkFreeLists (0x70 bytes each, inline in the allocator; size classes 8 to 640 bytes, 64 KB blocks) and per-thread caches (hkThreadMemory, 16 rows {head, u32 count} from +0x28, row sizes at +0x12C). In one region Havok used 121.5 MB of the 512: 1,864 large allocations (107.2 MB) and 106,677 free-list elements (12.8 MB); 389.7 MB past the break had never been handed out. The Havok world itself is reached through [base+0x2A86760]+0x20 (see Physics).

What the memory holds

Resources keep the engine's own byte accounting at res+0x38 (28,968 resources declaring 2,405 MB); for most resource types that figure covers GPU memory, not a CPU block. The largest single owner of CPU memory is the animation object vt_0x1E6F528 (constructor 0x57E4F4, destructor 0xC3A5A0; its resource [[obj+8]]+0x28 names an .anix file), whose buffers at +0x68 held about 790 MB; each object's owned memory matched its resource's +0x38 within 1 KB for 88.8% of objects (4.6% for a mismatched control).

Allocations that begin with a file signature identify themselves: 70 Wwise sound banks (BKHD, 216 MB), 27 behaviour graphs (15 MB), 18 Scaleform movies (10 MB), the GameDB blob (GameDatabase, 10 MB), 2,474 DXBC shaders, 2,024 effect databases (FXDB) and 9 Wwise packages (AKPK). The localization strings live in one 7.44 MB allocation built at load from the string tables: ascending rows {u32 P919(key), u32 character offset} (68,559 rows) followed by UTF-16 text (64,175 strings).

Direct3D resources are created through a few engine functions: 0x1B9D10 creates texture surfaces (ID3D11Device::CreateTexture2D, "Surface size mismatch!"), 0x1E71F0 creates textures, 0x39F930 creates constant buffers (CreateBuffer with bind flags 4; 89,080 allocations in a 150 s window) and 0x39F814 is the dynamic constant-buffer factory (usage dynamic, CPU write, "Failed to create constant buffer!").

Runtime primitives

The most-called functions without a library name:

Function Callers What it is
CRT::atexit 0x8C4608 3,580 _onexit(f) (0x8C45B8): every static's destructor registration
CRT::__chkstk 0xB39470 1,983 stack probe, one touch per 4 KB page down to the request
CRT::__security_check_cookie 0x8C3D50 1,520 the /GS cookie check; failure goes to 0x8C4690
ScriptArgsBegin 0x500930 1,574 called by every script binding: {+0 argc = (L.top - L.base) / 16, +4 argc - expected + returns, +8 lua_State}; "Incorrect number of arguments to function '%s.%s'"
ScriptArgsEnd 0x407F68 1,862 after every binding body: sets L.top (+0x48) to base (+0x50) + n * 16, filling with nil
lua_remove 0x565800 490 shifts the 16-byte values down one slot
PropRegister 0xC6DA00 18,963 the most-called function: registers one schema property (slot, name, type code, &default, ...) and hashes its name; GameDB and effect schemas both use it (see GameDB)
EngineAssert 0x819104 3,226 call sites the engine assert: a write to a null address at 0x819113 with the message in rcx
CompiledOutLog 0x19C83AC 630 spills r8 / r9 and returns: a varargs logger compiled out of the shipping build
NullSub 0x120405C 670 ret 0
Havok array growth 0x13F79C 903 the shape of hkArrayUtil::_reserveMore: array {data, u32 size, u32 capacity} with bit 31 of the capacity marking storage the array does not own and 0x3FFFFFFF masking the count; grows to twice the size (at least 1) through the allocator's buffer reallocation (vtable +0x28), or for storage it does not own its buffer allocation (+0x18) plus a copy
GetStaticLock 0x816D68 355 returns the critical section at 0x2A58C40, created once under a thread-safe static guard (0x2A58C38) with an atexit delete
IsNullOrEmptyStr 0x188C80 / StrNotEmpty 0x185FE0 767 / 873 C-string tests

Because the logger is compiled out, the shipping build is silent about most failures: a refused request returns without a message, and only the code (or running the function in an emulator against live memory) shows why.

Open questions

Shadow of War (Steam), game version 1.21. An unofficial fan research project, not affiliated with or endorsed by Warner Bros. Games, Monolith Productions or Middle-earth Enterprises. Middle-earth: Shadow of War belongs to its owners.