World: regions, layers, placements and world objects

How a region's world is assembled from streamed parts, what a world part holds (the object registry and its fourteen blocks, static placements, prefabs, terrain and vegetation), how placed objects look in memory, how the engine creates, binds and moves them, and the world data behind traversal (context-action lines), weather-gated effects and collision.

A region's world is a set of world parts, each streamed in as a section and turned into a world context full of placed objects. This chapter describes the parts on disk, the streamer that loads them, the objects they create and the world data that the rest of the engine queries.

Regions and world parts

The game has eight regions. Each has one main part (worlds\retail\<folder>\<folder>, the only part with terrain) and a set of layers under worlds\retail\<folder>\sectionsandlayers\: story and side content, fortress pieces, lighting and its per-Overlord variants. 1,303 layers ship (each identified by the paths its own container lists, never by its archive entry name):

Seven regions have a region archive and a lighting archive (<folder>.arch06, <folder>_lighting.arch06), and the patch archives add and replace parts. Lithlad has neither: its main part and all 221 of its layers are in the patch archives (the containers in Patch_DLC4, with standalone .objres and .scripts copies in later patches).

A world part is an EMBB container (see Archives and file formats). Every layer carries its own prefab (.inst), and 1,302 of 1,303 carry the same eight further parts, named after the layer: .objres (the object registry below), .terrain, .hkai (navigation mesh), .wldclnt (static placements), .veggrid, .scripts (its script; see Scripting), .uis and .uiwd; then the prefabs, meshes, materials, textures and effects it uses (176 layers also carry a world mesh, .wldmsh). A layer's .terrain is an empty 12-byte section; only main parts carry terrain.

Across all 1,311 parts (1,303 layers and 8 main parts) the object registries place 1,247,824 objects of 156 classes. The most common: AINodeMenacePlace 113,874, AINavMeshObstacle 67,714, SpecialFX 67,460, DynamicObjectGroupState 67,361, ScriptRelay 56,290, StaticLightOctreeBias 54,700, FireVolume 50,718, AISpawnerAmbientNormal 49,769, LightCube 47,771, Group 43,956, VolumeController 35,992.

Patched registries win. The engine does not load the registry embedded in a part when a patch archive ships a standalone .objres for that part (2,753 standalone registries ship). Lithlad's main registry in a running game is the copy in Patch_DLC5 (161,821 context-action lines), not the one embedded in the Patch_DLC4 container (161,828) or the copies in Patch_09 / Patch_10 (161,836). Layer scripts follow the same rule (see Scripting).

How a region's world is assembled

The section streamer

WorldBundleMgr (GameSystems\Core\WorldBundleMgr.cpp) is [GM+0x6D90]. It keeps a registry vector of world bundle groups at +0x00 / +0x08 (145 groups in Lithlad, each pointing back to the manager at +0x240). The scene tick walks the groups (see Threads and the frame) and services only those whose want-load byte +0x252 is set.

A group (0x258 bytes, constructor 0x1D297EC) is the head of a circular list of entries: +0x00 the first entry, +0x08 the last, closing back on the group in both directions. +0x251 set disables the duplicate check, +0x252 is the want-load gate. An entry (0x168 bytes, vt_0x1DB6558) is one requested section:

AddSection 0x4B41CC(group, path, &id) calls 0x4B420C, which rejects an empty path, appends .bndl, returns the existing entry when the path is already requested in that group (0x4B1FE4) and otherwise links a new entry at the front of the list. RemoveSection 0x4B1BCC(group, id) releases every entry with that id (0x4B5C34). The entry exists as soon as it is requested, long before its content has streamed. Requests are de-duplicated per group only, so the same section requested through two groups is loaded twice.

A plain bundle load (see Resource streaming) makes a part's resources resident without placing anything; a section requested through a group is what creates its world context and objects.

What a region loads

MainScene ([base+0x26FFAB0]) owns eight group slots. In Lithlad during play they held:

The lighting layer is requested by the weather manager in its own group ([[GM+0x6D98]+0x20]): the current weather pattern's Section field names it (see Rendering, "Weather"). In Lithlad the pattern was DS_OL_Outlaw_Clear, whose Section is worlds\retail\desert\sectionsandlayers\ol_outlaw.bndl, and that group held exactly that section. The ol_* layers are per-Overlord variants: Lithlad ships nine (ol_dark, ol_feral, ol_machine, ol_marauder, ol_mystic, ol_outlaw, ol_slaughter, ol_terror, ol_warmonger) beside its plain night01..03 layers; Nurnen, Cirith Ungol and Seregost nine each (Nurnen's and Cirith Ungol's include a shared ol_share01); Gorgoroth 27, each also in _bd and _bduc forms. Lithlad's ol_outlaw holds what a lighting layer holds (1,104 SpecialFX, 959 LightCube, 612 Sound, 175 WorldModel ... and an AtmosphereVolume), much like its night03 (1,492 SpecialFX, 959 LightCube, 612 Sound, 32 AtmosphereVolumes).

Of the 68 groups with entries, 63 hold only the menus' 3D scenes: 86 sections under prefabs\ui\mordormap\ (the world map's region pieces such as minasmorgul.bndl, and each fortress's keep and gate pieces by tribe, fortworldsections\keeps\blackguard\bgrdflairped05.bndl), 12 under prefabs\ui\store\, 6 battle-report scenes and 8 region-faction scenes (wuim_*_ws.bndl). 127 sections were loaded in all, 13 of them from worlds\.

Conditional sections. At load MainScene+0x760 .. +0x768 receives 16-byte pairs {section record, u32 id} copied from the region's WorldInfo.ConditionalWorldSections. The evaluator 0x1DFAC8 (called from 0x1DF500) reads each section's _StructuresWorldInfoConditionalSection.GameDataVariable: true with id 0 requests the section through [MainScene+0x758] (0x4B41CC); false with a nonzero id releases it (0x4B1BCC). Lithlad has none; Minas Ithil during the Shadow Wars has 12 (its pre-fall sections released, the post-fall ones loaded).

Region travel. The script call System:LoadWorld(DBWorldInfo.<world>) on the main thread starts a region load within 0.1-0.2 s (LoadWorldCustom, which no shipped script uses, shares its implementation 0x1780824). The current region record [[GM+0x6C68]+0x1B0] changes at the start of the load, before the destination's objects exist.

World contexts

The objects of a streamed part live in a world context (vt_0x1DB5D58). The object update manager's object sets (see Threads and the frame) are these contexts: 129 of 129 sets in Lithlad carry that vtable, and every placed object points back to its own at +0x58 (105,946 of 105,946).

A context's object-name table is [ctx+0x68]: 0x800 buckets of 0x18 bytes, bucket = hash & 0x7FF, list head at +0x08; the node is embedded in the object at +0x28 (+0x08 next, +0x10 hash, +0x18 name). 20 of the 129 contexts had a table, holding 105,946 objects; the largest is the main part's.

Matching each table against the shipped registries by object name:

Of the main part's names, 4,659 are not in the live table, among them every StartupCommand (*.SC) and some MapCanvas objects (*.MCNV_H); 317 names exist only live (Avian0 .. Avian11, CTP_D_103.CTP ...). The position differences are AI placements moved vertically: 20,211 of the 20,306 moved objects in the main part are AI nodes, ambient spawners and combat teleport points, and in every one of those classes the median horizontal change is 0.0 cm while the median vertical change is -47 to -285 cm (AINodeMenacePlace 6,802 of 6,869 moved, median -181 cm; AISpawnerAmbientNormal 4,343 of 4,920, -150 cm; AINodeRetreat 432 of 432, -285 cm; CombatTeleportPoint 751 of 849, -47 cm). The other 95 moved objects belong to other classes.

The object registry (.objres)

A part's .objres is an OBJR chunk registry:

'OBJR' | u32 13 | u32 hash | u32 block count
block count x { u32 name hash | u32 size | payload }

The chain ends exactly at the end of the section in 1,310 of 1,310 parts. Parts with context-action data carry 14 blocks (724 parts), the others 12 (no CAWR / CADB). Each block hash is StringHash of the name its handler registers with the resource manager; the names sit together in the image at 0x1E6B4E8-0x1E6B6B0, 14 of 14:

Across the 1,310 registries the blocks total 736 MB: ObjRes.WorldPhysics 416 MB, CADB 123 MB, OBJL 84 MB, VOLM 40 MB, GADB 23 MB, PFDG 18 MB, ONLS 14 MB, Clutter 11 MB, SCRB 3.1 MB, NLFE 2.2 MB; the rest are under 0.4 MB. Atmosphere volumes and their VOLM brushes are described in Rendering; the context-action blocks and SWFX have their own sections below.

The object list (OBJL)

'OBJL' | u32 5 | u32 sets | u32 N | u32 class bytes | u32 name bytes | u32 group bytes
0x1C   class names, object names, group names (NUL-terminated string tables)
then   per set: u32 set hash | u32 count | count x 50-byte record (unaligned)

The table ends exactly at the block end and every offset lands on a string start in 1,310 of 1,310 registries (re-read on five layers and Lithlad's main part, 104,489 objects; the 9,284 layer objects all have unit quaternions).

The loader (vt_0x22C0338 vf3, 0x502B8C) expands each 50-byte record into an 0x50-byte run-time record: +0x08 the name, +0x10 the group offset, +0x14 StringHash(name), +0x18 the property record index, +0x1C the position, +0x28 the quaternion, +0x38 the word at file +0x2A, +0x40 the class name. It builds, per object set, a table of {name hash, u16 index} pairs used to find an object by name. It reads the id at file +0x2E into a local and discards it: nothing at run time keys on it. The id is the authoring identifier of the object's template. Every copy of an object placed through the same prefab carries the same id (one AINodeMenacePlace prefab node: 9,275 copies over 74 parts share 0x2651E0E7; one hanging-light trigger: 8,749 copies over 109 parts), while objects authored directly in a part have their own: over the whole game, 1,247,824 objects carry 167,063 distinct ids, 151,160 of them used by a single object. The id never equals the hash of the object's name (0 of 1,247,824) and stays within one class (two exceptions: 0 on every WorldInitializer and WorldCollision object, and one pair of unrelated objects, as expected from chance collisions among 167,063 random 32-bit values). Most lists hold one set, the world set (hash 0x3BDE767D). Object names carry their prefab instance and their role: Barre_46.DOGS_Fire, MesaE_08.VC_NoHang, HideS_138.FP, pGCarags.spwCaptain. In all ten parts read, the list opens with a WorldInitializer object and then WorldCollision objects at the origin: PhysicsBSP (and LargeAIIgnore in some layers); Lithlad's main part has four (PhysicsBSP, PlayerBlocker, CameraBlocker, LargeAIIgnore).

The object property database (GADB)

ObjRes.ObjProps is a complete GameDB database (version 17) whose root type is ObjectPropertyDB, with the category ObjectDB and records named ObjectRec (see GameDB). An object's OBJL record index selects its record, and every property the object reads at creation comes from it. Records are de-duplicated: objects with identical properties share one (Minas Ithil night01: 3,160 objects, 1,886 records; Lithlad main: 95,205 objects, 21,430 records). In a running game an object keeps its record at +0x50.

Group members (ONLS) and script bindings (SCRB)

Both key objects by StringHash(object name).

• ONLS lists the members of every grouping object: 'ONLS' | u32 1 | u32 area bytes | u32 index bytes; the area holds groups {u32 n, n x {u16 length, name}}, the index {u32 hash(owner), u32 offset of its group} sorted by hash. Over 400 layers 6,360 of 6,360 owners are objects of the part (Group 5,515, AINodeRoute 735, AINodeSearch 83, static-light probes) and 107,097 of 107,097 member names exist in OBJL. A prefab instance's children (Barre_00.DOGS_Alive, .VC, .NMO, .FX_P ...) form one group.
• SCRB says which function of the part's script runs for which object and event: 'SCRB' | u32 1 | u32 string bytes | u32 bindings | u32 objects, then the strings, bindings {u32 hash(event), u32 event name, u32 hash(script resource path), u32 function name} and an object index {u32 hash(object), u32 first binding, u32 count} sorted by hash. Over 400 layers 18,129 of 18,129 scripted objects are named (ScriptRelay 8,906, ScriptObject 4,700, StartupCommand 2,795, QuestRelay 438, RemoteScript 365 ...), every script hash is the hash of the part's own worlds\retail\...\<layer>.scripts path (5,844 of 5,844), and every bound name exists in that part's shipped script. Events include Script, Script1, Start/Cleanup/Fail/CompleteQuestScript, PreActivateQuestScript, Departure/ArrivalScript, OnInitialize and OnEnterScript. A binding names either a function name = function(self, threadId, scriptId) ... end (4,938 of 6,312) or, for a ScriptObject's Script, a script object name = hmake <Type>_meta { ... } (1,374).

What a dynamic object draws (ObjRes.WorldPhysics)

The static placements (below) list only static prefabs. Everything the engine can burn, break, open or animate is an OBJL object, and its geometry and collision are found through this block:

u32 n_objects | u32 n_records
n_objects x u32 hash(object name)   (unsorted)
n_objects x u16 record index
records:  {u32 1, u16 0, f32[6]}                                   no path: world collision
          {u32 flag, u16 length, path, u32 key, f32 cull, f32 half[3], f32 centre[3]}   a prefab (.inst)
then the records' Havok shapes (tagfile TAG0, SDKV20160200, hknpCompoundShape)

Lithlad's main part lists 18,943 objects over 585 records. In Cirith Ungol's main part 12,638 of 12,639 hashes are OBJL object names, all of geometry or physics classes (FireProp 2,491 of 2,491, DynamicObjectGroupBrush 1,382 of 1,382, WorldModel 590 of 590, AINavMeshObstacle 6,397, the world-collision objects). A record's bounds are its prefab's decoded bounds (the orc furnace base: half-extents 18.1 x 8.9 x 22.1 m), and records are per (prefab, class), so a prop and its navigation obstacle use different records of the same prefab.

Motion paths, effects, ambient life and clutter

• KEYF (CClientKeyframer): 'KEYF' | u32 1 | u32 n | u32 data bytes, n x {u32 id, u32 offset}; an entry is {u32 5, u32 keys, u32 frames, u32 frame bytes}, keys {u32 hash(name), u32 frame} (Open, Closed, Start, End), frames of 40 bytes {f32 time (s), f32 offset[3], f32 quat[4], u32 0xFFFFFFFF, u32 tag}. The id is the KeyDataIndex property of a KeyFramer object (449 of 449 resolve). 45 of 1,310 blocks carry paths (431 paths; quaternions unit 4,848 of 4,848, times monotonic in 431 of 431); a gate leaf slides 400 cm over its 10 s TotalPathTime.
• SFXS, static effects that are not objects (falling leaves, furnace smoke): header 1 | off_a | off_b | n | n2 | 0 | 0, .fxp paths, at off_a n x {f32 position[3], f32 cull range}, at off_b n x {f32 quat[4], u32 .fxp path offset, u32 3, 0, 0}. The loader (vt_0x2301300 vf3, 0x4BB9E4) refuses a block unless (off_b - off_a) / 16 = n and n2 <= n, squares every cull range at load, and keeps n at +0x20 and n2 at +0x24. n2 counts the distance-culled entries: the file puts every entry with a positive cull range first and the entries with range 0 after them. In all 60 non-empty blocks (5,737 entries) no entry before n2 has range 0 and none after it has a positive range; the game has exactly two range-0 entries, Minas Ithil's two distant waterfalls (mi_waterfallfar01_base and _top, n 1,273, n2 1,271).
• NLFE, ambient-life emitters: 'NLFE' | 1 | 0x1C | n | group-name bytes | name bytes | groups; the group names are NativeLife/Definition records (con_TourchLight, con_MeatFlies), then n group indices, n instance names (light_05L.Moths; prefixed by their prefab instance, not OBJL objects), n zeros and n positions.
• Clutter (CClutterMgr, ClientShellDLL\ClutterMgr.cpp): instanced static meshes carried with their geometry. Header u32 1 | u32 string bytes | u16 meshes | u16 instances | u16 elements | u16 vertex buffers | u32 buffer bytes; .mat strings; per mesh {u32 material, u16 elements, u16 vertex buffers, u16 vertices, u16 indices, u16 users} with its vertex elements, 56-byte vertices {pos[3], normal[3], uv0[2], uv1[2], tangent[3], u32 colour} and u16 triangle indices; then 40-byte instances {f32 pos[3], f32 quat[4], f32 scale, u16 left, u16 right, u16 mesh, u16 1} forming a binary tree with one root per block. 51 blocks are non-empty (275 meshes, 9,630 instances); Lithlad's main part has 23 meshes and 1,657 instances.
• PFDG, the prefab tree of the debug viewer (registered by CPrefabDebugViewer 0x1796274, which draws a cube CPrefabDebugViewer.Box scaled by each node's extent): 'PFDG' | 1 | n | string bytes, n x {f32 centre[3], f32 extent[3]} (the extent is signed: 21% have a negative component), n packed 21-byte nodes {u32 parent, u32 first child (0xFFFFFFFF none), u8 r, g, b, a, u32 path, u32 name, u8 flag}, strings. 381,195 nodes, every link valid. The viewer is created and its handler registered unconditionally at client start-up (CGameClientShell::InitializeSubsystems 0x1772E1C), so the block is parsed on every load; the handler's finalize step (vf5 0x483F78) hands each tree, keyed by StringHash("<part>.objres"), to the viewer (0x483FDC) and to nothing else. No other code names the resource type.

Static geometry

Placements (WDSC, .wldclnt)

The whole section is read by one handler (vt_0x230A800 vf3 0x6B3510 -> 0x6B3914, then 0x6B3664), in this order; the walk ends exactly at the section end in 1,310 of 1,310 parts:

0x00  'WDSC' | u32 9
0x08  u32 CELLS | u32 total boxes | u32 INSTANCES | u32 total groups | u32 MESH ENTRIES
      | u32 total materials | u32 total polygons | u32 total polygon vertices | u32 STRING BYTES
0x2C  strings: the part's .wldmsh, its materials, every prefab .inst it places
      MESH ENTRIES x {u32 n, u32 mesh path, u32 value, n x u32 material path}   (1 when the part has
                                                                               its own world mesh)
      INSTANCES x 76 bytes (unaligned)
      CELLS x cell
      the trailing block

The header's totals size one allocation for everything the cells hold (Lithlad: 1 cell, 16 boxes, 2 groups, 6 polygons with 24 vertices). An instance:

The reader 0x1CCB150 turns each into a 0x60-byte run-time record: bounds at +0x00 / +0x10, the pivot at +0x40, the quaternion at +0x4C, the scale at +0x5C, the squared cull distance at +0x3C (a default when the file value is not positive), and at +0x38 the kind in bits 0-1 and "flags non-zero" in bit 2. The +0x40 word selects which LOD group of the prefab the instance draws: the reader passes StringHash(prefab path) and this tag to 0x58A928, which keeps, per prefab path, a small table of the tags placed with it (three inline, then blocks of 14) and returns the tag's index, stored at +0x3A. The tags are the ones the prefab's LOD records open with (see Prefabs): in Lithlad and Minas Ithil 18,937 of 18,937 instances whose prefab ships in the part's own container name one of their prefab's groups; 163 of them use a prefab with several groups and 45 pick one other than the first. Composite prefabs use the same tag on their child records. Lithlad's 12,877 instances use 15 distinct tags, one per prefab for 241 of 242 prefabs.

Lithlad's main part places 12,877 static instances of 242 prefabs (quaternions unit 12,877 of 12,877, scale 0.12-2.03, flags 1 on 12,789; cull distances mostly 60 m, 90 m, 350 m and 4 km). The kind is a property of the prefab (constant for 242 of 242): 0 on 11,659, 3 on 1,210 (cliff-base rocks, torches, fort stairs, doors, pickaxes) and 1 on 8 (a zipline). Layers place few or none (0 to 17 in five read). These are the static buildings, walls and rocks; prefabs that can change (destructible, animated, interactive) are objects instead.

Cells (read by 0x6B3CD0). Each main part has one cell; 351 layers have one, 65 two, 2 more, and 884 none:

u32 boxes | u32 polygons | u32 groups
boxes    x 32 bytes {f32 min[3], f32 max[3], u8, u8, u8, u8, u16, u16}
polygons x {u8 n, n x f32[3]}                     (four vertices in every main part)
groups   x {u32 key, u32 n, n x u32 instance index}

For each group the engine builds two lists of instance bounds, each entry the instance's bounds with its squared cull distance in the fourth component: one of every instance in the group, and one that leaves out the instances of kind 3. A cell's groups cover its instances: Lithlad's cell lists 12,714 instances under key 0 and 163 under 0xD7D4D586 (12,877 in all).

The trailing block (read by 0x6B3664) opens with 13 words, four of which must be 0: A count | first cell of a range | 0 | B count | ? | 0 | total s | total hull references | hull count | ? | 0 | total A-to-A references | 0 (the loader keeps a pointer to the cells from "first cell" on, and the totals size its allocation); then:

hulls x {f32, u32, u32 vertices, u32 n1, u32 n2, vertices x f32[3], n1 x u16, n2 x u16}
A     x {60 bytes, ending with u32 count c1, u32 count c2, i32 cell (-1 none), u8 count s;
         then s x 16 bytes, c1 x u32 hull references, c2 x u32 references to other A records}
B     x {u32 n, u32, f32, i32, i32, n x u32 references to A records}

The engine computes each hull's bounds from its vertices. Every main part has one or two A records and one B record; Barad-dur and the Black Gate have no hulls, the others 86 to 207. 1,217 of the 1,302 layers carry only the empty 52-byte header, and 85 carry records.

Prefabs (PRFB, .inst)

A prefab's baked geometry; static world geometry is never .mesh (prefab .mesh files are the destructible and animated pieces). 'PRFB' | u32 8 | u32 hash | ... with header words for draws [4], materials [5], child prefabs [6] and vertex formats [9], then:

• LOD records from 0x38, 44 bytes each with -1.0 at +28, in groups that open with a per-group tag (0xD238B9DD on a keep wall, 0xD238B9DE on human buildings, 0x3249 on lanterns, an extra leading group 0x159A on fortress pieces); a placement names the tag of the group it draws (see Placements): a root {tag, -1, 0, 0, next = first unit, f32 cull distance} and units {tag, name hash, first draw, draw count, next unit (-1 = finest), f32 switch distance, f32, -1.0, 0, 0, distinct draws}. Per instance the engine walks coarse to fine and draws the first unit whose next is -1 or whose switch distance the camera is beyond (a keep wall switches at 320, 150 and 80 m and is culled at 7 km). A unit may draw nothing at its range; draws in no unit are depth-prepass and shadow casters. Header word [7] is not the unit count (88 of 431 chained prefabs disagree).
• {u32 draws, u32 vertex bytes, u32 index bytes}, then per draw one vertex block ending at its vend (stream 0 then stream 1, stride-aligned), then u16 indices (idx_min .. idx_max, rebased by idx_min).
• The tail: vertex formats {u8 stream, u8 type, u8 usage, u8 index} (types 2 float3, 12 half4, 10 half2, 4 ubyte4; usages 0 position, 1 normal, 2 tangent, 3 binormal, 4 uv, 5 colour), 52-byte draw records {0, n, n, index start, base, triangles, material, 0, 0, format, vend, f32, f32}, then the material paths. Positions are prefab-local with the pivot at the base centre. A composite prefab ends with a block of placed child prefabs.

A layer carries the prefabs it places as PRFB sections of its own container (Minas Ithil's sabotagecages_00 carries guard_caragors.inst and three footprint-tracking prefabs).

World meshes, texture arrays, terrain and vegetation

• MESH version 5 (.wldmsh) is a part's own world-space mesh, laid out like the PRFB tables with the vertex buffer at 0x2C (header [3] draws, [4] formats, [9] vertex bytes, [10] index bytes); its materials index the part's WDSC list. Draws wider than 20 km are sky domes and vista cards (504 km in Minas Ithil's night01, 960 km for Cirith Ungol's mt_v01).
• In Cirith Ungol about 90% of the world's triangles use texture-array materials (.texa), which blend array slices by a mask and a height map.
• TERR version 9 (<region>.terrain), main parts only (Barad-dur has none), read by 0x7AC0F4: per terrain a root grid (4x4 or 8x8), position (the centre), quaternion, extent, height range, a terrain shader parameter (400; 500 in Lithlad, 1000 at the Black Gate: with Lithlad loaded, k_vTerrainHullParams0.y and k_vTerrainDomainParams1.w both hold 500, next to the height range 13,000 in k_vTerrainDomainParams1.y), the node count, a material and a global texture (a 32x32 flag map, one cell per depth-3 node), then nodes {u32 flags; unless flags & 0x20: f32 min, f32 max, u32 tile} and tiles of three texture paths; the file ends exactly after them in every region. Each root is a quadtree stored in pre-order (flag bits 0-3 = children present, 0x40 root, 0x20 no data); every root's tile 0 is the region overview, deeper nodes own a 68x68 16-bit height tile and two 132x132 BGRA maps (four texture-array slice indices and their weights). World placement: U = (x - pos.x) / size + 0.5, V = (z - pos.z) / size + 0.5, y = pos.y + h * range. The terrain shader disp_terrain.fxo keeps the four heaviest of up to 24 slices per point; slice index 255 means no layer, which marks the holes (outside the play area, and pockets such as forts and caves).
• VEGT (.vegtile), vegetation, read by 0x503A0C: 'VEGT' | u32 version (1-3) | f32 centre[3] | f32 half[3] | u32 species, then per species {u16 length, name, u32 n, n x 16-byte instance}. An instance is u16 qx, qy, qz (position = (centre - half) + q * step, step = max(half) * 2 / 65535), u8 scale (b / 127.5), a u8, four rotation bytes, a 3-byte bounding-sphere offset and u8 flags (bit 0 hidden). The rotation is decoded on the GPU (vegetation_sort_nomotion.fxo): cos yaw = 2 b0 / 255 - 1, sin yaw = 2 b1 / 255 - 1, a2 = b2 * 2pi / 255 - pi, a1 = b3 * 2pi / 255 - pi, world = s Rz(a2) Ry(yaw) Rx(a1) local + position. Species are Vegetation/Type records; 108 species and 1,291,234 instances ship, in tiles worlds\retail\<folder>_<n>v.vegtile (inside the region folder for Gorgoroth and Seregost) and in layer tiles. A tile's file ends exactly after its last species.

World objects in memory

Layout

Read on every object of the 20 named contexts in Lithlad (105,946 objects):

Classes. The runtime class entry is 0x38 bytes in a heap array: +0x00 the static class descriptor, +0x18 StringHash(class name), +0x20 the class name, +0x30 align << 32 | size (WorldInitializer: 0x800000100, alignment 8 and size 0x100). The static descriptor (in .idata) holds +0x00 the name, +0x08 the base class's descriptor, +0x10 flags, +0x18 the constructor, +0x20 the destructor, +0x2C the instance size and +0x30 the alignment. A shape-validated sweep of the image finds 433 descriptors with 433 distinct names (431 in .idata, 2 in .xtext; 121 constructors lie outside .arch). 320 classes descend from BaseClass, 256 of them from GameBase (0x120 bytes, constructor 0x7DCC78), and 64 from BrushBase. The image also lists the static descriptors in one place: the leaf function 0x19CBDFC stores 435 descriptor pointers, one after another, into the array 0x2A61A30 .. 0x2A627C0 (stride 8).

The most numerous live classes in Lithlad were AINavMeshObstacle 11,588, AINodeMenacePlace 7,249, FireVolume 7,189, DynamicObjectGroupState 6,251, ScriptRelay 5,993, AISpawnerAmbientNormal 5,002, Group 4,628, FireProp 4,129 and VolumeController 3,882.

Position. +0xAC is the logical position for world objects; prefabs are authored with their pivot at the base centre. Characters share the transform offsets (see Characters).

Creating, binding, moving and removing objects

All of this runs on the main thread.

The object server

The object server [base+0x26FF058] (vt_0x22C0B28) is the world-object factory:

The creator 0x1CD454 (also called by the part loader 0x11B4EC0 for every object a part streams in) requires an object set at params+0x1A0, allocates the object through the class holder at params+0x40, runs its ReadProps (vtable +0x60, which pulls every property from the ObjectRec), builds the transform from the parameters into +0xAC, registers the name, links the object into its context, builds its components (vtable +0x68) and activates it (+0x80, +0x70, +0x48). A part's loader creates every object first and post-creates each afterwards.

CObjectCreateParams (defaults 0x43DAA0):

Names

The name registrar 0x4DBF08 either copies the name (and sets +0xDE to 1) or keeps a pointer into the part's strings (+0xDE = 0), hashes it into the node at +0x28 and calls the context's vf10 0x4DBF78, which picks the bucket (hash & 0x7FF, 0x4DBF84) and appends the node at the tail (0x115ED3C, which asserts "Error: Adding a link that is already inside of a list" only when the node itself is already linked). Duplicate names are legal: nothing checks the name. The lookup 0x4DC078 compares the hash and then the name case-insensitively, walking from the tail, so the newest object of a name answers.

Render and collision are bound by name. The component build (vtable +0x68 -> 0x333430) asks the context's vf5 0x3334F0, which looks the registered name (+0x40) up in the context's resource table (ctx+0x98). An object created under a name its part does not list gets no render component and no physics; created under an existing object's name it receives that object's model and collision. Because the newest object answers a name, the members of a prefab created after the original members bind to each other during post-create.

Binding from script

A script's Create does not construct anything: the objects already exist in the part before any script runs, and scripts bind to them by name. The <Class>.Create(scriptId, name, b3, relative) stubs of 117 of 125 script classes share 0x5013A8 -> 0x501B64, which composes the name (a #. prefix or relative prepends the prefab instance's name) and calls the object server's vf14 with the scriptId's own context; the lookup is not given the class. A missing name or a class mismatch returns Class.None; ScriptObject.Create is a function of its own, 0x501CAC. A scriptId is {PrefabName, PropertyName, Instance}; Instance is a weak handle (prefab instance at payload +0x48) that becomes null when the prefab's part streams out.

Moving

GameBase:CopyXForm(A, B) 0x1823B70 is SetPosition(A, B+0xAC) 0x1454EC followed by SetRotation(A, B+0xB8) 0xEF690; render state and spatial structures follow the setters, and it works across contexts. Collision follows ownership: the setter path 0x274C3C forwards to the physics body at +0x98, so only pieces the object owns move. 3,382 of 3,491 FireProps own no pieces (their collision is baked into the part's static PhysicsBSP), so a moved bench or barrel stack keeps its old collision; DynamicObjectGroupBrushes (607 of 634) and cage brushes (vt_0x1FA3C00, 84 of 84) own theirs and stay solid where they go. A destructible prop is a prefab of sibling objects (brush, explosion, effects, nav obstacle, volume controller): moving one member leaves the others behind.

Removing

Objects live and die with their part's context. When the part streams again its objects are created again from the file, so a moved object is back at its authored transform and an object created into that context is gone. Nothing about placed objects is saved. A class's destructor is at +0x20 of its static descriptor.

Dynamic object groups

Cages, nests, barrels, gates and traps are dynamic object groups: a DynamicObjectGroup object (the root) with DynamicObjectGroupBrush members (the breakable or interactive geometry), DynamicObjectGroupState members named after their states (Barre_46.DOGS_Fire, WaspN_03.DOGS_Shoot) and the prefab's effects. The brushes bind the group's composite prefab (caragorcage, waspnest, ghulcloset: no draws of their own) and sit at fixed offsets from the root (35 of 35, 53 of 53 and 226 of 226 in those three); the drawn art is the prefab's children at the root's transform.

The data is in four GameDB categories:

A transition is a guarded rule: the attack categories and parent states it accepts, and the state tag it enters. The state tags carry no data; what a state shows is the part's DynamicObjectGroupState objects for it and the scripts bound to them.

At run time a group's current state is DOG+0x188 (the state object; its +0x180 is the StateType record), and DOG+0x190 holds its 32 state links. The group's post-create 0x4871190 collects the links but does not enter a state; a group streamed in with its part receives its initial state afterwards. DynamicObjectGroup:Script_SetState is 0x5ADF0C(dog, StateType record): when the ready flag +0x22C is set the request is queued at +0x160, otherwise it is applied at once (0x5AEABC). A brush's current state object is brush+0x148; while it is null, hits on the brush are routed nowhere. A brush's ReadProps does not fill the visibility flags (a FireProp's does), so a brush created without them is invisible.

Placed lights and sounds

Light and sound objects read their colours and radii from their property records at creation and keep them in the object:

In Lithlad the live values equal the decoded property records: LightColor, SpecularColor and IntensityScale 950 of 950, LightRadius 903 of 903, Sound OuterRadius 1,326 of 1,326 (each control read 0). 2,810 lights were matched by name; 45 sat more than 1 cm from their decoded position. 3 of those sit exactly where the patch .objres places them. The other 42 are lantern lights (LightHangOrc*_l.LC_Out01, light_NN.LC_Out01) displaced 2-22 cm horizontally (vertical median 0.4 cm, none downward), and 16 of them moved between two reads a minute apart.

Context actions: traversal markup

Where Talion can hang, pull up, vault, rail-run or leap from a mount is world data: context-action lines owned by VolumeController, WorldInitializer and StreamingStamp objects, built into a world grid on the ContextActionBuilderThread (see Threads and the frame) and queried by the movement behaviour graphs. ContextActionDB (ContextActionDB.cpp, initialised by 0x1B43A90, vt_0x23010A8) is [GM+0x6DA0]; it registers the cntx resource type.

On disk: CADB and CAWR

CADB (read by 0x7CB194 for the header and vt_0x1FF6B98::vf3 0x501478 for the body; exact in 724 of 724 parts, all version 3):

'CADB' | u32 version | u32 nA | u32 nL | u32 nP | u32 nV | u32 nO          (0x1C bytes)
nA x u32   action palette: hash of a Character/Models/ContextActions record
nL x 0x34  line   {f32 p0[3], f32 p1[3], f32 normal[3], u128 mask}   mask bit i = palette action i
nP x 0x10  plane  {f32 normal[3], f32 d}
nV x 0x20  volume {f32 min[3], f32 max[3], u32 first plane, u32 plane count}
nO x 0x34  owner  {u32 key, f32 min[3], f32 max[3], u32 first line, u32 line count,
                   u32 list-1 first volume, u32 count, u32 list-2 first volume, u32 count}

Across 724 parts: 7,883 of 7,886 palette entries name one of the 124 Character/Models/ContextActions records (fields Group, DebugColor, AdditionalReferencedActions; so a global mask fits 128 bits); 2,209,930 lines, every one owned exactly once and inside its owner's box; 314,834 planes (all unit); 50,668 volumes; 32,104 owners, whose key is the hash of an OBJL object name (32,104 of 32,104: VolumeController 31,803, WorldInitializer 288, StreamingStamp 13), sorted. Lines are not level in general (1,551,979 slope) and 68,951 normals have a vertical component. The busiest actions by line count: Edge_Hang 997,374, Edge_Pullup 388,445, Edge_NoHang 350,260, Block_Movement 326,177, Block_Movement_RailRun 219,953, Edge_Hang_Dangle 216,272, Hurdle 94,327, Edge_MountJumpTo 93,946, VaultOver_Mount 91,633, Edge_NoTrollJump 83,358, Rail_Running 56,365.

CAWR (ObjRes.ContextLineDB) is the owner index: 'CAWR' | u32 1 | u32 n | n x {u32 key, u32 asset, u32 key} (exact in 724 of 724). key is the hash of the object name; the middle word is 0 on disk and is filled at load with the hash of the cntx resource that holds the owner (2,777 of 2,777 live).

Loading

A loaded cntx object is 0x80 bytes (0x7CD444): lines at +0x20 (stride 0x38: the mask at +0x00, the vectors from +0x10), planes +0x30, volumes +0x40, owner keys +0x50, owners +0x60, the asset's hash at +0x1C, and a node at +0x08 in the database's list DB+0x68 (vf5 0x79A948 asserts "Duplicate asset being loaded."). vf4 0x41E564 rewrites every line mask from palette bits to global bits: the database's action table (DB+0x10 count, 124; DB+0x18 entries of 0x28 bytes {u32 record hash, u128 own bit, u128 closure}, sorted, "Improperly sorted database records!") closes each action over its AdditionalReferencedActions to a fixed point, and a line stores the OR of its actions' closures. A query for one action therefore also finds every line whose action lists it.

Live, for the four loaded parts with a shipped source, 163,048 of 163,048 lines equal the file in geometry and in the predicted global mask (the unconverted file mask equals the live one in only 806 of 161,821), as do 2,364 of 2,364 planes, 344 of 344 volumes and 2,091 of 2,091 owners.

Registration

Register 0x2B9E9C(DB, &handle, object, cut group) is called by three classes: VolumeController (vf21 0x2B9758), WorldInitializer (vf21 0x7A53B8) and StreamingStamp (vf16 0x1C2CF04). It resolves the object in two steps: CAWR (list DB+0x98) binary-searches the hash of the object's name (0x2BA01C), then the owner is found in the cntx object named by the middle word (list DB+0x68) by its key (0x2BB7B8). Live, 3,889 objects carried the VolumeController vtable, 3,748 of them named *.VC* (.VC, .VC_NoHang, .VC_Pullup, .VC_Side, .VC_Alive, .VC_Destroyed) and the rest other markup names; 126 were WorldInitializers and one a StreamingStamp (SS_00).

A registration row (DB+0x50, 0x38 bytes) is {u32 A, u32 B, ptr handle, u8 active +0x10, f32 pos[3] +0x14, f32 quat[4] +0x20, u32 cntx hash +0x30, u32 owner key +0x34}. A is a serial from DB+0x48. B is the cut group: the hash of the object's ContextCuttingGroup property, one of the 11 field-less Character/Models/ContextActionCutGroup records (PropKit, WallKit, WallFlair, GateKit, GateFlair, KeepWallFlair, KeepWallKit, TowerFlair, TowerKit, WildFlair, WildTower); only VolumeControllers pass one. VolumeController::SetEnabled 0x5B21A8 (a script function; the StartEnabled property seeds it) calls 0x2BA65C, which writes the row's active byte and sets DB+0x4C, the rebuild flag.

Building the grid: KILL and CUT volumes

QueueRebuild 0x2BB1C8 files each row by its transform (full rotation, translation only, identity). The build 0x2BA9F8 first instances every owner's two volume lists (planes transformed by 0x2BB01C, {A, B} copied, covered cells flagged), then inserts every line through 0x257C70:

• List 1 = KILL. The engine calls these volumes MarkupKiller (4 to 64 cutting planes, asserted in 0x694AC4; the editor class is MarkupKillerBrush). A line that touches an applicable kill volume is dropped whole.
• List 2 = CUT. 0x2582F8 removes the part of a line inside an applicable cut volume; the outside pieces survive if they are at least 25 cm long (the constant 0x267A4D8, compared squared: grid +0x28 = 625). Uncut lines keep any length (20,199 live lines are under 25 cm).
• Which volumes apply (0x258280): a volume applies to a line unless the line belongs to the volume's own owner (same A) or both have the same nonzero cut group. A line with A = 0 is never touched. A disabled owner's cut volumes are still built (125 of 1,051), but its own lines are hidden from queries.

Measured in a running game (two reads with the world re-streamed between them; 186,023 and 184,993 lines): no line lies more than 1 cm inside a volume that applies to it, for 106 kill and 1,051 cut volumes. Control: the same volumes moved 300 cm along x would contain 31 kill and 1,890 cut lines.

The world grid

[DB+0xB8]:

The cell target is 2000 x 8000 x 2000 cm (0x267A448 .. 0x267A450). Lithlad's grid spans 1,120 x 244 x 1,220 m in 56 x 4 x 61 = 13,664 cells of 2000 x 6105 x 2000 cm (the extent divided by the target, rounded up, on all three axes) and holds 184,993 lines. The inner tables: +0x10 cells (0x28 bytes: u32 index, u8 x, y, z, u8 has kill +7, u8 has cut +8, a run vector at +0x10), +0x20 owner instances (0x20: u32 A, u8 active +4, run vector +8), +0x30 kill volumes, +0x40 cut volumes (0x38: box, u32 plane count +0x18, planes +0x20, u32 A +0x28, u32 B +0x2C, u8 cell range[6] +0x30), +0x50 world planes (16 bytes {n[3], d}, inside when n . p - d <= 0). A world line is 0x40 bytes: u8 flags (bit 0 = owner active), u32 home cell | bit 31, u128 global mask +8, f32 p0 +0x18, p1 +0x24, normal +0x30.

The query

ContextActionDB::Query 0xFEAFB0(DB, AABB box, u128 any, u128 all, u128 none, u128 not_all, results ) locks DB+0x20 and runs 0xFEAE40 over every cell the box covers (0xFEB3B0) and every run in it (0xFEB070). It returns each active line once (de-duplicated by its home cell) for which mask & any != 0, mask & all == all, mask & none == 0 and mask & not_all != not_all (a zero mask skips its test) and whose segment meets the box (slab test 0x5AA774). Results are {u32 n, n x 0x38 {u128 mask, p0, p1, normal}}, at most 256. Running the engine's own 0xFEAE40 in an emulator against live memory, on boxes around the player and on random lines with five or six mask sets, gave the same answer as this model in 96 of 96 and 80 of 80 queries; returning inactive lines would have changed the answer in 18 of 19 and 9 of 9.

The callers build the masks from the action table's own bits (0x41F954): any from a node's GameDBContextAction and GameDBContextAction_OR, all from _AND, none from _NOT_OR, not_all from _NOT_AND. The seven callers of 0xFEAFB0 are behaviour-graph nodes (see Animation): Predicate.DetectedEdge (DetectEdgeData 0x1C6EF90: fields Direction, VerticalOffset, Radius, Height and the five action lists; box = position + (0, VerticalOffset, 0) +- (Radius, Height / 2, Radius); defaults -104, 100 and 10 cm), Predicate.FoundRailing / Modifier.FindRailing (RailingData 0x1C700D4), Modifier.MoveWithBrush (vt_0x2303CD8::vf11 0x1C46DE0), Predicate.FoundContextActionBasic (0x41F39C), Character/Models.LockPhysicsContextActionRecords (0x41E70C -> 0x41FBF0), 0x1C6FA58 (from 0x1BE703C, R_Foot) and 0x3ADC94.

Weather-gated static effects (SWFX)

ObjRes.WeatherFx places effects that play only under a given forecast. It is the empty 16-byte header in 1,300 of 1,310 parts; the ten others are seven region main parts (not Barad-dur), Minas Ithil's t1prefallprops and t1postfallprops, and Gorgoroth's fx_environment (10 types, 51 points).

'SWFX' | u32 1 | u32 n_types | u32 n_points
n_types  x {u32 record hash, u32 first point, u32 END point}     ranges tile [0, e0), [e0, e1) ...
n_points x f32 position[3]
n_points x f32 quat[4]

The ranges tile the points in 10 of 10 blocks (79 of 79 types; quaternions unit 8,110 of 8,110). The type hashes are records of Weather/FX/Static (the category cached at 0x2700050; 32 records, 28 of 28 distinct hashes resolve): Ledge_1m / 3m / 5m / 10m / Corner, Curtain01_20m, Curtain02_40m, Curtain_CeilingFill01 / 02, Lightning_Far0N, Lightning_Near0N, Lightning_VOL_Far0N, Lightning_VOL_Near0N, Desert_AtmoDust_Lrg01, ISLD_Pollen_5m / 10m. A record has a BubbleRadius (cm: Ledge_1m 2,000, Curtain01_20m 15,000, Desert_AtmoDust_Lrg01 45,000, Lightning_VOL_Near 125,000, Lightning_Far 300,000) and Data = _StructuresWeather.StaticFXData {Predicate -> _StructuresWeather.StaticFXPredicate {Forecast -> Weather/Predicates/Forecast/<RainLight | RainMedium | RainHeavy | ...>}, Effect (fx\weather\rain\edgedrips\rainlight_edgedrip01_1m.fxp, fx\weather\lightning\static_lightning_rainheavy_far01.fxp)}: play this effect here, under that forecast, within the bubble radius. Lithlad's main part has six types and 1,646 points (Ledge_1m 590, Ledge_3m 323, Ledge_5m 242, Ledge_10m 6, Ledge_Corner 421, Desert_AtmoDust_Lrg01 64).

The rain drips sit on the climbing ledges. In Lithlad the nearest live context-action line is within 25 cm of 538 of 590 Ledge_1m points (median 5.1 cm), 296 of 323 Ledge_3m, 233 of 242 Ledge_5m, 6 of 6 Ledge_10m and 310 of 421 Ledge_Corner; with every point moved 3 m sideways the counts fall to 34, 23, 15, 0 and 24. The dust points (64) lie a median 9.7 m from any line.

Collision and navigation

• World collision objects. Every part opens its object list with WorldCollision objects (vt_0x1DCD4A8; PhysicsBSP, and in a main part also PlayerBlocker, CameraBlocker, LargeAIIgnore). Their records in ObjRes.WorldPhysics carry no prefab path; the block ends with the records' Havok shapes (hknpCompoundShape in a TAG0 tagfile, SDK SDKV20160200; see Physics for the shape classes). A FireProp that owns no physics pieces is covered by its part's static PhysicsBSP collision instead.
• Navigation. Every part carries one HKAI section, an HKAI header around a Havok tagfile holding hkaiNavMesh data (see Physics for the navigation-mesh layout). Placed objects carry more navigation data: AINavMeshObstacle (11,588 live in Lithlad), AINavMeshDynamicNavMeshPiece (the props' own navigation pieces, .dnmp; 1,998 live) and DynamicOffMeshLink (2,986 live).
• A prop's navigation obstacle and context-line controller are prefab siblings at one position, named with the prefab instance's prefix (barre_02.VC, barre_02.NMO, barre_02.FV_00).

Open questions

• What the 109 world contexts without a name table hold, and whether every loaded section maps to exactly one context: 127 loaded sections, 129 contexts and 126 live WorldInitializer objects (every part read opens with one) suggest it.
• What the WDSC cell groups' keys and the trailing block's hulls and records are for. The handler's module also manages visibility layers (render commands that set a layer's state, keyed by two words; asserts "Setting visibility layer state on a visibility layer that does not exist" and "ReleaseVisibilityLayer must be paired with AddVisibilityLayer"), and group keys such as 0xD7D4D586 / 0xD7D4D587 / 0xD7D4D58B differ only in their last character, as numbered layer names would; matching the keys against the layer state requests would settle it. What kind 3 excludes instances from (the second bounds list) is part of the same question.
• Whether NativeLife emitter positions are world space: of 48 axis and sign choices the identity is by far the closest to the prefab's own objects (median 41 m), but no live emitter has been read.
• What spawns a SWFX effect at run time: the block handler's vf5 0x6FEAF90 only queues the object on [GM+0x30]. Whether the rain-drip points are generated from the same ledge markup as the context-action lines, as their placement suggests.
• Whether the lantern lights that sit 2-22 cm from their authored points follow an animated or simulated lamp socket (16 of 42 moved within a minute).
• Which context-action nodes Talion's own behaviour graph uses, and whether the other node types that carry the same action lists (FoundContextAction, AlignToBrush, FoundContextActionEnd, FoundRailingCorner, CameraContextActionData, FindMoveAndPinIK) reach the grid through the seven known callers.
• Why the AI placements move vertically at load (class medians 0.5 to 2.9 m down, single objects up to 38 m down and 7 m up): a projection onto the navigation mesh or the ground would explain a move without a horizontal component.
• Whether the context's resource table at ctx+0x98, which the render and collision binding queries by name, is built from the part's ObjRes.WorldPhysics block.
• What the TERR shader parameter does inside the terrain shaders (400 in most regions, 500 in Lithlad, 1000 at the Black Gate). The terrain domain shader computes its texture coordinate as world.xz / cb0[30].w; if that register is k_vTerrainDomainParams1, the value is the world size of one texture repeat. Matching the register to the constant would settle it.

Shadow of War (Steam), game version 1.21. An unofficial fan research project, not affiliated with or endorsed by Warner Bros. Games, Monolith Productions or Middle-earth Enterprises. Middle-earth: Shadow of War belongs to its owners.