Audio (Wwise) and video (Bink)

How the engine plays sound through a statically linked Wwise 2016 and video through Bink 2: the engine interface, the sound manager and its game objects, posting an event end to end, banks, the AudioHooks dialogue graph, and the movie manager, video resources and the movie data.

The stack

gameplay / AudioHooks / presentations              UI, presentations, the UI.PlayVideo command
        |                                                   |
emitter play 0x3AD6A8 -> 0x3AD7AC                   play-movie-record 0x32F3AC (Interface/Movies)
        |                                                   |
engine interface [base+0x26FEFD0] (vt_0x1E6AEC0)    engine interface [base+0x26FEFD0]
   vf10 -> [+0x98] sound manager vt_0x1E6D5F8          vf27 -> [+0x88] WBMovieMgr vt_0x230B308
        |                                                   |  vf2 create (".vid" -> resource type 0x10)
   vf13 PostEvent / vf32 LoadBank                   video resource vt_0x2005158 (0x168 bytes), list 0x267E400
        |                                                   |
Wwise (static, inside the executable)               Bink player vt_0x1DB1740 (0x3F0 bytes) -> bink2w64.dll
   message queue -> AK::EventManager thread            decoded every frame on the render thread (0x3322BC)
        |                                               drawn with drawprimvideobink*.fxo
xaudio2_9.dll                                       movie sound is Wwise (738 of 739 files have no audio track)

The engine interface

[base+0x26FEFD0] is the engine interface (vt_0x1E6AEC0, 0xD8 bytes, built by 0x117F8B0 via 0x1178778). Neither media service has a global of its own; both are reached through it:

Threads

Wwise's threads carry no engine name: AK::EventManager (created by 0x1310C10; the audio render thread), AK::BankManager (vt_0x22D65D8 vf1 0x1312008), AK::IOThread (0x132E474, streaming) and AK Suspended (0x1309C78). GameAudioMgrThread is an engine thread (routine 0x181F638). Bink runs five threads inside bink2w64.dll, started by BinkStartAsyncThread from the render-side initialisation 0x1CB60C0 (via 0x1CD21A0, which also calls BinkSetMemory); Bink's own audio output is set up there too (BinkSetSoundSystem2 + BinkOpenXAudio2, 0x1CD22AC), and 0x1CB6194 stops the threads and frees Bink's globals.

GameAudioMgrThread is started by 0x178C98C when byte [base+0x266D9D9] is set. Its routine registers with an engine service, then runs a job loop (0x6AB618) until its stop flag is set. Its first job (0x181F704 -> 0x181ED58) builds the AudioHooks compiled cache (below).

VideoClipRequestThread does nothing on PC. The engine interface builds its service once (0x117D315: 0x118 bytes, vt_0x22C0DD0, stored at +0xB50), and the thread (routine 0x11C7488, loop 0x11C7274) waits on an event, takes the request at +0x100 and dispatches its op code (request +0x08: 0-6 and 8-12) through a table of 13 function pointers that the constructor 0x119A5CC fills. All 13 are the same stub, mov eax, 5; ret. The handler stores that result at request +0x358 (2 while running), and a non-zero result finishes the request with state 3 (request +0x0C), so every request ends at once with result 5. The vtable is referenced only by the constructor and the destructor, so no class replaces the table.

The sound manager

The sound manager (vt_0x1E6D5F8, 47 methods) tests its enabled byte at +0x09 at the start of nearly every method.

Methods identified: vf1 Init 0x11AC2E4 (registers the "SoundManagerGlobalEmitter"), vf2 Shutdown, vf5 language, vf8 0x3AD880 IsEnabled, vf12 0x74ED6C PostEvent without an emitter (2D), vf13 0x3ADC1C PostEvent on an emitter, vf17 0x49BFBC SetSwitch by name, vf18 / vf19 SetRTPCValue by name (0x3AEC6C -> 0x3AEB8C -> 0x3AEBCC), vf20 0x5D7FFC0 (hashes a name and queues message 7), vf32 0x54FF78 LoadBank, vf33 0x551618 UnloadBank, vf39 0x47CF6C name -> id.

Sound objects and emitters

A sound object (vt_0x1DC35E8, pool stride 0x48) is one Wwise game object:

Sound objects are created on demand by 0x6861A4 when an emitter has none, up to 725 in all (cmp eax, 0x2D5 at 0x6861C2), and initialised by 0x686230, which registers the id with Wwise (messages 0xB and 0xE) and sends the position (0x3AE0BC). PostEvent refuses a 51st event on one object (0x3ADC5B); the end-of-event callback counts the events down.

The emitter is the engine-side structure: +0x00 u32 id, +0x08 sound object, +0x10 position, +0x1C orientation, +0x28 registered, +0x29 enabled, +0x2B has a position, +0x34 last playing id, +0x38 last event id, +0x3C time, +0x60 name, +0x90 / +0xC8 / +0x100 the last event names (set by 0x3AD6A8).

In an open-world scene the vector held 52 and then 121 sound objects a few minutes apart; 121 of 121 passed the back-link check (emitter +0x08 equals the object and emitter +0x00 its id); ids were unique; 94 were positioned and 44 had an event playing. Two sat exactly at the player's position: the player's emitters.

Posting a sound

caller (dialogue 0x54D7E0, audio-key job 0x17A210, ...)
 -> 0x3AD6A8 emitter PlayEvent(emitter, event name, external file, flags); emitter +0x29 must be set
 -> 0x3AD7AC(audio context = [[CGameClientShell]+0xA28]+0xCBA48, emitter, name, file, flags)
      vf8 IsEnabled; the streaming-bank gate 0x3AD888 (false = no post, see "Banks")
      with an emitter -> vf13 0x3ADC1C; without -> vf12 0x74ED6C
 -> vf13: sound object = emitter+8, or create one (0x3AE2F0 / 0x6861A4); push the position (0x3AE0BC);
      refuse at 50 events playing; 0x3AD358(id, object, name, file, flags); on success object+0x30 += 1
 -> 0x3AD358: flags = AK_EndOfEvent | ((flags & 4) << 18); callback 0x3AE47C, cookie = the sound object;
      with an external file: AkExternalSourceInfo {cookie 0x24DB9834 = FNV-1("External_Source"),
      codec 4 (Vorbis), the file name widened with MultiByteToWideChar}
 -> 0x3AD47C AK::SoundEngine::PostEvent(const char*, ...): copies the external sources (0x22F630)
 -> 0x3ADBC0 GetIDFromString: 32-bit FNV-1 (basis 0x811C9DC5) of the lower-cased name
 -> 0x3AD54C PostEvent(AkUniqueID, ...): event = lookup in [base+0x26FF998]+0xC0 (0x3AD664);
      reserve a queued message (0x114BDF8, type 1, 0x38 bytes); playing id = ++[base+0x26FF97C];
      on failure the message becomes type 0x39 (no operation) and the event is released
 -> the AK::EventManager thread executes the queue; at the end of the event Wwise calls 0x3AE47C,
      which queues {cookie, playing id} on the sound manager's +0xB8 list

0x12F2194 is a second PostEvent entry by event id that copies external sources and calls the same core 0x3AD54C. In an idle open-world scene the playing-id counter advanced 30-40 times per second.

Dialogue lines are Wwise external sources. The dialogue-event player 0x54D7E0 (it reads the AudioHooks/DialogEvents category, cooldowns, facial animation and subtitle behaviour) passes a line's file name as the external source of a generic event; the file is Vorbis, looked up in sound\<language>\External*.pck. 0xD24F80 builds the event name ("%s%s") and asks CanPlay 0x66BC00 first.

Wwise inside the executable

Wwise is linked statically: its code lies mostly in 0x12E0000-0x1340000, its file layer in 0xC35060-0xC3BC90 and the engine's integration in 0x11A2000-0x11DC000. The executable exports AK::GetWwiseXAudio2Interface; xaudio2_9.dll is loaded at run time, not imported. Banks are BKHD version 118 (six of six sampled, init.bnk included).

The executable exports 286 named symbols, 263 of them Wwise's (the AK::SoundEngine, Query, MemoryMgr, StreamMgr, DynamicSequence, MotionEngine, MusicEngine and Monitor APIs with their full C++ signatures, and Wwise's globals); the rest are Scaleform system and matrix functions and two memory pool ids. The exported names make these globals exact:

Every API call reserves a message on Wwise's queue with 0x114BDF8(&message, type, size) (55 call sites). Reading the type each exported API function passes gives the message types by name:

The string overloads hash with 0x3ADBC0 and the wide-string ones with 0x11A2B0C (also FNV-1).

Banks

On disk. Banks live inside the archives: 6,974 loose sound\...\*.bnk entries plus 1,459 BKHD sections inside snd_*.embb bundles, 8,433 banks with 2,723 distinct names and 12,747 distinct event ids. Language banks are in sound\<language>\. Streamed media and voice are loose Wwise packages under game\sound\: eight Streaming*.pck files (about 1.16 GB) and, per language, External*.pck and vo*.pck. The file layer formats sound\%s\%u.bnk / .wem / .pck and sound\%u.wem / .pck (0x11D2550): files by id, with a language folder.

Streaming bank data (Sound/): Sound/StreamingSoundBanks (1,958 records: EventPrefixes, EventSubStrings, GlobalStateEvents, SoundEventPlaybackStrategy such as PlayWhenLoaded, SoundBundleToLoad naming sound\bundles\snd_*.bndl, Priority, StopEventsBeforeUnloading, AlwaysInUse, WaitSecondsBeforeUnloading, SoundBankGroup), StreamingSoundBankGroups (13), StreamingSoundBankSubLists (19), StreamingSoundBankList (1, GlobalMemoryBudgetMB), StreamingGlobalAssetLists (4), StreamingGlobalStateEvents (11) and DlcSuppressLists (2). An event whose name matches a record's prefix loads that record's sound bundle, a bare .bndl manifest that names the .bnk with no container behind it.

The bank gate. 0x3AD888(context, emitter, name, flags) runs before every post, and a false result means the event is not posted (0x3AD7AC returns 0). Until the bank system is ready (context+0x121 clear) it queues the event at context+0xC0 and returns false. Then it hashes the name and binary-searches a sorted table (context+0x90, 8-byte entries). An event that is not in the table, or whose entry has no bank (u16 +0x04 = 0xFFFF), plays at once. Otherwise, for the entry's bank (built by 0x1D2B658: +0x50 SoundEventPlaybackStrategy, +0x60 Priority, +0x94 WaitSecondsBeforeUnloading in ms, +0x98 AlwaysInUse):

1. a bank with byte +0x9C set refuses the event;
2. the bank's last-use time +0xAC is stamped;
3. a bank not yet requested (+0x9D clear) refuses the event when its size would pass the context's limit and its priority is below context+0x1C; otherwise it is marked requested and joins the load list context+0xA8. The limit +0x10 is GlobalMemoryBudgetMB in bytes and +0x14 GlobalMemoryCloseToBudgetMB (62,914,560 and 52,428,800 live, exactly 60 and 50 MiB); +0x18 is the amount in use (48.0 MiB in one loaded Lithlad, with 4,596 events in the table);
4. the emitter's id joins the bank's list +0xE0;
5. a bank that belongs to an exclusive slot (+0x88, byte +0x99) replaces the slot's previous bank in the map at context+0x48, which is released;
6. a loaded bank (+0x9B) plays the event now;
7. otherwise the event waits on the bank only when its strategy is PlayWhenLoaded (1) and fewer than 20 events are waiting (+0xB0). Under DoNotPlay (2) the event is dropped and only the load goes ahead. DoNotPlay is set on 76 of the 1,958 records, PlayWhenLoaded on 1,882.

Loading. A bank is an engine resource: the bank loader (vt_0x22C3850 vf4 0x54FEA4) gets the file's bytes, builds a 0x18-byte handle (+0x00 the bank id, +0x04 size, +0x08 data, +0x10 "not init.bnk", +0x11 "not a _Metadata bank") and calls the sound manager's vf32 0x54FF78, which, under the bank lock, parks banks until init.bnk has loaded. 0xC378A0 loads one bank: the size must be at least 0x14 and the bank's language id (BKHD+0x10) must be 0 or the current language, so banks of another language are refused; then 0x12F1A00 copies it into a Wwise pool (or 0x550088 loads it in place).

The live bank list. The bank manager [base+0x26FF9B0] keeps a hash table at +0x90 (buckets, u32 count of buckets, capacity, item count); entries point at bank slots (vt_0x1DBAA38: +0x10 u32 bank id, +0x18 data, +0x20 next in the chain, +0x38 u64 size). In an open-world scene 83 banks (210 MB) were loaded, all identified by id; the largest were global_ui (26.7 MB), wraith (25.9), cbt_insta_chordkills (13.2), wld_glb (13.0), mus_global (12.4), char_troll_glb (10.8) and cbt_player (10.5). Only 51 of the 83 data pointers still pointed at the bank: the source buffer of a copy-loaded bank is freed, so +0x18 is not reliable. Two more tables exist at +0x78 (12,289 buckets, 8,775 items) and +0xA8 (29 buckets, 4 items).

AudioHooks: the dialogue graph

AudioHooks data (keys, hooks, conditions, game conditions, dialogue events) decides which line a character says. It is compiled into the audio cache at load by 0x1872B88 on the audio thread (builders 0x1871D7C, 0x1871C44 and 0x1872800); a change to the GameDB records afterwards has no effect. Fields read live instead: the dialogue-event priorities (0x66BCA0 / 0x66BD18), DialogCooldowns.Duration, Faction/Events/Events.KeepPriority, catalyst RandomChance and Presentation/Cooldowns.Duration.

• Condition types are a 146-value string enumeration (registrar 0x83BC8C; CharacterEvent = 95, CharacterEventCount = 96), evaluated by the switch 0x189EC0 (case table 0x18B944). The CharacterEvent matcher 0x74EC88 compares an event's age (now minus the event time) with the condition's minimum and maximum.
• AddKey 0x18E560 runs every hook of a key, without stopping at the first match. CanPlay 0x66BC00 lets a new line take over a talking character only if its PriorityInterrupt is strictly greater than the current line's PriorityContinue; between equal priorities the first line wins.
• A CharacterModel game condition (type 121) compiles to one bit (the payload's u16 at +0x62), tested against the speaker's audio-character bit set at +0x188 (0x18A8BA -> 0x18BBD8). 124 such conditions exist; the player's audio character is [GameAudioManager+0xA5138].
• Audio characters (vt_0x1F797A8; a pool inside the audio manager at +0x38, stride 0x4F8) keep a copy of their captain's event history at +0x248, taken when the character spawns.
• CCharacter.AddAudioHookKey 0x2D9B1C(character, key) fires a key on a character directly; keys without conditions (such as the per-line keys of scripted scenes) play whenever they are fired.

Video

WBMovieMgr (vt_0x230B308, 5 methods, WBMovieMgr.cpp). vf2 0x32F930 Create(out, name, flags): a .vid name goes to 0x33048C (a resource request of type 0x10 through the resource system, malloc(0x168), constructor 0x3306A0, then vf22 with the UHD setting [base+0x272EC80]); any other name goes to 0x1CA0538, which accepts only .bk2, .bik and .mp4, and only while byte [base+0x2A37714] is set (it is clear in the shipped game). vf3 0x1CA0608 and vf4 0x331170 call vf5(bool) and vf16(float) on every live video under the list lock.

A video resource (vt_0x2005158, 0x168 bytes; base class vt_0x1DB1800, constructor 0x33092C: +0x08 list node, +0x18 reference count): +0x20 the Bink player, +0x28 a lock, +0x50 a copy of the play state, +0x58 / +0x60 state. vf22 0x33058C queues it to the render ring; 0x3305D0 links it into the global video list (head 0x267E400, node = resource + 8, lock 0x2A37718). vf23 0x529BF8 opens the file (Bink player vf6) and vf25 0x33232C is the per-frame update (decode via 0x33236C, then copy player +0x168 to +0x50). The render command 0x3322BC walks the list on the render thread every frame and calls vf25 on each video.

The Bink player (vt_0x1DB1740, 0x3F0 bytes, constructor 0x330830): +0x08 the HBINK, +0xF8 open state (0 open, 3 failed), +0x168 play state (2 = failed to open), +0x170 the file name. vf14 0x331D78 Open(name, flags) calls BinkOpenWithOptions with a custom file callback 0x7A9EA8 that reads through the engine's file layer (0x1CD2194). vf15 0x331358 pauses, vf7 0x331620 sets volume and speakers, vf11 0x332420 registers frame buffers. HBINK: +0x00 width, +0x04 height, +0x08 frames, +0x0C current frame. On the army screen one video was live (interface\videos\kraken\traits\resistant_beast.vib, 1024 x 512, 239 frames, looping); in the open world the list was empty and no Bink player existed.

The console variable Video.UHD (WBMovieMgr.cpp, handler 0x1CA04F8: 0 never, 1 always, 2 default) is stored in [base+0x272EC80].

Movie data and files

Interface/Movies holds 950 records with 29 fields: Movie (the path), AudioBundle, AdditionalAssetBundle, Loop, MenuBackgroundVideo, CanSkip, FullScreen, RenderGameInBackground, ScreenPosition, PauseGame, PauseAI, PauseMovieOnLastFrame, RenderBars, PlayCreditsOnFinish, fade-in and fade-out frames, SyncVideoToWwiseAudio, WwiseAudioKey, SkipSoundKey, GlobalBusKey, StopGlobalBusKey, CommandSet, Sounds, Events, OverlayVideos, OverlayMaterial and RequiredSubtitleEntries. Records name a .vid; the engine opens the .vib beside it. 939 records carry a path: 725 resolve to a shipped movie (711 distinct files) and 214 name movies that do not ship (cut skill-progression, attribute and trait clips). Flag counts: SyncVideoToWwiseAudio 947, PauseGame 943, CanSkip 919, Loop 740, FullScreen 210. Event lists live in Interface/Movies/Events/{PresentationEvents, BossInfoEvent, UIVideoElementEvents, HUDWidgetConfig, GameEvents}.

The game ships 743 movie files under game\cinematics\videos (about 10 GB) and game\interface\videos (about 2 GB): 12.85 GB and 3.8 hours in all. 668 are Bink 2 (KB2j 661, KB2i 7) and 71 Bink 1 (BIKi), all named .vib; the header word at +0x04 is the file size minus 8 in 739 of 739. Resolutions: 1024 x 512 (479, the UI clips), 1920 x 1080 (125, the cinematics), 512 x 256 (76), 4 x 4 (56 placeholder clips of the third story DLC), 1280 x 720 (2) and 3840 x 2160 (1). Only interface\videos\intro.vib has an audio track: movie sound comes from Wwise.

Who plays a movie. 0x32F3AC (it reads the Interface/Movies fields and the master volume and starts the interface's cinematic state) plays a movie record. Its callers include the presentation element start 0x5D20C4 (via 0x32F3A0), the console command UI.PlayVideo (handler 0x17C0ADC), a video render-target path 0x335CF4 and the skip prompt 0x17F0E48. The UI video player 0x17FC5C0 (callers include CacheLoadWorldVideo 0x1B76988) creates videos through WBMovieMgr vf2; Video.GotoFrame is handled by 0x17FE1C8.

Open questions

• What sets a bank's byte +0x9C, and what replays the events waiting on a bank when it finishes loading.
• The order in which the Wwise file layer searches language folders, packages and loose files.
• Which engine objects own the emitters, and where positions are pushed each frame.

Shadow of War (Steam), game version 1.21. An unofficial fan research project, not affiliated with or endorsed by Warner Bros. Games, Monolith Productions or Middle-earth Enterprises. Middle-earth: Shadow of War belongs to its owners.