User:DavidJCobb/Stack dumping

Stack dumping is a Papyrus event that can cause unpredictable and potentially severe errors in a player's savegame. My understanding of the phenomenon is as follows:

Overview

Whenever the game engine calls a function, a call stack is generated. A call stack keeps track of the function's state -- which functions have been called (if X calls Y, Y is added to the X call stack), and what values their variables have. Papyrus can only have a certain number of call stacks running at a time; if too many stacks accumulate, some may be suspended (paused). Stacks may also be suspended for other reasons:

• A function that calls Utility.Wait(n) will suspend its call stack for n seconds.
• If a function tries to access a resource that another script is currently accessing (a "shared resource"), the function will be suspended until the resource becomes available.
  • One common example of a shared resource is the player object.
• Some events' handlers cannot run concurrently with themselves. If the event occurs several times in a short period, only one call to its event handler will run immediately; the other calls will exist as suspended stacks.

If too many suspended stacks accumulate, the Papyrus engine will not be able to hold them all; suspended stacks will be selected at random and discarded, and information about those stacks will be printed to the Papyrus logs, prefixed by this message:

Suspended stack count is over our warning threshold, dumping stacks:

This means that if a mod has a problem that leads to stack dumping, that mod will interfere with itself, with other mods, and with Bethesda-authored scripts (because Papyrus doesn't care where the problem is coming from; it'll dump as many stacks as it can, indiscriminately, until it feels safe). The problem will have unpredictable consequences, which may or may not break other content or otherwise lead to save corruption. As a modder, the best approach to take here is the maximally cautious one, because even if a stack dumping issue doesn't wreck your savegame, it could wreck someone else's. If your mod has a flaw that leads to stack dumping, fixing that needs to be a high priority.

Specific methods to minimize stack dumping

• There are a number of events that can rapidly generate suspended stacks, because they can occur several times in a short period (such that several calls to your event handlers are queued). If you must listen to these events, then minimize the length of your event handlers, so that the resulting stacks are executed as quickly as possible.
  • Don't use Utility.Wait(n) in these event handlers. It will by definition make your event handlers take longer to execute. For significant delays, calling RegisterForSingleUpdate(n) and returning (and doing the rest of your processing in OnUpdate) is a good alternative: your stack won't stick around during the wait time.
  • Minimize the amount of times you access shared resources (such as the player), in order to shorten your event handler's overall execution time.
    • Remember that your own resources can be shared. If you have a magic effect that coordinates itself via a quest, anyone with that magic effect will be sharing that quest.
  • If you don't need instantaneous processing and aren't using Utility.Wait(), you can "eat" some of the stacks by having your event handlers RegisterForSingleUpdate(n), and doing your actual processing in an OnUpdate() event. If x events occur within n seconds of each other, you'll process all of those events just one time.
    • Effectively, those x events will register for an update and return immediately, clearing x stacks out of memory. Then, n seconds after the xth event fired, one new stack will be created to do your processing.
  • If you need to quickly process the first event in a bunch, then you can "eat" a few of the redundant stacks by using a boolean variable to prevent concurrency. Check if it's true at the start of the function, and return if it is. Otherwise, set it to true, do your processing, and then set it back to false.

Events that can trigger large numbers of suspended stacks

If you try to do advanced processing within these events, you're gonna have a bad time. In a worst-case scenario, the player will rapidly (or even instantly) generate more suspended stacks than Papyrus can handle, leading to stack dumping.

• OnHit can generate a large number of suspended stacks, because the event is duplicated if the victim is hit with an enchanted weapon.
  • Consider the case of an encounter between a player and five bandits, each of which takes four to seven hits to kill, and each of which has an OnHit event handler. Without an enchanted weapon, the player will generate 20 to 35 OnHit events to kill the bandits. If the player's weapon has two enchantments, she will generate 60 to 105 OnHit events. Meanwhile, the bandits will generate many OnHit events on the player (who can heal herself). The problem can quickly balloon out of control.
    • Incidentally, there's an area in Pinewatch where you can end up fighting five or six bandits all at once. It's also possible to obtain a sword with two enchantments very early in the game, courtesy of Meridia.
  • OnHit can run concurrently with itself, so (ignoring outside influences) OnHit stacks should only be suspended if too many of them are generated too quickly.
• OnItemAdded and OnItemRemoved can generate a large number of suspended stacks if no inventory event filter is used. The player can transfer large masses of items (e.g. in their home), for example.
  • Even batch operations, like RemoveAllItems(), can cause stack dumping. One stack is generated for every type of item transferred, and inventory events can't run concurrently with themselves, so all but one of those stacks will be suspended the instant it is created.
    • Friendly reminder: built-in, Bethesda-authored content calls RemoveAllItems() on the player. Diplomatic Immunity, The Forsworn Conspiracy, The Mind of Madness, Restoration Ritual Spell, and Unbound all remove the player's inventory at some point.
  • For OnItemAdded and OnItemRemoved, I list some additional methods for avoiding stack dumping here.
  • OnItemAdded and OnItemRemoved don't appear to be able to run concurrently with themselves, so their stacks will be suspended as a matter of course, and it'll take longer for those stacks to clear.

Miscellaneous facts

• RegisterForSingleUpdate(n) does not create a stack and suspend it for n seconds. I have confirmed this experimentally.

Code snippets

These code snippets have not been tested directly, and the benefits are purely theoretical. The only way to know for sure if they'll help is to devise stress tests for whatever you're trying to do. Run a stress test with code that doesn't use these methods, and a stress test with code that does, and check the logs to see what code dumps stacks first and worst.

Process a task OnHit, and wait for X seconds during or after the task

This simple example blocks concurrent calls, and replaces a Utility.Wait(X) call with a RegisterForSingleUpdate(X) call. If we used Utility.Wait(X), we'd suspend our stack for X seconds; by using RegisterForSingleUpdate and OnUpdate, we get rid of the stack entirely during those X seconds, and create a new stack to continue processing after the delay.

Even if we didn't prevent concurrency using pbBusy, we'd still get rid of all OnHit stacks at the start of the wait, and create just one OnUpdate stack at the end of the wait; however, with every hit, the wait time would be reset to X seconds. Whether or not we block concurrent calls, hits that occur during the wait time will be ignored.

Bool Property pbBusy = False Auto Hidden ; prevents concurrent execution

Event OnHit(ObjectReference akAggressor, Form akSource, Projectile akProjectile, Bool abPowerAttack, Bool abSneakAttack, Bool abBashAttack, Bool abHitBlocked)
   If pbBusy
      Return
   EndIf
   pbBusy = True
   ;
   ; Do your processing here.
   ;
   RegisterForSingleUpdate(x) ; replace "X" with the seconds you want to spend waiting
EndEvent

Event OnUpdate()
   ;
   ; Continue processing here (or do nothing, if your goal was to wait X seconds after processing a task before allowing more tasks to be processed).
   ;
   pbBusy = False ; We're done!
EndEvent

Process a task OnHit, without outright discarding concurrent calls

We want to avoid doing processing during concurrent calls, but we may not want to outright discard concurrent calls. If an enemy is hit four times in rapid succession, we might want to process all four hits, but we'll still want to do our actual processing in just one stack, if possible. To accomplish this, we block concurrent calls, but we keep track of how many such calls we've blocked; and in a loop, we run our task that number of times.

Bool Property pbBusy = False Auto Hidden ; prevents concurrent execution
Int Property piHitsToProcess = 0 Auto Hidden

Event OnHit(ObjectReference akAggressor, Form akSource, Projectile akProjectile, Bool abPowerAttack, Bool abSneakAttack, Bool abBashAttack, Bool abHitBlocked)
   piHitsToProcess += 1 ; We need to process another hit.
   If pbBusy
      Return ; ...but we're busy right now, so we'll get to it later.
   EndIf
   DoProcess()
EndEvent

Event OnUpdate() ; This function exists to start processing hits that happened while we're busy.
   If pbBusy
      Return ; ...but we can't do that if we've become busy again.
   EndIf
   DoProcess()
EndEvent

Function DoProcess()
   If pbBusy
      Return ; We're busy. Return; we'll deal with new calls later.
   EndIf
   pbBusy = True                         ; Block concurrent calls.
   Int iTasksToProcess = piHitsToProcess ; This is how many times we plan to process our task.
   piHitsToProcess = 0                   ; Reset the hit counter, so we can handle hits that happen while we're processing right now.
   Int iIterator = 0
   While iIterator < iTasksToProcess
      ;
      ; Do your processing here.
      ;
      iIterator += 1
   EndWhile
   pbBusy = False ; Stop blocking concurrent calls.
   If piHitsToProcess
      ;
      ; If this runs, it's because more events fired while we were processing. 
      ; We need to re-process. Using an update instead of a recursive function 
      ; call will shift us to a "fresh" call stack -- we shouldn't run into 
      ; errors with too much recursion.
      ;
      RegisterForSingleUpdate(0.1) ; Extending this delay may be a good idea.
   EndIf
EndFunction