Editing TES5Edit Cleaning Guide - TES5Edit

[[Category:TES5Edit]]
{{Languages}}
=Introduction=

Clean mods are essential to the creation of a stable modded game environment in which there are minimal conflicts between the mods used. This guide is designed to introduce mod authors and mod users to the concept of mod cleaning and to direct them to instructions on how to clean mods using TES5Edit. There is also a [[TES5Edit_Cleaning_Guide_-_TES5Edit#Glossary|Glossary]] at the end of the guide that may be used as a quick reference if you forget what a term means.

While this guide may seem intimidating, cleaning is not hard once you understand it. Devote some time to reading through this guide and you'll be well on your way to have a squeaky-clean mod list and a fun, more stable game environment.

= Understanding Mod Cleaning =

== Dirty Edits ==

Many mods make unintentional or unnecessary edits to game elements, collectively known as dirty edits. They are often a side-effect of mod creation, and are often due to bugs in the utilities Bethesda has provided to create mods, rather than bad practice by mod authors. Dirty edits can cause a wide range of issues, including incorrect game settings, missing content, broken quests and crashing to desktop. The more dirty edits there are in a mod, and the more mods with dirty edits you use, the more likely you are to experience issues.

=== Types of Dirty Edit ===

==== Identical To Master ====

Identical to Master (ITM) edits are the most common type of dirty edit. They're where a mod has overwritten something in the game without actually changing anything. If another mod higher up in the load order makes an intentional change to that thing, it will have its effect cancelled out by the ITM record, which can cause problems. ITM edits can be cleaned automatically using TES5Edit.

:'''''Example 1: Identical To Master Edits

: Consider a quest mod in which the author has also changed the properties of some sneaking settings to better fit with other areas of the quest mod. The author then later decides the sneaking changes are unnecessary, and sets them back to their original values. If this quest mod were then loaded after a mod that overhauls the sneaking system, the sneaking overhaul's effects would be at least partially undone by the quest mod, which is not the effect intended by the author of either mod, nor the effect desired by the user.

: The solution would be to clean the quest mod, which would remove the ITM edits and allow the sneaking overhaul mod to function as intended.

==== Deleted Reference ====

Deleted references are a significant cause of crashes and game bugs. This is because if one mod tries to modify a reference that another mod has deleted, then the first mod cannot find the reference, and this then causes problems. A deleted reference can be fixed by undeleting then disabling the references and moving it to a position where it is no longer visible in the Creation Kit (TES5Edit sets the Z axis position to -30,000 units). This has the same effect as deleting the reference, without causing issues. Deleted references can be cleaned automatically using TES5Edit.

==== Deleted Navmeshes ====

Like deleting records, deleting navmeshes can cause crashes. However, these cannot be automatically corrected, and require manual cleaning.

==== Wild Edits ====

Wild edits are unintentional or misdirected edits to game elements by a mod author that cause unnecessary conflicts with other mods. They are called wild edits because they often appear to have been made haphazardly with no relation to the author's stated intentions. Because detecting wild edits involves knowing what the mod is supposed to do, they must be cleaned manually.

One example of a wild edit would be when a container is incorrectly modified to include an item, but instead of only that one container including the item, the result is that all containers of the same type include the item. If the description of the mod states that an item is added to a specific container, then it is clear this is a wild edit and the mod should be cleaned by fixing the wild edit.

=== What (Not) to Clean ===

The [[Skyrim_Dirty_Plugins_List|Skyrim Dirty Plugins List]] is an incomplete list of information on how to manually clean certain plugins and what plugins should not be cleaned. As it's an incomplete list, mod descriptions and readme files should be read in case a mod author has stated a plugin should not be cleaned there, but not updated the list.

== What Is Cleaning? ==

Cleaning very broadly defined is a process that analyzes all the changes a mod makes and determines which edits are actually supposed to be in the mod, then removes those that are not.

Cleaning can be automated when dealing with Identical To Master edits and deleted references, but must be performed manually when dealing with deleted navmeshes and wild edits.

=== Automated Cleaning ===

Automated cleaning requires knowledge of exactly which other mods the mod to be cleaned is dependent on, both implicitly and explicitly.

If cleaning dirty edits, the program analyses every record edit a mod makes and compares each one against the record without the edit, removing any edits that match the technical definitions of dirty edits.

If cleaning deleted references, the program analyses each reference a mod edits checking if it has been deleted or not, then undeleting and disabling any deleted references encountered.

=== Manual Cleaning ===

Manual cleaning is a fairly advanced technique that requires those performing it without the aid of specific step-by-step instructions to know how TES5Edit works and to know a bit about how mods work. Mod users will generally not need to perform manual cleaning most of the time, and so the specifics are not covered in this guide.

Manual cleaning is all about using your judgement to decide whether an edit is a valid, intentional edit or a wild edit that should be removed. Wild edits may be obvious such as in the example below, or not obvious, such as some rocks that were accidentally moved a small amount while editing a cell. Once you have judged that an edit is wild, you must then use your knowledge of modding to decide on the correct method of fixing it, and perform the fix correctly. Incorrectly judging an edit to be wild, choosing the wrong method of fixing a wild edit and performing the fix incorrectly will all lead to making the situation worse, which is why knowledge and experience are required when manually cleaning plugins.

:'''''Example 2: Manually Cleaning A Wild Edit'''''

: Consider a mod that is intended to add a powerful sword named the Super Sword of Slaying to the bottom of a new dungeon. However, what it instead does is to change the Silver Sword so that it is now the Super Sword of Slaying, and adds one to the bottom of a new dungeon. The result is that every Silver Sword in the game will now be a Super Sword of Slaying.
: To fix this wild edit, the FormID of the edited Silver Sword record in the mod plugin can be changed from 00###### to 01######, which will make it a new type of weapon instead of an edited Silver Sword. The reference placed in the dungeon can then be updated to use the new FormID. This will result in a Super Sword of Slaying being present in the dungeon, while all Silver Swords remain Silver Swords.

If you don't understand the above example and the fix, you don't possess the knowledge required for manual cleaning, so don't worry about it and just stick to automated cleaning.

== How To Clean? ==

There are two cleaning tutorials available on this wiki:

* [[TES5Edit_Mod_Cleaning_Tutorial|TES5Edit Mod Cleaning Tutorial]]
* [[Fixing_Navmesh_Deletion_Tutorial|Fixing Navmesh Deletion Tutorial]]

There are also several video tutorials linked to on TES5Edit's [http://www.nexusmods.com/skyrim/mods/25859/? Nexus Mods] page.

[[File:Tes5EditVideo.jpg|425px|TES5EDIT : Cleaning your mods]]
: [http://www.youtube.com/watch?v=UOQO2S6HDBw TES5EDIT : Cleaning your mods]

Even when performing automated cleaning, it is important to ensure everything is set up so that cleaning can be performed correctly. Gophers video on TES5Edit: Cleaning your mods is a great way to become familiar with the cleaning process and gives a good example of why it's important to fix deleted references. The information below covers some information that may not be covered in the tutorials.

=== Order Of Cleaning ===

It is recommended that you begin cleaning at the top of your load order and just do down the list. This will ensure that dependencies will be cleaned before the mods that are dependent on them, so long as your load order is correct.

It is important to ensure a mod's dependencies are clean before cleaning the mod itself because the automated cleaning process will assume that the dependencies are clean. If this assumption is incorrect, it may lead to the mod being incorrectly cleaned.

=== Loading Plugins Correctly ===

The terms '''explicit dependency''' and '''implicit dependency''' are used in this section. It is important that you understand these terms, so before continuing, here are their definitions:

: An '''explicit dependency''' is a plugin that another plugin relies on, and which has its filename explicitly referenced within the latter plugin. Explicit dependencies are easily determined because they can be viewed in Wrye Bash, and if an explicit dependency is missing or loaded after the plugin that depends on it, the game will crash on startup. A plugin's explicit dependencies are also known as its masters.

: An '''implicit dependency''' is a plugin that another plugin depends on, but which is not referenced by the latter plugin. Implicit dependencies must be determined by checking if the mod author has listed any required mods for their mod, and by considering the nature of the mod. For example, compatibility patch plugins are often implicitly dependent on the plugins of the mods they resolve compatibility issues for.

Having the correct load order set for your mod plugins is essential for them to be cleaned correctly. Load ordering is outside the scope of this guide, but the easiest way to get a load order that is likely to be correct is to use [https://code.google.com/p/better-oblivion-sorting-software/downloads/list BOSS].

It is highly recommended that when cleaning a plugin, only the plugin itself and the plugins it depends on should be loaded. Having any other plugins loaded introduces the risk of their contents interfering with the cleaning process and leading to invalid results.

There are two absolute requirements to loading plugins for cleaning:

# The plugin to be cleaned must be the last loaded plugin in your load order.
# Any plugins upon which the plugin to be cleaned depends must be loaded. This includes both implicit and explicit dependencies.

Due to these rules, you cannot clean more than one mod in the same TES5Edit session. If you have multiple plugins to clean, then they must each be loaded into separate TES5Edit sessions and cleaned separately.

:'''''Example 3: Loading Explicit And Implicit Dependencies'''''

: Consider a plugin "VillageMod-QuestMod Patch.esp" that acts as a compatibility patch between VillageMod.esp and QuestMod.esp.

: VillageMod.esp changes (among other things) some of the names of the NPCs in the village and changes the landscaping around it, but QuestMod.esp includes voice-acted dialogue that uses the original NPC names, and adds a wall around the village that is partially buried by the landscape changes of VillageMod.esp when the two are used together.

:"VillageMod-QuestMod Patch.esp" reverts the names of the NPCs back to their originals to match with QuestMod.esp's voice acting and moves the walls added by QuestMod.esp so that they are no longer buried. As it changes the NPC names, it has Skyrim.esm as an explicit dependency, and as it changes the walls, it has QuestMod.esp as an explicit dependency. However, because it doesn't change anything added by VillageMod.esp, VillageMod.esp is an implicit dependency.

:"VillageMod-QuestMod Patch.esp" will refuse to load without also loading Skyrim.esm and QuestMod.esp as they are explicit dependencies, but it is possible to load it without also loading VillageMod.esp. However, if this is done then the name changes in "VillageMod-QuestMod Patch.esp" will appear to be Identical To Master edits, as they will be identical to the names in Skyrim.esm and QuestMod.esp does not touch them, and so the name changes in the patch will be incorrectly removed. If all three plugins are loaded, then the name changes will not appear to be Identical To Master edits, as they are different to the names in VillageMod.esp.

== A Community Effort ==

The principal reason for the presence of dirty mods is a lack of awareness of the issues surrounding dirty mods and the importance of mod cleaning. As such, a community-wide effort is required to raise awareness and so reduce the incidence of dirty mods.

It's '''NOT''' a good idea for everyone to just go ahead and clean their full load order and then forget about it.

The problem is twofold. First, to properly clean mods usually requires an understanding of the '''intent''' of the mod author. There might be implicit dependencies on other mods which are not part of the master list. Second, and following from the first, depending on how the cleaning was done, people end up with slightly different versions of these modules, which is going to make support in case of problems a nightmare for the mod author.

Mod quality is a community-wide problem that needs to be properly addressed on a community level. Mod authors need to ensure that their mods are clean, because they only need to clean their mod once for all future users to benefit, whereas each and every user of a dirty mod would have to clean it themselves, which is hugely inefficient.

The BOSS team provides a focus for community efforts to raise awareness regarding mod cleaning, by accepting reports of dirty mods and attaching them to the relevant plugins in the BOSS masterlist as messages containing a link to this wiki page, along with any additional information supplied. This data is then extracted and displayed to the user by BOSS when it runs, and the data can also be accessed by other utilities that make use of the BOSS API. The result is a central repository of information that can be accessed by a large number of people.

For mod authors:
# '''Clean''' your mods properly. All of them. You are the one who really knows the intention of your mod. Closely review all changes to make sure they preserve the intent of your mod.
# Cleaning mods is NOT just limited to running the automated functions and be done with it. Manually review every override record your module(s) contain to make sure that that's really what you intended to do.
# '''List in your documentation''' that you HAVE cleaned your mods and which version of TES5Edit you used for it. If there were any implicit dependencies that you had to observe while doing the cleaning, specifically list them and the rational for them.

For mod users:
# '''Check''' any mods you use that do not specifically mention being properly cleaned (by basically doing a dry run of the cleaning process).
# If any of your mods are dirty, '''inform the author'''. The mod authors are the ones that really understand the intention of their mods. They are the ones that are in the best position to properly clean them. And if they only publish properly cleaned mods, the whole community benefits from it.
# If a mod has been deserted by its author or the author is unwilling to fix his mess, just '''leave a short note''' on the comments and/or in the RELz thread to save other mod users from wasting their time. Remember: stay civil; flaming doesn't help anyone.
# If any of your mods are dirty, submit them to the BOSS team as directed in the BOSS readme, so that others can benefit from your findings.

= Glossary =

This glossary contains definitions for the general modding terms and cleaning-specific terms used in this guide.

'''Mod''': A modification for Skyrim. Can be player-made or official DLC.

'''Plugin''': A file with a .esp or .esm file extension (well, it's what's inside the file that matters, but the file extension is a pretty good clue).

'''Record''': A data structure inside a plugin that holds the information on one element of the game. For example, each type of weapon, each race, each potion, etc. have different records.

'''Rule of One''': Only one plugin's changes to a record can be applied by the game. If multiple plugins change the same record, then the changes from the last of those plugins in the load order will be applied. There are a few exemptions, eg. if multiple plugins put things into the same cell, then all those things will be seen in-game, but in general the rule holds for non-complex record types (and complex record types are beyond the scope of this guide).

'''Load order''': The order in which plugins are loaded.

'''Dirty mod''': A mod that contains dirty edits, wild edits, deleted references or any combination of them.

'''Dirty edit''': An edit made which is unnecessary, often caused by Creation Kit bugs or authors making unintended edits then undoing them incorrectly. Such edits can be cleaned automatically.

'''Identical To Master edit''': A type of dirty edit made where the edited value(s) is/are identical to the value(s) without the edit.

'''Wild edit''': An edit made which is not consistent with the intent of the mod, often caused by authors accidentally making small edits or trying to perform an edit in the wrong way. Such edits must be cleaned manually.

'''Deleted reference''': A reference to a record that has been deleted. For example, placing a Silver Sword into the game world creates a Silver Sword reference there, which may then be removed by deleting it.

'''Dependency''': A plugin upon which another plugin relies. For example, Skyrim.esm is a dependency for most mods.

'''Explicit dependency''': A dependency that is recorded within the dependent plugin, such that the game cannot be loaded without the dependency being satisfied.

'''Implicit dependency''': A dependency for which there is no indication given within the dependent plugin, and which can only be determined by considering author intent.

= Additional Resources =

* [[TES5Edit|TES5Edit]]
* [[TES5Edit_Mod_Cleaning_Tutorial|TES5Edit Mod Cleaning Tutorial]]
* [[Fixing_Navmesh_Deletion_Tutorial|Fixing Navmesh Deletion Tutorial]]
* [[Skyrim_Dirty_Plugins_List|Skyrim Dirty Plugins List]]
* [[TES5Edit_Scripting_Functions|TES5Edit Scripting Functions]]
* [https://bethesda.net/community/topic/57570/relz-sseedit/ Official Elderscrolls TES5Edit forum topic]

= Closing Remarks & Credits =

This guide has been adapted from the [https://cs.uesp.net/wiki/TES4Edit_Cleaning_Guide TES4Edit Cleaning Guide]. Much of the content of that guide has been removed for this guide, with all lists of mods being absent: this is partly because there are as yet no mods to list, but mainly because such lists belong on a separate wiki page from the guide. The special cleaning instructions section is also missing for the same reason, and the tips on avoiding dirty mods section is missing because I felt it was counter to the topic of the guide. I also left out the review of key concepts because I felt that having such a section would cause many readers to skip the rest of the information, and so put them in a position of false confidence from where they could do significant harm to their modded game.

I have also created a [[Skyrim_Dirty_Plugins_List|Skyrim Dirty Plugins List]] page for the listing of any plugins that need cleaning, plugins that should not be cleaned, and plugins for which manual cleaning instructions are required.

Many thanks go to the various authors of and contributors to that article, including but not limited to: Brumbek, Arthmoor, Vorians, ElminsterEU, WrinklyNinja, and dev_akm.

Thanks also go to ElminsterEU, Sharlikran, and Zilav for their work in getting TES5Edit released.

Finally, thanks go to the BOSS team members responsible for Skyrim's masterlist, for their efforts in collating dirty mod reports.

Still to do:
* Include pictures for the instructions a la [http://cs.elderscrolls.com/index.php/TES4Edit/Mod_cleaning_tutorial_with_TES4Edit TES4Edit/Mod cleaning tutorial with TES4Edit].
* Get the correctness of the guide verified by the TES5Edit team.
* Create a separate wiki page to hold the list of mods needing cleaning (and which should not be cleaned, if there will be any), and instructions on how to generate the list from BOSS's masterlist.
{{Languages}}