Project:Help wanted: Difference between revisions
imported>Leoetlino (→Debug leftovers: update on StageSelect) |
imported>Leoetlino No edit summary |
||
Line 1: | Line 1: | ||
Many secrets still remain to be uncovered. If you have experience with reverse engineering or if you are looking for a fun project to start with RE (or both!), please help us investigate the following mysteries. | Many secrets still remain to be uncovered. If you have experience with reverse engineering or if you are looking for a fun project to start with RE (or both!), please help us investigate the following mysteries. | ||
== Reverse engineering tips == | |||
{{expand section}} | |||
=== GameData flags === | |||
GameData flags can be read and set from many different systems: | |||
* '''Event flows'''. A very convenient way to determine if a flag is being set in an event flow (or queried, passed to the executable, ...) is to grep [https://github.com/MrCheeze/botw-tools/tree/master/event MrCheeze/botw-tools:event] for the flag name. | |||
* '''Map units'''. LinkTags can be used to conditionally spawn map objects based on GameData flags (which are called SaveFlags in map units) and also to set them. It is extremely common for Area tags to be used for triggering LinkTags (which in turn write to GameData flags). | |||
* '''Executable'''. Of course, the executable also checks for game data flags. Do note that the name of the flag is not always hardcoded, and may instead come from a configuration file (example: the Defeated counters for the [[difficulty scaling]] system). | |||
== Formats == | == Formats == |
Revision as of 15:06, 18 November 2018
Many secrets still remain to be uncovered. If you have experience with reverse engineering or if you are looking for a fun project to start with RE (or both!), please help us investigate the following mysteries.
Reverse engineering tips
This section needs expansion. You can help by adding to it. |
GameData flags
GameData flags can be read and set from many different systems:
- Event flows. A very convenient way to determine if a flag is being set in an event flow (or queried, passed to the executable, ...) is to grep MrCheeze/botw-tools:event for the flag name.
- Map units. LinkTags can be used to conditionally spawn map objects based on GameData flags (which are called SaveFlags in map units) and also to set them. It is extremely common for Area tags to be used for triggering LinkTags (which in turn write to GameData flags).
- Executable. Of course, the executable also checks for game data flags. Do note that the name of the flag is not always hardcoded, and may instead come from a configuration file (example: the Defeated counters for the difficulty scaling system).
Formats
The following tasks require reversing file formats. It is strongly recommended to also examine the executable to gain more reliable knowledge about formats and figure out how they are used by Nintendo exactly.
XLink databases
To make the development process more efficient, Nintendo introduced a system called SoundLink (aka SLink)[1][2] (and its counterpart for effects: EffectLink[3]) during development. Both systems were at some point unified and are now handled by a single xlink2 library which handles both the SLink and ELink systems.
Two files are used to map effect/sound names to the actual resources. In Breath of the Wild, they are called ELink2DB.belnk and SLink2DB.bslnk (found in ELink2 and SLink2 respectively).
The format has not seen a lot of research as of September 2018, despite the executable being full of debug strings that include field names and the fact that other recent Nintendo games such as Splatoon 2 ship the library with debugging symbols.
Event timelines
bfevtm files (binary event timelines) are extremely similar to bfevfl files (binary event flowcharts), but have a few significant differences that make them incompatible with the evfl library as of September 2018.
Because timelines are exclusively used by the event manager — unlike flowcharts which are also used by smaller subsystems like TipsMgr — for demos, it has proven difficult to find the code that reads timelines.
If you are willing to take up this task, please ping leoetlino for the Switch 1.5.0 IDC.
Havok packfiles and structures
Nintendo uses Havok binary packfiles for all Havok resources. Thus, it is difficult to manipulate physics files.
Fortunately the Havok library relies on reflection data to load resources. This makes it possible to parse the binaries and reconstruct a human readable representation with every single field name and even enum definitions. Reflection information has been dumped for the Switch version.
The following tasks remain to be done:
- Dump reflection data for the Wii U version. Data offsets are different since the Switch uses a different compiler, is a 64 bit little-endian platform unlike the Wii U (32-bit big endian machine).
- Write a library to manipulate binary packfiles: getting, modifying, adding and removing Havok objects from a packfile (and perhaps converting them to XML too?)
- Figure out how the compression for
hkpBvCompressedMeshShape
s works. Nintendo uses compressed mesh shapes for all static compound collision files.
Game logic
The following tasks require reversing the executable or some serious experiments to have reliable information on how the mechanics work.
One-Hit kill protection
It is well known that Link cannot die in one-hit under specific cases. What are the exact conditions for this protection to apply? Interestingly, it appears this mechanic is gone in Master Mode.
NoDeathDamageBase
(in Link's bgparamlist) might be related.
EnvPalettes and areas
The WorldMgr configuration file, normal.bwinfo, has a set of 207 environment-related settings called "env palettes". The link between palette numbers and maps/areas/shrines is currently unknown.
The Master Cycle Zero's speed
How is the motorcycle's speed configured? It's not in GeneralParamList, so it might be physics-based or even hardcoded.
amiibo drops
Full article: amiibo drops
- How scaling (weapon upgrades) works for amiibo weapons.
- amiibo weapons don't entirely follow the regular weapon scaling and bonus rules.
- What determines the drop table list that gets used?
- amiibo DropTables generally contain 8 tables: Normal, SmallHit, BigHit({Normal,Parasail,Remain}), and GreatHit({Normal,Parasail,Remain}). Parasail tables are presumably selected after the player receives the paraglider, and Remain after Link completes a divine beast; however this is pretty much only speculation. It is also currently unknown what determines which xxxxxHit table gets used.
The Blood Moon
Full article: Blood moon
How do Blood Moons work? It is now well known that scheduled Blood Moons are based on a timer, and that (surprise!) WorldMgr is the thing that is responsible for keeping track of the schedule. However, what are the exact conditions that stop this timer?
(Panic Blood Moons are well understood now.)
Resource sizes
Investigating this requires reversing the resource system (in the executable) and possibly patching it.
The resource size table (RSTB) determines how much memory should be allocated to a given resource (i.e. the size of the resource heap). Unfortunately, the sizes that are listed in there don't quite match the actual file size.
In June 2018, it was discovered that the RSTB sizes include internal memory consumption by the C++ resource classes (see Resource system for an explanation about the concepts), some of which further dynamically allocate memory from the resource heap! Taking the resource object sizes into account is easy, since they are constant. What's not so easy is determining the amount of memory that will be dynamically allocated for complex resources such as bfres and most AAMP files.
A way to guarantee RSTB sizes are large enough is to reverse each resource class, tracking calls to memory allocation functions. However, this is extremely time consuming given the sheer amount of resource classes in this game and the fact that the same thing would have to be done for both Switch and Wii U.
The current theory is that either the RSTB is machine generated by special versions of the game, or Nintendo is only calculating approximate file sizes.
A possible way to check whether the listed size is exact or approximate is to dump the memory of the game process and investigate resource heap usage.
In any case, figuring out how to calculate sizes correctly for complex resources or rewriting parts of the resource system to remove the fixed heap size limitation would be extremely useful.
Executable analysis
Everything in Breath of the Wild is implemented in the executable. Some aspects can be modified by editing assets; other things are hardcoded, meaning it's impossible to change how the game works by merely editing content files.
Analysing the executable is the only way to understand how things are implemented internally and how the game actually works. Eventually, after enough information has been collected, it becomes possible to predict how something will behave exactly and also to directly edit the game core. With a more complete understanding, more in-depth modifications can be done by hooking into functions and injecting custom code.
This is similar to the process of creating cheat codes, but different in that this relies on static analysis more and requires understanding how all the different subsystems interact.
Because of the sheer size of the executable, it is infeasible for this long-term project to be completed by a single person. If you are willing to help, please ping leoetlino for the Switch 1.5.0 IDC.
Debug leftovers
The following tasks require reversing the executable and possibly patching it to re-enable functionality.
Demo mode
Setting the ROM type to demo ROM types (found in the executable) make the game crash. What is locked behind them?
Actor debug utilities
In a similar fashion, Nintendo has also left an entire ActorDebug subsystem in all release versions, which can print information to the screen and interact with many different actor system components and things like map units.
Stage select screen
Full article: StageSelect
The release version has a stage select screen left (see Executable for more details). It should be possible to get it to work.
- ↑ BotW CEDEC 2017 talk
- ↑ 0x7101E271AE in Switch 1.5.0
- ↑ 0x7101E3790B in Switch 1.5.0