BYML: Difference between revisions

From ZeldaMods (Breath of the Wild)
Jump to navigation Jump to search
imported>Leoetlino
No edit summary
(link to EPD wiki for v7+)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<onlyinclude>'''BYML''' is a simplified, binary form of [[wikipedia:YAML|YAML]].</onlyinclude>
<onlyinclude>'''BYML''' is a simplified, binary form of [[wikipedia:YAML|YAML]].</onlyinclude>


Only the version 2 and 3 formats are documented in this article.
Only the version 2+ format is documented in this article. For newer versions (v7+), see [https://epd.zeldamods.org/wiki/Byaml the EPD wiki for more information].


== Structure ==
== Structure ==
Line 14: Line 14:
| align="center" | 0x00
| align="center" | 0x00
| align="center" | 2
| align="center" | 2
| String “BY” (big endian) or “YB” (little endian) in ASCII (file identifier).
| “BY” (big endian) or “YB” (little endian).
|-
|-
| align="center" | 0x02
| align="center" | 0x02
| align="center" | 2
| align="center" | 2
| Version 0x0002 in Breath of the Wild. Values of 1 and 3 are accepted as well.
| Version 2 in ''Breath of the Wild'', Versions 4 to 7 in ''Tears of the Kingdom''. 1 and 3 are also valid version numbers.
|-
|-
| align="center" | 0x04
| align="center" | 0x04
Line 30: Line 30:
| align="center" | 0x0c
| align="center" | 0x0c
| align="center" | 4
| align="center" | 4
| Offset to the root node, relative to start. May be 0 if the document is totally empty. Must be either an array node (0xc0) or a hash node (0xc1).
| Offset to the root node, relative to start. May be 0 if the document is totally empty. Must be either an array node (0xc0) or one of the hash nodes (0x20, 0x21 or 0xc1).
|}
|}


Line 40: Line 40:
! align="center" width="8%" | Identifier
! align="center" width="8%" | Identifier
! align="center" width="17%" | Type
! align="center" width="17%" | Type
! width="73%" | Description
! width="63%" | Description
! width="10%" | Version
|-
| align="center" | 0x20
| align="center" | Container
| Plain hash. Node is a mapping from 32-bit hashes to other nodes.
| 7+
|-
| align="center" | 0x21
| align="center" | Container
| Value hash. Node is a mapping from 32-bit hashes to other nodes and a second 4 byte value.
| 7+
|-
|-
| align="center" | 0xA0
| align="center" | 0xA0
| align="center" | Value (Index)
| align="center" | Value (Index)
| String. Value is an index into the string table.
| String. Value is an index into the string table.
| 2+
|-
| align="center" | 0xA1
| align="center" | Value (Special)
| Binary data.
| 4+
|-
| align="center" | 0xA2
| align="center" | Value (Special)
| File data. Node contains the binary data of another file.
| 5+
|-
|-
| align="center" | 0xC0
| align="center" | 0xC0
| align="center" | Container
| align="center" | Container
| Array. Node is an array of nodes, typically, though not necessarily, all of the same format.
| Array. Node is an array of nodes, typically, though not necessarily, all of the same format.
| 2+
|-
|-
| align="center" | 0xC1
| align="center" | 0xC1
| align="center" | Container
| align="center" | Container
| Hash. Node is a mapping from strings in the hash key table to other nodes.
| String hash. Node is a mapping from strings in the hash key table to other nodes.
| 2+
|-
|-
| align="center" | 0xC2
| align="center" | 0xC2
| align="center" | Container (Special)
| align="center" | Container (Special)
| String table. Special purpose node type only seen in the hash key table and the string table.
| String table. Special purpose node type only seen in the hash key table and the string table.
| 2+
|-
|-
| align="center" | 0xD0
| align="center" | 0xD0
| align="center" | Value
| align="center" | Value
| Bool. Node is 1 or 0 representing true or false respectively.
| Bool. Node is 1 or 0 representing true or false respectively.
| 2+
|-
|-
| align="center" | 0xD1
| align="center" | 0xD1
| align="center" | Value
| align="center" | Value
| Int. Node is a signed 32 bit integer value.
| Int. Node is a signed 32 bit integer value.
| 2+
|-
|-
| align="center" | 0xD2
| align="center" | 0xD2
| align="center" | Value
| align="center" | Value
| Float. Node is a binary32 floating-point number.
| Float. Node is a binary32 floating-point number.
| 2+
|-
|-
| align="center" | 0xD3
| align="center" | 0xD3
| align="center" | Value
| align="center" | Value
| UInt. Node is an unsigned 32 bit integer value. The game uses this for some CRC32 hashes and for masks.
| UInt. Node is an unsigned 32 bit integer value. The game uses this for some CRC32 hashes and for masks.
| 2+
|-
|-
| align="center" | 0xD4
| align="center" | 0xD4
| align="center" | Value (Special)
| align="center" | Value (Special)
| Int64. Node is a 64 bit integer value. Not seen in Breath of the Wild.
| Int64. Node is a 64 bit integer value.
| 3+
|-
|-
| align="center" | 0xD5
| align="center" | 0xD5
| align="center" | Value (Special)
| align="center" | Value (Special)
| UInt64. Node is an unsigned 64 bit integer value. Not seen in Breath of the Wild.
| UInt64. Node is an unsigned 64 bit integer value.
| 3+
|-
|-
| align="center" | 0xD6
| align="center" | 0xD6
| align="center" | Value (Special)
| align="center" | Value (Special)
| Double. Node is a binary64 floating-point number. Not seen in Breath of the Wild.
| Double. Node is a binary64 floating-point number.
| 3+
|-
|-
| align="center" | 0xFF
| align="center" | 0xFF
| align="center" | Value
| align="center" | Value
| Null. Value is always 0. Not seen in Breath of the Wild.
| Null. Value is always 0.
| 3+
|}
|}


Line 97: Line 130:
For string nodes, this encoding is simply a four byte index into the string table respectively.
For string nodes, this encoding is simply a four byte index into the string table respectively.


For special value nodes, the value is an offset to a 64 bit integer / floating-point number relative to the start of the file.
For special value nodes, the value is an offset relative to the start of the file.
 
==== 0x20 - Hash Node ====
 
Hash / dictionary nodes are used to encode name value collections. Entries must sorted by their hash value.
 
{| class="wikitable"
! align="center" | Offset
! align="center" | Size
! Description
|-
| align="center" | 0x00
| align="center" | 1
| 0x20 node type.
|-
| align="center" | 0x01
| align="center" | 3
| Number of entries in dictionary.
|-
| align="center" | 0x04
| align="center" | 8*N
| Dictionary entries.
|-
| align="center" | 0x04+8*N
| align="center" | 1*N
| Array of N types, on for each entry in the dictionary. This is padded to 4 bytes.
|}
 
Each entry in the dictionary has the following structure.
 
{| class="wikitable"
! align="center" width="5%" | Offset
! align="center" width="3%" | Size
! width="90%" | Description
|-
| align="center" | 0x00
| align="center" | 4
| Hash. A 32-bit hash of the key.
|-
| align="center" | 0x04
| align="center" | 4
| Value. For regular value nodes, this is the 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.
|}
 
==== 0x21 - Value Hash Node ====
 
This type is the similar to the hash node with the addition of a 4-byte value in each dictionary entry.
 
{| class="wikitable"
! align="center" width="5%" | Offset
! align="center" width="3%" | Size
! width="90%" | Description
|-
| align="center" | 0x00
| align="center" | 4
| Value. For regular value nodes, this is the 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.
|-
| align="center" | 0x04
| align="center" | 4
| Hash. A 32-bit hash of the key.
|-
| align="center" | 0x08
| align="center" | 4
| Unknown. Always 0 in ''Tears of the Kingdom''.
|}
 
==== 0xA1 - Binary Data Node ====
{| class="wikitable"
! align="center" | Offset
! align="center" | Size
! Description
|-
| align="center" | 0x00
| align="center" | 4
| Data length
|-
| align="center" | 0x04
| align="center" | variable
| Data
|}
 
==== 0xA2 - File Node ====
{| class="wikitable"
! align="center" | Offset
! align="center" | Size
! Description
|-
| align="center" | 0x00
| align="center" | 4
| File length
|-
| align="center" | 0x04
| align="center" | 4
| Unknown. Always 0x1000.
|-
| align="center" | 0x08
| align="center" | variable
| File data
|}


==== 0xC0 - Array Node ====
==== 0xC0 - Array Node ====
Line 125: Line 256:
N’ is N rounded up to the nearest multiple of 4.
N’ is N rounded up to the nearest multiple of 4.


==== 0xC1 - Hash Node ====
==== 0xC1 - String Hash Node ====


Hash / dictionary nodes are used to encode name value collections. Entries must be lexicographically sorted<ref>Nintendo's BYML library performs a binary search when looking up items by key in a hash node.</ref>.
Hash / dictionary nodes are used to encode name value collections. Entries must be lexicographically sorted<ref>Nintendo's BYML library performs a binary search when looking up items by key in a hash node.</ref>.
Line 197: Line 328:
| ''Breath of the Wild'' (Switch) || Little endian || 2 || Uses SMO's library (in al::), but with unused code and support for big endian eliminated
| ''Breath of the Wild'' (Switch) || Little endian || 2 || Uses SMO's library (in al::), but with unused code and support for big endian eliminated
|-
|-
| ''Super Mario Odyssey'' (Switch) || Little endian (supports big endian as well) || 2 or 3 ||
| ''Super Mario Odyssey'' (Switch) || Little endian (supports big endian as well) || 3 ||
|-
|-
| ''Splatoon 2'' (Switch) || Little endian (supports big endian as well) || 2 or 3 || Uses SMO's library, but in the Lp::Utl:: namespace
| ''Splatoon 2'' (Switch) || Little endian (supports big endian as well) || 3? || Uses SMO's library, but in the Lp::Utl:: namespace
|-
| ''Animal Crossing: New Horizons'' (Switch) || Little endian (supports big endian as well) || 4 ||
|}
|}


The v3 format seems to be identical to v2 but with the 64-bit node types added.
The v3 format seems to be identical to v2 but with the 64-bit node types added.
=== Extensions ===
Nintendo uses several file extensions for BYML files in BotW. The basic format is the same even though each kind has a different purpose and contains different parameters. Known extensions:
* [[Baischedule|.baischedule]]
* [[Baniminfo|.baniminfo]]
* [[Bgdata|.bgdata]]
* [[Bgsvdata|.bgsvdata]]
* [[Bquestpack|.bquestpack]]
* [[BYML|.byml]]
* [[BYML|.byaml]]
* [[Mubin|.mubin]]
TotK has added the following extensions:
* .bgyml


== Tools ==
== Tools ==

Latest revision as of 18:47, 7 July 2024

BYML is a simplified, binary form of YAML.

Only the version 2+ format is documented in this article. For newer versions (v7+), see the EPD wiki for more information.

Structure

The file is broken up into a node structure, with possible interlinking between the nodes. Each node has a one byte format identifier. The File begins with a header which points to three special nodes; the hash key table; the string table node and the root node.

Header

Offset Size Description
0x00 2 “BY” (big endian) or “YB” (little endian).
0x02 2 Version 2 in Breath of the Wild, Versions 4 to 7 in Tears of the Kingdom. 1 and 3 are also valid version numbers.
0x04 4 Offset to the hash key table, relative to start (usually 0x010). May be 0 if no hash nodes are used. Must be a string value node (0xc2).
0x08 4 Offset to the string table, relative to start. May be 0 if no strings are used. Must be a string value node (0xc2).
0x0c 4 Offset to the root node, relative to start. May be 0 if the document is totally empty. Must be either an array node (0xc0) or one of the hash nodes (0x20, 0x21 or 0xc1).

Nodes

Every node format has a unique one byte identifier as follows. Some nodes are considered value nodes as indicated below. The container nodes have a longer encoding in the file, which must be four byte aligned. The order of encoding full nodes within the file does not seem to matter.

Identifier Type Description Version
0x20 Container Plain hash. Node is a mapping from 32-bit hashes to other nodes. 7+
0x21 Container Value hash. Node is a mapping from 32-bit hashes to other nodes and a second 4 byte value. 7+
0xA0 Value (Index) String. Value is an index into the string table. 2+
0xA1 Value (Special) Binary data. 4+
0xA2 Value (Special) File data. Node contains the binary data of another file. 5+
0xC0 Container Array. Node is an array of nodes, typically, though not necessarily, all of the same format. 2+
0xC1 Container String hash. Node is a mapping from strings in the hash key table to other nodes. 2+
0xC2 Container (Special) String table. Special purpose node type only seen in the hash key table and the string table. 2+
0xD0 Value Bool. Node is 1 or 0 representing true or false respectively. 2+
0xD1 Value Int. Node is a signed 32 bit integer value. 2+
0xD2 Value Float. Node is a binary32 floating-point number. 2+
0xD3 Value UInt. Node is an unsigned 32 bit integer value. The game uses this for some CRC32 hashes and for masks. 2+
0xD4 Value (Special) Int64. Node is a 64 bit integer value. 3+
0xD5 Value (Special) UInt64. Node is an unsigned 64 bit integer value. 3+
0xD6 Value (Special) Double. Node is a binary64 floating-point number. 3+
0xFF Value Null. Value is always 0. 3+

Value Nodes

Value nodes can only be encoded as children of container nodes. Each value node has a direct four byte encoding.

For string nodes, this encoding is simply a four byte index into the string table respectively.

For special value nodes, the value is an offset relative to the start of the file.

0x20 - Hash Node

Hash / dictionary nodes are used to encode name value collections. Entries must sorted by their hash value.

Offset Size Description
0x00 1 0x20 node type.
0x01 3 Number of entries in dictionary.
0x04 8*N Dictionary entries.
0x04+8*N 1*N Array of N types, on for each entry in the dictionary. This is padded to 4 bytes.

Each entry in the dictionary has the following structure.

Offset Size Description
0x00 4 Hash. A 32-bit hash of the key.
0x04 4 Value. For regular value nodes, this is the 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.

0x21 - Value Hash Node

This type is the similar to the hash node with the addition of a 4-byte value in each dictionary entry.

Offset Size Description
0x00 4 Value. For regular value nodes, this is the 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.
0x04 4 Hash. A 32-bit hash of the key.
0x08 4 Unknown. Always 0 in Tears of the Kingdom.

0xA1 - Binary Data Node

Offset Size Description
0x00 4 Data length
0x04 variable Data

0xA2 - File Node

Offset Size Description
0x00 4 File length
0x04 4 Unknown. Always 0x1000.
0x08 variable File data

0xC0 - Array Node

Offset Size Description
0x00 1 0xC0 node type.
0x01 3 Number of entries in array.
0x04 N Array of N node types; the type of each element in the array.
0x04 + N’ 4*N Array of N node values. For regular value nodes, this is a 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.

N’ is N rounded up to the nearest multiple of 4.

0xC1 - String Hash Node

Hash / dictionary nodes are used to encode name value collections. Entries must be lexicographically sorted[1].

Offset Size Description
0x00 1 0xC1 node type.
0x01 3 Number of entries in dictionary.

What then follows is a variable length array of dictionary entries. Each entry has the following structure.

Offset Size Description
0x00 3 Name. Value is an index in the hash key table.
0x03 1 The node type.
0x04 4 Value. For regular value nodes, this is the 4 byte node value. For other nodes, this is a 4 byte offset to the node relative to the start of the file.

0xC2 - String Table Node

Offset Size Description
0x00 1 0xC2 node type.
0x01 3 Number of entries in the string table.
0x04 Y=4*(N+1) Array of N+1 offsets to each string relative to the start of the node. The last entry is an offset to the end of the last string.
0x04+Y’ variable Array of N null-terminated strings stored in alphabetical order.

Y’ is Y rounded up to the nearest multiple of 4.

Usage in Nintendo games

Game Endianness Version Notes
Breath of the Wild (Wii U) Big endian 2 Uses SMO's library (in al::)
Breath of the Wild (Switch) Little endian 2 Uses SMO's library (in al::), but with unused code and support for big endian eliminated
Super Mario Odyssey (Switch) Little endian (supports big endian as well) 3
Splatoon 2 (Switch) Little endian (supports big endian as well) 3? Uses SMO's library, but in the Lp::Utl:: namespace
Animal Crossing: New Horizons (Switch) Little endian (supports big endian as well) 4

The v3 format seems to be identical to v2 but with the 64-bit node types added.

Extensions

Nintendo uses several file extensions for BYML files in BotW. The basic format is the same even though each kind has a different purpose and contains different parameters. Known extensions:

TotK has added the following extensions:

  • .bgyml

Tools

Tool Cross-platform Setup Known issues
byml-v2 Yes

Install the byml package. [CLI help]

None

ninten-file-tool Yes
  1. Install the Java runtime environment (JRE).
  2. Download the latest release from GitHub.
  • Wii U only: Only works with big endian files and unable to handle Switch (little endian) documents.
  • Incomplete support for newer node types: Newer node types (0xd4, 0xd5, 0xd6, 0xff) are not supported. In practice, this is not an issue for Breath of the Wild as these are not known to be used in the game.
Byaml Editor Yes
Wild Bits Yes
  1. Ensure you have a 64 bit version of Python 3.7+ before continuing. Windows users must also install the latest x64 Visual C++ redistributable. You will not be able to install or launch Wild Bits otherwise. See Help:Setting_up_tools.
  2. (Optional) For best results on Windows, install cefpython3. [CLI help]
  3. Install the wildbits package. [CLI help]

On some system configurations, launching with wildbits doesn't work and you must use python -m wildbits instead.

Switch Toolbox No
  • Simply download the latest compiled release from GitHub, unzip, and launch the exe. Note that the latest release has an old date, but it is current, as it automatically rebuilds with each commit.

This article was adapted from documentation in the handsomematt/botw-modding repo.

  1. Nintendo's BYML library performs a binary search when looking up items by key in a hash node.