SARC: Difference between revisions
imported>Leoetlino mNo edit summary |
m (→Hash algorithm) |
||
(16 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
'''SARC'''s are | <onlyinclude>'''SARC'''s are archive files.</onlyinclude> | ||
== Structure == | == Structure == | ||
The basic structure of SARC files is documented on the [[mk8:SARC (File Format)|MK8 wiki]]. | The basic structure of SARC files is documented on the [[mk8:SARC (File Format)|MK8 wiki]]. | ||
=== Hash algorithm === | |||
Nintendo's algorithm uses an unsigned 32-bit integer for the hash variable. Also note that Nintendo iterates over each byte, not each string character. | |||
Here is an accurate implementation of the algorithm, written in C++: | |||
<source lang="c++"> | |||
// key is equal to 0x65 | |||
u32 calcHash32(const sead::SafeString& str, u32 key) | |||
{ | |||
const char* str_ = str.cstr(); | |||
u32 result = 0; | |||
// Each character must be treated as a signed value. | |||
// The cast to s8 (not s32) is necessary to avoid unsigned conversions. | |||
for (s32 i = 0; str_[i] != '\0'; i++) | |||
result = result * key + s8(str_[i]); | |||
return result; | |||
} | |||
</source> | |||
Note that in older versions of sead, characters were not explicitly casted to <code>s8</code> so the end result would depend on the signedness of <code>char</code> (which is implementation-defined). For GHS compiled code on the Wii U, <code>char</code> is unsigned whereas it's signed with Clang (AArch64 target). PC tools most likely use a signed <code>char</code> as that is the default signedness for x86 targets. | |||
In newer versions of sead, an explicit cast to <code>signed char</code>, aka <code>std::int8_t</code>, aka <code>s8</code>, was added to ensure hashes are the same regardless of the target platform. Without the cast, non-ASCII characters (e.g. Japanese UTF-8 bytes) would result in hashes being different across platforms. | |||
== Usage in ''Breath of the Wild'' == | == Usage in ''Breath of the Wild'' == | ||
Line 11: | Line 36: | ||
The following extensions are specific to the game: | The following extensions are specific to the game: | ||
* pack | * [[pack]] | ||
* bactorpack | * [[bactorpack]] | ||
* bmodelsh | * bmodelsh | ||
* beventpack | * [[beventpack]] | ||
* stera | * stera | ||
* stats | * stats | ||
Line 27: | Line 52: | ||
Some files have specific alignment requirements (e.g. for GPU data). Because Nintendo's SARC library returns file data by giving pointers to the data section directly, special care must be taken to pack files in a way that satisfies all alignment requirements. | Some files have specific alignment requirements (e.g. for GPU data). Because Nintendo's SARC library returns file data by giving pointers to the data section directly, special care must be taken to pack files in a way that satisfies all alignment requirements. | ||
Nintendo libraries do not use | Nintendo libraries do not use BotW's resource system and expect files to be properly aligned. This is the case for layout archives (blarc) and agl environment files (Pack/Bootup.pack/Env/env.genvb). | ||
However unlike most other Nintendo games, for files that are managed by the game's resource system, aligning archive file data is usually unnecessary because the system will automatically allocate an aligned buffer and copy the archive data into it. | However unlike most other Nintendo games, for files that ''are'' managed by the game's resource system, aligning archive file data is usually unnecessary because the system will automatically allocate an aligned buffer and copy the archive data into it. | ||
== Tools == | == Tools == | ||
Because of the alignment problem and file size limitations due to ''Breath of the Wild'''s [[Resource system|resource system]], only the following tools are recommended: | Because of the alignment problem and file size limitations due to ''Breath of the Wild'''s [[Resource system|resource system]], only the following tools are recommended: | ||
{{tool table|category=Tools (SARC)|notcategory=Unrecommended tools}} | |||
'''We strongly advise against using other tools because they do not handle file alignment properly.''' | |||
[[Category:File formats]] | [[Category:File formats]] |
Latest revision as of 23:45, 13 January 2021
SARCs are archive files.
Structure
The basic structure of SARC files is documented on the MK8 wiki.
Hash algorithm
Nintendo's algorithm uses an unsigned 32-bit integer for the hash variable. Also note that Nintendo iterates over each byte, not each string character.
Here is an accurate implementation of the algorithm, written in C++:
// key is equal to 0x65
u32 calcHash32(const sead::SafeString& str, u32 key)
{
const char* str_ = str.cstr();
u32 result = 0;
// Each character must be treated as a signed value.
// The cast to s8 (not s32) is necessary to avoid unsigned conversions.
for (s32 i = 0; str_[i] != '\0'; i++)
result = result * key + s8(str_[i]);
return result;
}
Note that in older versions of sead, characters were not explicitly casted to s8
so the end result would depend on the signedness of char
(which is implementation-defined). For GHS compiled code on the Wii U, char
is unsigned whereas it's signed with Clang (AArch64 target). PC tools most likely use a signed char
as that is the default signedness for x86 targets.
In newer versions of sead, an explicit cast to signed char
, aka std::int8_t
, aka s8
, was added to ensure hashes are the same regardless of the target platform. Without the cast, non-ASCII characters (e.g. Japanese UTF-8 bytes) would result in hashes being different across platforms.
Usage in Breath of the Wild
SARCs are used extensively in Breath of the Wild to keep related data in memory and minimise loading times.
Extensions
Archives have a wide range of extensions. However, the file format is completely the same regardless of the extension.
The following extensions are specific to the game:
- pack
- bactorpack
- bmodelsh
- beventpack
- stera
- stats
The following extensions are used by Nintendo libraries that are included in the game (non exhaustive list):
- sarc
- bgenv
- genvb
- blarc
Data alignment
Some files have specific alignment requirements (e.g. for GPU data). Because Nintendo's SARC library returns file data by giving pointers to the data section directly, special care must be taken to pack files in a way that satisfies all alignment requirements.
Nintendo libraries do not use BotW's resource system and expect files to be properly aligned. This is the case for layout archives (blarc) and agl environment files (Pack/Bootup.pack/Env/env.genvb).
However unlike most other Nintendo games, for files that are managed by the game's resource system, aligning archive file data is usually unnecessary because the system will automatically allocate an aligned buffer and copy the archive data into it.
Tools
Because of the alignment problem and file size limitations due to Breath of the Wild's resource system, only the following tools are recommended:
Tool | Cross-platform | Setup | Known issues |
---|---|---|---|
SARC Tool | Yes |
|
|
sarc | Yes |
Install the |
None |
BotW Unpacker | Yes |
Download the latest release from GitHub. |
|
Wild Bits | Yes |
|
On some system configurations, launching with |
Switch Toolbox | No |
|
We strongly advise against using other tools because they do not handle file alignment properly.