Havok: Difference between revisions

Revision as of 22:01, 19 February 2019

Havok is a middleware software suite developed by the Irish company Havok. Havok provides a physics engine component and related functions to video games.

On September 14, 2007, Intel announced it had signed a definitive agreement to acquire Havok Inc.^[1] In 2008, Havok was honored at the 59th Annual Technology & Engineering Emmy Awards for advancing the development of physics engines in electronic entertainment. On October 2, 2015, Microsoft announced it had acquired Havok.^[2]

Products

The Havok middleware suite consists of the following modules:^[3]

Havok Physics: It is designed primarily for video games, and allows for real-time collision and dynamics of rigid bodies in three dimensions. It provides multiple types of dynamic constraints between rigid bodies (e.g. for ragdoll physics), and has a highly optimized collision detection library. By using dynamical simulation, Havok Physics allows for more realistic virtual worlds in games. The company was developing a specialized version of Havok Physics called Havok FX that made use of ATI and NVIDIA GPUs for physics simulations;^[4] however, the goal of GPU acceleration did not materialize until several years later.^[5]
Havok AI: In 2009, Havok released Havok AI, which provides advanced pathfinding capabilities for games. Havok AI provides navigation mesh generation, pathfinding and path following for video game environments.
Havok Cloth: Released in 2008, Havok Cloth deals with efficient simulation of character garments and soft body dynamics.
Havok Destruction: Also released in 2008, Havok Destruction provides tools for creation of destructible and deformable rigid body environments.
Havok Animation Studio (discontinued): Havok Animation Studio is formally known as Havok Behavior and Havok Animation. Havok Behavior is a runtime SDK for controlling game character animation at a high level using finite state machines. Havok Animation provides efficient playback and compression of character animations in games, and features such as inverse kinematics.
Havok Script (discontinued): Havok Script is a Lua-compatible virtual machine designed for video game development. It is shipped as part of the Havok Script Studio.
Havok Vision Engine (discontinued): On August 8, 2011, Havok announced their acquisition of German game engine development company Trinigy and their Vision Engine and toolset.

Usage

Breath of the Wild uses the Havok Physics, Havok AI and Havok Cloth.

File Extensions
Extension	Product	Description	Locations
hknm2	Havok AI	Navigation meshes (used by AI for path finding)	`NavMesh/`
hkrb	Havok Physics	Rigid body physics	`Physics/RigidBody/`
hkcl	Havok Cloth	Cloth physics	`Physics/Cloth/`
hkrg	Havok Physics	Rag doll physics	`Physics/Ragdoll/`
hksc	Havok Physics	Static compound physics	`Physics/StaticCompound/`
hktmrb	Havok Physics	Terrain mesh rigid body physics	`Physics/TeraMeshRigidBody/`

File Types

hknm2

hkrb

Used in actors.

hkcl

hkrg

hksc

Used in shrines and the field sections.

hktmrb

Havok File Specification

Havok files serve as a data container for multiple Havok classes per file and sometimes multiple Havok files per file^[6]. The class names are defined in each file.

Note that this specification only applies to version 2014.2.0-r1

File Structure

File structure

Each file contains a header and three sections: classnames, types and data

classnames

The classnames section (__classnames__ internally) contains a list of Havok classes used by the file to correctly decompile the binary data into a usable data structure.

types

The types section (__types__ internally) does not appear to be used by Breath of the Wild.

data

the data section (__data__ internally) contains binary data to be serialized into Havok data structures for use by the Havok engine.

Header

The header seems to have a 16-byte alignment and can be 64-bytes (short header) or 80-bytes (long header) in length, depending on if Unknown 13 equals 0 (short header) or 16 (long header).

Header Specification
Offset (h)	Size	Data Type	Description
`0x00`	8	Bytes	Havok file signature (magic) `57 E0 E0 57 10 C0 C0 10`
`0x08`	4	Unsigned Int	Unknown 00 (always 0; possibly padding)
`0x0c`	4	Unsigned Int	Unknown 01 (always 11)
`0x10`	1	Byte	Unknown 02 (always 4)
`0x11`	1	Byte	Unknown 03 (always 0)
`0x12`	1	Byte	Unknown 04 (always 0)
`0x13`	1	Byte	Unknown 05 (always 1)
`0x14`	4	Unsigned Int	Unknown 06 (always 3)
`0x18`	4	Unsigned Int	Unknown 07 (always 2)
`0x1c`	4	Unsigned Int	Unknown 08 (always 0)
`0x20`	4	Unsigned Int	Unknown 09 (always 0)
`0x24`	4	Unsigned Int	Unknown 10 (always 75)
`0x28`	16	String	Havok version `hk_2014.2.0-r1` for BotW files
`0x38`	4	Unsigned Int	Unknown 11 (always 0; possibly padding)
`0x3c`	2	Short	Unknown 12 (always 21)
`0x3e`	2	Short	Unknown 13 (0 or 16; if 16, the header is 16 bytes longer)

Optional Header Bytes

There are optional header bytes for a long header. If Unknown 13 equals 16, an additional 16 bytes are added to the header for a total of 80 bytes.

Optional Header Bytes
Offset (h)	Size	Data Type	Description
0x40	2	Short	Unknown 14 (always 20)
0x42	2	Short	Unknown 15 (always 0; possibly padding)
0x44	4	Int	Unknown 16 (always 0; probably padding for alignment)
0x48	4	Int	Unknown 17 (always 0; probably padding for alignment)
0x4c	4	Int	Unknown 18 (always 0; probably padding for alignment)

Section Headers

FF bytes indicate section header boundaries

Each section has a 64-byte header which includes the section name, absolute offset to the section start and six relative offsets to segments within the section. All three section headers follow the file header and are delimited by 16 delimiter bytes (0xFF).

Section Header Specification
Offset (h)	Size	Data Type	Description
`0x00`	20	String	Section name
`0x14`	4	Unsigned Int	Absolute offset
`0x18`	4	Unsigned Int	Relative offset 1
`0x1c`	4	Unsigned Int	Relative offset 2
`0x20`	4	Unsigned Int	Relative offset 3
`0x24`	4	Unsigned Int	Relative offset 4
`0x28`	4	Unsigned Int	Relative offset 5
`0x2c`	4	Unsigned Int	Relative offset 6

Section Name

There are only three section names: __classnames__, __types__ and __data__. The section name is a null-terminated string with a delimiter byte (0xFF) in byte 20.

Absolute Offset

The absolute offset to the section.

Relative Offsets

The relative offsets point to different large structures within the section.

For example, given the following section header:

5F 5F 64 61 74 61 5F 5F 00 00 00 00 00 00 00 00 | __data__........
00 00 00 FF 00 00 01 60 00 00 3C 00 00 00 3C 10 | .......`..<...<.
00 00 3C 10 00 00 3C 20 00 00 3C 20 00 00 3C 20 | ..<...< ..< ..< 
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF | ................

The section name is __data__.

The absolute offset is the beginning of the data section, right after the class names.

This following tables shows to which sections the offsets are pointing:

Data Header Specification
Offset (h)	Size	Data Type	Description / Section
`0x00`	20	String	Section name
`0x14`	4	Unsigned Int	Beginning of data section
`0x18`	4	Unsigned Int	Pointer - Data Pointer
`0x1c`	4	Unsigned Int	Pointer - Linked Entries
`0x20`	4	Unsigned Int	Pointer - Class Mapping
`0x24`	4	Unsigned Int	EOF
`0x28`	4	Unsigned Int	EOF
`0x2c`	4	Unsigned Int	EOF

Class Names Section

Class names binary data

The classnames section header includes the absolute offset to the class names data and a single unique relative offset to the end of the section. The section is made up of an array of hash ids and a null-terminated string class name.

Class Name

Class names are read until the end of the section or a delimiter byte (0xFF) is is reached.

Class Name Specification
Offset (h)	Size	Type	Description
0x00	4	Unsigned Int	An integer associated with the class type
0x05	1	Byte	Unknown (always `0x09`)^[7]
0x06	n	String	Class name (null-terminated string)

Types Section

The types section is unused in Breath of the Wild. The absolute offset points to the beginning of the data section with all null (0x00) relative offsets.

Data Section

Data structure

Data

The bulk of the file is devoted to the data section which is divided into smaller segments dependent on the file type. All files follow a general structure as you can see in the picture.

Right after the header begins the data section which contains multiple chunks, luckily, the four first chunks are always the same.

After that, the several more chunks can be found depending on the file.

Pointer

The next section, at the end of the file, contains three different pointer arrays. It is used to connect the whole file together, and should be used as an entry point while reading the file.

The offsets to these three sections can be found in the header.

Data Pointer

An array of offsets relative to the beginning of the data section, pointing into various chunks.

Which offsets ends up in this list, depends completely on the chunks used in the data section.

However, since the first four classes are always the same, the first 14 offsets will be the constant.

Data Pointer

Linked Entries

Contains pointer-pairs to connect chunks together.

Each pair has the following format:

Pointer Pair
Offset (h)	Size	Type	Name	Description
0x00	4	Unsigned Int	base	Offset somewhere in the parent chunk
0x04	4	Unsigned Int	flags	Unknown (always `0x00000002)`
0x08	4	Unsigned Int	target	Target chunk (always beginning of the chunk)

To connect a chunk with another, a base pointer somewhere inside the base-chunk (depending on the class/type) is used as the first pointer.

Then, the target pointer is the offset of the beginning of the target chunk.

Linked Entries

Class Mapping

The last section contains the mapping between the chunks and the classes they are using.

They, again, use pointer pairs:

Class Mapping Pointer
Offset (h)	Size	Type	Name	Description
0x00	4	Unsigned Int	data	Offset to the start of an chunk, relative to data-section
0x04	4	Unsigned Int	flags	Unknown (always `0x00000000)`
0x08	4	Unsigned Int	class	Offset to the class name, relative to class-section

To link a chunk to a class, the data pointer is set to the beginning of the chunk.

The class pointer is set to the first character of the class name found in the header section.

Class Mapping

This also marks the end of the file, if the offset is not aligned to 16 bytes, the rest is filled with 0xFF.

Data Classes

All known class that chunks can used are listed here:

↑ Intel To Acquire Havok
↑ Havok to join Microsoft
↑ Product Overview
↑ Havok Intros Havok FX Engine to Compute Physics Effects on GPUs
↑ Havok Physics Playstation 4 Demo
↑ Static Compound files (hksc) include two Havok file definitions. The first file defines StaticCompoundInfo and the second defines hkpPhysicsData (which includes hkpRigidBody, hkpStaticCompoundShape, hkpBvCompressedMeshShape, hkpConvexVerticesShape, et. al.).
↑ It is assumed to be a delimiter between the id and the class name string. However, 0x09 is also the tab character and could imply something to the engine.

[1] Intel To Acquire Havok

[2] Havok to join Microsoft

[3] Product Overview

[4] Havok Intros Havok FX Engine to Compute Physics Effects on GPUs

[5] Havok Physics Playstation 4 Demo

[6] Static Compound files (hksc) include two Havok file definitions. The first file defines StaticCompoundInfo and the second defines hkpPhysicsData (which includes hkpRigidBody, hkpStaticCompoundShape, hkpBvCompressedMeshShape, hkpConvexVerticesShape, et. al.).

[7] It is assumed to be a delimiter between the id and the class name string. However, 0x09 is also the tab character and could imply something to the engine.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

@@ Line 63: / Line 63: @@
 ==== hkrb ====
-{{Empty section}}
+{{Empty section}}Used in actors.
 ==== hkcl ====
@@ Line 72: / Line 72: @@
 ==== hksc ====
-{{Empty section}}
+{{Empty section}}Used in shrines and the field sections.
 ==== hktmrb ====
@@ Line 279: / Line 279: @@
 === Relative Offsets ===
-The relative offsets point to different large structures within the section. There are usually a few unique offsets and then multiple duplicated offsets. It is unknown if only unique offsets are significant at this time.
+The relative offsets point to different large structures within the section.
 For example, given the following section header:
@@ Line 287: / Line 287: @@
 00 3C 10 00 00 3C 20 00 00 3C 20 00 00 3C 20 | ..<...< ..< ..<
 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF | ................
-</syntaxhighlight>The section name is <code>__data__</code>. The absolute offset is <code>0x160</code> with the relative offsets
+</syntaxhighlight>The section name is <code>__data__</code>.
+The absolute offset is the beginning of the data section, right after the class names.
-* <code>0x3c00</code>
+This following tables shows to which sections the offsets are pointing:
-* <code>0x3c10</code>
+{| class="wikitable"
-* <code>0x3c10</code>
+|+Data Header Specification
-* <code>0x3c20</code>
+!Offset (h)
-* <code>0x3c20</code>
+!Size
-* <code>0x3c20</code>
+!Data Type
+!Description / Section
+|-
+|<code>0x00</code>
+|20
+|String
+|Section name
+|-
+|<code>0x14</code>
+|4
+|Unsigned Int
+|Beginning of data section
+|-
+|<code>0x18</code>
+|4
+|Unsigned Int
+|Pointer - Data Pointer
+|-
+|<code>0x1c</code>
+|4
+|Unsigned Int
+|Pointer - Linked Entries
+|-
+|<code>0x20</code>
+|4
+|Unsigned Int
+|Pointer - Class Mapping
+|-
+|<code>0x24</code>
+|4
+|Unsigned Int
+|EOF
+|-
+|<code>0x28</code>
+|4
+|Unsigned Int
+|EOF
+|-
+|<code>0x2c</code>
+|4
+|Unsigned Int
+|EOF
+|}
+<br />
 == Class Names Section ==
@@ Line 329: / Line 374: @@
 == Data Section ==
+[[File:Havok-data-structure.png|thumb|390x390px|Data structure]]
+=== Data ===
+The bulk of the file is devoted to the data section which is divided into smaller segments dependent on the file type.
+All files follow a general structure as you can see in the picture.
+Right after the header begins the data section which contains multiple chunks, luckily, the four first chunks are always the same.
+After that, the several more chunks can be found depending on the file.
+=== Pointer ===
+The next section, at the end of the file, contains three different pointer arrays.
+It is used to connect the whole file together, and should be used as an entry point while reading the file.
+The offsets to these three sections can be found in the header.
+==== Data Pointer ====
+An array of offsets relative to the beginning of the data section, pointing into various chunks.
+Which offsets ends up in this list, depends completely on the chunks used in the data section.
+However, since the first four classes are always the same, the first 14 offsets will be the constant.
+[[File:Havok data pointer.png|none|thumb|Data Pointer]]
+<br />
+==== Linked Entries ====
+Contains pointer-pairs to connect chunks together.
+Each pair has the following format:
+{| class="wikitable"
+|+Pointer Pair
+!Offset (h)
+!Size
+!Type
+!Name
+!Description
+|-
+|0x00
+|4
+|Unsigned Int
+|base
+|Offset somewhere in the parent chunk
+|-
+|0x04
+|4
+|Unsigned Int
+|flags
+|Unknown (always <code>0x00000002)</code>
+|-
+|0x08
+|4
+|Unsigned Int
+|target
+|Target chunk (always beginning of the chunk)
+|}
+To connect a chunk with another, a base pointer somewhere inside the base-chunk (depending on the class/type) is used as the first pointer.
+Then, the target pointer is the offset of the beginning of the target chunk.
+[[File:Havok pointer linked.png|none|thumb|Linked Entries]]
+<br />
+==== Class Mapping ====
+The last section contains the mapping between the chunks and the classes they are using.
+They, again, use pointer pairs:
+{| class="wikitable"
+|+Class Mapping Pointer
+!Offset (h)
+!Size
+!Type
+!Name
+!Description
+|-
+|0x00
+|4
+|Unsigned Int
+|data
+|Offset to the start of an chunk, relative to data-section
+|-
+|0x04
+|4
+|Unsigned Int
+|flags
+|Unknown (always <code>0x00000000)</code>
+|-
+|0x08
+|4
+|Unsigned Int
+|class
+|Offset to the class name, relative to class-section
+|}
+To link a chunk to a class, the data pointer is set to the beginning of the chunk.
+The class pointer is set to the first character of the class name found in the header section.
+[[File:Havok class mapping.png|none|thumb|Class Mapping]]
+This also marks the end of the file, if the offset is not aligned to 16 bytes, the rest is filled with <code>0xFF</code>.
+== Data Classes ==
 {{Expand section}}
-The bulk of the file is devoted to the data section which is divided into smaller segments dependent on the file type.
+All known class that chunks can used are listed here:
+<br />
 <references />
 [[Category:File formats]]

Havok: Difference between revisions

Revision as of 22:01, 19 February 2019

Contents

Products