Meta Reflection System

Understand the inner workings and API of the C++ meta system implementation.

Here I will assume you have read the meta system page in the core documentation section.

The meta system is a C++ reflection system in which the types are all virtual. This specifically is unique to this API and is not used pretty much anywhere else because it's unneeded.

Here, it is vital. We are dealt with a problem: we have a big set of another set of classes for each game snapshot. The classes change in terms of their member variables and structure each game snapshot.

We know a set of those types don't change at all (for example BoundingBox or TRect<Float>). However pretty much all of the rest do.

Most C++ reflection APIs deal with describing a class's members layout and functionality as well as possibly virtual function table (abstraction) information. And those layouts are static at compile time and don't change, for example in each Telltale game snapshot (the executable stays the same - therefore the layout of each class, eg Mesh, does too). Here, they aren't. We have a massive abstraction that every single class is not guaranteed to be the same, or even exist!

This meta system is designed in a way that members and other information (enum descriptors, etc) are all dynamically specified and pushed at runtime in the Lua scripts. By registering a class, a reflection information record is cached internally for that class.

All of the Meta system is found in the meta header file. The meta system also holds information about each game snapshot too, as it is tied closely to the classes. Take a look at that header file and read through the struct definitions as well as classes at the top of the file. Like all headers in this library, each code file is split into sections as they tend to grow quite large.

Class IDs

A class is specifically identified by its 4 byte class ID. This class ID is a XOR of the of the class name hash as well as version hash with some bit flips. Each class name has a type name hash, the lower case hash of the type name of that class (eg 'class String' ⇒ to its lower case CRC64 (ECMA)). The version CRC is a hash function which CRC32's a buffer of data which is unique to that class. It is specific to each game snapshot. We do this because the version hash is stored in all file headers in Telltale meta streams (serialised versions of a class), along with the type name hash. But the version hash allows for global snapshot independent hash of that specific class. Although this function does change once or twice across the set of all games, it generally doesn't change much. All implementations are therefore in Lua and can be found in the script here.

The type name hash - a CRC64 (all lower case) (or sometimes just type name) as well the version hash (custom CRC32 routine) is always present for each non-intrinsic type which is serialised in a meta stream file, in the header part of the file. This is checked exactly for each class, and it must match before that file can be read (for every header entry). If not, then the class is wrong and reading will produce the incorrect results.

Version Number

In a lot of legacy games there are lots of classes with the same name (therefore type name hash) but different class layouts (members etc) due to different versions being created of that class in development of the engine. Very very rarely there are some files with different format in the same game snapshot (for example, see RTBs mesh importer script here, some of the known bugs mention different formats). Format is synonymous to class layout, set of members. This is likely when the class was defined, a file was written, but then the class was updated in the game engine implementation, but the file was not updated so its left in this old format. This should never happen but does rarely so we include this version number. It is an integer, from 0 to 9, which just specifies the degeneracy (for physics people) if you like, of that class. These are not included in the class ID at all and there can be multiple class IDs with the same version number (if their version hashes are the same, but they never are because that's the point of version numbers). All classes from the Lua API and internally are retreived from the class name and version number. By default, its zero and pretty much is all of the time. But if there is an older conflicting format, then this is present.

Implementation

All of the inner workings are in the '_Impl' namespace (synonymous with detail namespace's in C++).

Script References

Briefly gone over in the meta system page, internally script references are important to talk about.

There are three types of script references to an instance of a meta class.

• Strong,

• Weak,

• Transient.

A strong reference means that the Lua object will keep the instance alive and when it is GC'ed it will then be released. Not used much and rarely should be used unless running from a user mod script.

A weak reference is the most common. It is essentially a weak pointer, non owning. When the C++ meta instance expires, then any accesses to the Lua version with return nil.

A transient reference is a cool feature which allows for meta instanced which are owned by collections to be tracked with an internal transience counter. Collection modifications which should change the memory layout (such as pushing, popping, clearing, sorting etc) will increment the internal counter. This juncture then stops accesses after its increased past the cached value. An assertion will be fired because this is generally up to the programmer to realise, as everything is by reference.

You can push and acquire references to and from the stack using the push functions inside the meta instance class and acquire from the Meta namespace.

Functionality

Most functionality is found in the Meta namespace towards the lower middle part of the header file. You can create instances, copy and move instances as well as create an instance from a meta stream (Read a file) and write instances back to a file. You can also get members by name, and even perform operators on them.

Operators

In actual telltale engine, meta classes are not just member descriptions of the class. They include a lot of code specific to that class in relation to what essentially is operator overloading. For certain types such as string, integer (most PODs - plain old data) etc, these are defined which are used for sorting and other stuff. In this meta implementation, there is no need to implement them all (there are loads from managing load dependent resources to rendering the class in the Telltale Tool UI). At the moment, we have the equality operator (check if equal), less than operator (for sorting) and to string which is useful for debugging. The to string operator can be done for any type. If not overloaded (not possible by Lua, only for PODs), then it just results to a string representation of each member in a table.

Coersion

Coersion is used a lot in the Meta system internally. Its main functionality which is exported are Meta::CoerceMetaToLua , Meta::CoerceLuaToMeta , Meta::CoerceTypeErasedToLua , Meta::ExtractCoercableInstance , Meta::ImportCoercableInstance , Meta::ExtractCoercableLuaValue and finally Meta::ImportCoercableLuaValue . These functions deal with converting between C++ classes/structs, Lua objects (not meta instances), and normal meta instances.

Coercing from meta to Lua will push onto the stack the Lua object which best represents that class. Not all classes are coercable. See the implementation for all of this here, around the middle of the inline file for meta coersion. For the meta class 'class String', this will push onto the stack a string. For floats and integers, it will push a number, etc. For some constant classes which never change throughout Telltale's implementation, such as BoundingBox, a table is pushed with MinX MaxZ etc.

The extract and import set of functions are for coercing between C++ classes (see utility types at the top of the Meta header, as well as all PODs) and Lua and Meta instances. These are templates. The template T must be define as coercable in the meta coersion inline file.

Coercing a type erased class is the same but the without templates. Specify the class ID.

Also please note I realised that coerce is spelt wrong in the implementation but ill leave that because who cares (neeks, wait...).