Data Streams
These are the fundamental source of bytes. Understand how the API is designed here.
The data stream class is an abstract class found here. It has many sub-classes which are all different implementations of a byte source.
The main ones are as follows:
File (aka disk),
Deferred,
Memory,
Buffer,
SubStream,
LegacyEncrypted,
Container,
Null,
Append,
Sequential.
As you can see, there are quite a few. They are all described roughly in the header linked above.
Most of these streams exist in Telltale's game engine too. Some of them have different meaning. Although we are guessing you don't have access to the Telltale Tool's source code anyway.
Creating streams
Since it is an abstract class, prefer to use the DataStreamRef type, which is just a managed pointer to one. The following code could create one: DataStreamRef myStream = TTE_NEW_PTR(DataStreamSubStream, MEMORY_TAG_DATA_STREAM, parent_stream, 50, 100); . This creates a substream which reads bytes 50 to 150 in the parent stream ref. The use of that macro as well as just TTE_NEW and TTE_ALLOC (for raw bytes) is common across all of the codebase. The allocations are tracked (with their memory tag) and you can detect and dump all currently non-freed pointers and memory using DumpMemoryLeaks from the global namespace.
File Stream
This is the most useful. Reads and writes from a file. File is always opened.
Example:
DataStreamRef stream = TTE_NEW_PTR(DataStreamFile, MEMORY_TAG_DATA_STREAM, ResourceURL("/relative/or/absolute/path/file.bin"));
Deferred Stream
This stream is another abstract class and you probably won't need to come across it. Streams such as container and legacy encrypted use this to read data in pages of fixed size which may have a byte mapping function (eg to decrypt and/or decompress to a variable size).
Buffer Stream
This stream is a readable and writable stream from a fixed size memory buffer specified at construction. You can pass in the optional initial buffer and whether it should be freed on destruction as well.
This stream should be created, like most others, using the data stream manager class. A lot of constructors are kept protected (with exception of file stream) but are accessible by that manager class. This requires the tool context to be present and initialised.
Example:
DataStreamRef stream = DataStreamManager::GetInstance()->CreateBufferStream("my_temp_buf.bin", 1000, optionalPreAllocatedBuffer, optionalShouldDeallocatePreAllocatedPassedInBuffer);
Memory (aka Cache) Stream
This stream is similar to the buffer stream. It is also readable and writable but the key difference is that the backend memory is not of fixed size and can expand to any size. The backing buffer page array can grow and truncate. This is most useful for writing files and then flushing this memory stream to a file after, so when writing its faster. Not recommended for big files of course.
Example:
Ptr<DataStreamMemory> stream = DataStreamManager::GetInstance()->CreatePrivateCache("my_temp_stream.bin", 1000);
Notice here we use the pointer explicitly with the type of the data stream, not just the data stream ref which is just the abstract class. This provides more functionality specific to that data stream class. Only done for streams which have functionality that isn't already exposed by the abstract data stream class.
Notice also that it is called a cache stream. You can call create public or private cache. If public, you can call FindCache in part of the data stream manager class too get the pointer back to the returned pointer. The manager keeps a reference to it, so make sure to call ReleaseCache after as well.
You can also call Publicise as well, if you previously created it with CreatePrivateCache .
Sub Stream
This stream is pretty self expanatory and is very useful and used quite a lot internally. It just specifies a stream that reads a sub-section of contiguous data of another data stream. It holds a reference to that parent stream (strong).
Example:
DataStreamRef stream = DataStreamManager::GetInstance()->CreateSubStream(parentStream, 50, 100); Which creates a sub stream which reads from the parent stream from offset 50 to 150 (100 bytes). The size of the stream will always return 100 and if you read the first byte, it will read the byte at offset 50 of the parent stream - which translates to offset 0 in the returned stream.
Append Stream
This stream is a write-only data stream which writes any incoming write calls to every single child data stream at their current offsets. Getting the size, the position or reading from this stream will throw a failed assertion. Use the explicit pointer creation.
Example:
DataStreamRef stream = TTE_NEW_PTR(DataStreamAppendStream, MEMORY_TAG_DATA_STREAM, streamRefArray, numStreams);
The stream reference array is just a pointer to an array of DataStreamRef's. Pass in the number of pointers in this array as well. The array is copied so you can construct it on the stack.
Sequential Stream
This stream is the opposite of an append stream. It is read-only and any writes will fail. You can set the position and get the position as well as add streams to it. This stream will then read from the start of the first stream to the end of the last stream. Do not add streams between reads otherwise you will get unexpected and undefined behaviour and also it might detect it and throw an assertion.
Example:
Ptr<DataStreamSequentialStream> stream = DataStreamManager::GetInstance()->CreateSequentialStream("my_temp_seq_strm.bin");
Once the stream is created, then you can call stream->PushStream(another) where another is another DataStreamRef not equal to the stream itself.
Container Stream
This stream is Telltale specific and used in quite a few places. This stream is the wrapper stream responsible for compressed and encryption in the newer set of games made in the Telltale tool. It deals with the 'TTNC/TTCe' etc magic header numbers. This stream works slightly differently. For reading, create as normal passing in the parent file stream (mostly is a file, doesn't have to be). Any reads from this stream will decompress and decrypt as required under the hood. When writing, use the FlushContainer function of the data stream manager class as well. This stream is not writable in any way and flush container should be used for that purpose. See the header file documentation for more information.
Example for reading:
DataStreamRef stream = DataStreamManager::GetInstance()->CreateContainerStream(srcStream);
Legacy Encrypted
This stream shouldn't really be used publically but is here for convenience. This deals with reading legacy encrypted meta streams which use a series of combinations of XOR pages and blowfish alternating blocks. Take a look at the source code for more information in the source files, but generally this is managed in the meta system by default and decrypted automagically if ever needed.
Null Stream
This stream is a convenience stream which always returns 0 or does nothing. Will print a warning message when trying to read or write from it.
Last updated