In-Game HD Video Capture using Real-Time YUYV-DXT Compression
This article describes a method of recording in-game HD videos without the large impact on frame rate as with an external video capture software.
The method used in this approach was inspired by an article about Real-Time YCoCg-DXT Compression which presented a real-time GPU compression algorithm to DXT formats.
Standard DXT texture formats aren't very suitable for compression of general images like the game frames, the higher contrast results in artifacts like color bleeding and color blocking. The article introduced YCoCg-DXT format that encodes colors to YCoCg color space (intensity and orange and green chrominance). It also contains the source code for real-time GPU compression and comparison of achieved results.
The YCoCg format is suitable for decompression on GPU, because decoding YCoCg values back to RGB only takes a few shader instructions. However, for the purpose of decoding the frame data in a video codec, a better format is a YUV-based one that allows to decode the data directly to the video surface without additional conversions. The best format for this seemed to be YUYV with 16 bits per sample, which means there's one U and V value per 2 horizontal samples.
The compression algorithm differs from the YCoCg-DXT one in the initial color space conversion to YUYV and in that it encodes 4x4 YY, U and V blocks in the way alpha component is encoded in DXT5 format.
The algorithm is as follows:
- Video frames are compressed with fragment shader to YUYV-DXT format by render to texture technique, reducing the data to 1/3 of its original size.
- The compressed textures are asynchronously read back to CPU.
- The data are continuously written to disk.
To capture the frame buffer data the application first renders to an intermediate target. The compression shader uses this as the input texture, rendering to a uint4 target with one quarter width and height of the original resolution, that is then read back to CPU memory.
The next step is decoding the captured video. To make this easy I've written a custom video codec and video format plugin for ffmpeg library. The format was named Yog (from YCoCg) as the encoding was originally in YCoCg format, changed only later to YUYV.
The game produces *.yog video files that can be directly replayed by ffplay or converted to another video format with the ffmpeg utility. They are also recognized by any video processing software that uses ffmpeg or ffplay executables or uses the avcodec and avformat dlls from the suite, such as WinFF or FFe or many others.
ResultsAfter starting the video recording in our game the frame rate drops only by a few fps, and it's still playable normally, unlike when recording for example with Fraps. Disadvantage is that this has to be integrated into the renderer path.
Quality wise the results are quite good, as it can be seen on the following screen shots:
YUYV compressed, note this is slightly lighter because of an issue in ffmpeg that has to be solved yet.
The difference, 4X amplified
Writing the video fileAs the textures with video frames are being read to CPU memory, they can be directly dumped to the output file, frame by frame. The file must begin with a simple 16-byte header that is used by the YOG handler. The format of the header follows:
|four cc code||4||char||'YOG '|
|type||2||short||1 = YCoCg|
2 = YUV planar
3 = YUYV packed
|decoded to RGB|
decoded to YUV444P
decoded to YUYV422
|fps||2||short||>0 = value/10|
<0 = -value/100
|width||2||ushort||width in pixels||multiple of 4 internally|
|height||2||ushort||height in pixels||multiple of 4 internally|
|frames||4||uint||number of frames in the file|
All values are little-endian. The number of frames has to be correct so at the end of recording it has to be updated with a correct value by seeking to offset 12 in the file and writing it.
Source code and binariesThe shader code for compression is written in Cg, here: yog.cg.
The codec is not an official part of ffmpeg, but here's the patch for recent 0.5.1 svn version of ffmpeg to build under MinGW.
Here are also the binaries for windows: ffmpeg-yog.zip. They can be used directly or to replace the ones in a front-end program using it.
They were build using the following configuration:
./configure --enable-decoder=YOG --enable-demuxer=YOG --datadir=. --target-os=mingw32 --disable-shared --enable-static --enable-memalign-hack --enable-avisynth --enable-pthreads --enable-libx264 --enable-swscale --enable-gpl --extra-ldflags="-L/usr/local/lib -Wl,-add-stdcall-alias" --extra-cflags="-I/usr/local/include -mno-cygwin -mms-bitfields"
© Outerra 2010