| Games and SDL SDL Installation SDL for Embedded SDL API SDL Events | SDL Graphics SDL Threads Thread Example SDL Animation SDL Sound | Raw Video Player Video Formats Video Compression | Game Trees About The Author |
Video File Formats
| 1 | 2 |
In the previous Chapter, we have discussed how to play a video using data saved in the raw format of a file. In reality, video data are saved in a predefined format. There have been numerous video formats around and the data of most of them are saved in compressed form. In this Chapter, we shall give a brief discussion on some popular formats and do a case study on the .avi file format. Just as raw pixel data are often saved in .bmp format, and raw PCM sound samples in .wav format, raw video data are often saved in .avi format. We shall learn how to extract the raw video data from a .avi file so that we can play the video as we discussed in the previous Chapter or process them in our own way.
Requirements of Internet Video File Format
There exists a large number of video file formats in the market not only because competing companies create their own formats, hoping to push out competitors and to make their formats standards, but also because there are legal needs of not overstepping competitors' so called intellectual property. The following sections examine some common characteristics between the popular file formats. We could summarize the requirements for video file format to be successful as follows. A video file format should be able to
Common Internet Video Container File Format
A container file format is hierarchical in structure, and can hold different kinds of media ( audio, video, text .. ) synchronized in time. The following are some popular container formats, which can save various types of media.
Simple Raw or Stream Video Format
Simple raw storage or stream formats store the compressed data without extra headers or metadata. These are essentially live audio-video streams saved to disk. Below are some examples.
Internet Playlist, Index, and Scripting Format
Index formats have pointers linking to other resources. The following are some of this kind.
AVI ( Audio Video Interleaved ) is a file format defined by Microsoft for use in applications that capture, edit and play back audio-video sequences. Just as raw pixel data are often saved in .bmp format, and raw PCM sound samples in .wav format, raw video data are often saved in .avi format. It is a special case of RIFF (Resource Interchange File Format) and is the most commonly used format for storing audio/video data in a PC. An AVI file can be embedded in a web page using a link like:
AVI is often regarded as an obsolete video/audio file format as it lacks many contemporary and crucial features to support streaming and image processing. However, it has been extended by OpenDML to include some of those features.
RIFF FILE Format
The AVI file format is based on the RIFF (resource interchange file format) document format. A RIFF file consists of a RIFF header followed by zero or more lists and chunks; it uses a FOURCC ( four-character code ) to denote a text header. A FOURCC is a 32-bit unsigned integer created by concatenating four ASCII characters. For example, 'abcd' = 0x64636261. The AVI file format uses FOURCC codes to identify stream types, data chunks, index entries, and other information. The RIFF file format has the following form.
The following are sample data from and AVI file. The data are displayed in hexadecimal; the corresponding ASCII characters are printed on the right if they are printable otherwise a dot is printed. Some comments are shown at the far right to indicate what the data represent.
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0123456789012345]
000000000: 52 49 46 46 DC 6C 57 09 41 56 49 20 4C 49 53 54 |RIFF.lW.AVI LIST|RIFF fileSize fileType LIST
000000016: CC 41 00 00 68 64 72 6C 61 76 69 68 38 00 00 00 |.A..hdrlavih8...|listSize listType avih structureSize
000000032: 50 C3 00 00 00 B0 04 00 00 00 00 00 10 00 00 00 |P...............|microSecondPerFrame maxBytesPerSec
000000048: A8 02 00 00 00 00 00 00 01 00 00 00 00 84 03 00 |................|totalFrames initialFrames streams suggestedBufferSize
000000064: 40 01 00 00 F0 00 00 00 00 00 00 00 00 00 00 00 |@...............|width height
000000080: 00 00 00 00 00 00 00 00 4C 49 53 54 74 40 00 00 |........LISTt@..|
000000096: 73 74 72 6C 73 74 72 68 38 00 00 00 76 69 64 73 |strlstrh8...vids|
000000112: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000128: 64 00 00 00 D0 07 00 00 00 00 00 00 A8 02 00 00 |d...............|
000000144: 00 84 03 00 10 27 00 00 00 00 00 00 00 00 00 00 |.....'..........|
000000160: 40 01 F0 00 73 74 72 66 28 00 00 00 28 00 00 00 |@...strf(...(...|
000000176: 40 01 00 00 F0 00 00 00 01 00 18 00 00 00 00 00 |@...............|
000000192: 00 84 03 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000208: 00 00 00 00 69 6E 64 78 F8 3F 00 00 04 00 00 00 |....indx.?......|
000000224: 01 00 00 00 30 30 64 62 00 00 00 00 00 00 00 00 |....00db........|
000000240: 00 00 00 00 0C 44 00 00 00 00 00 00 00 40 00 00 |.....D.......@..|
.
.
000017408: 4C 49 53 54 38 F9 56 09 6D 6F 76 69 69 78 30 30 |LIST8.V.moviix00|LISTlistSize listType indexBlock ( ix00 )
000017424: F8 3F 00 00 02 00 00 01 A8 02 00 00 30 30 64 62 |.?..........00db| .... 00db ( uncompressed video frame )
.
.
|
| Table 1. Sample AVI data |
As you can see from the above sample data, a two-character code is used to define the type of information in the chunk.
| Two-character code | Description |
|---|---|
| db | Uncompressed video frame |
| dc | Compressed video frame |
| pc | Palette change |
| wb | Audio data |
AVI RIFF Format
As shown in Table 1, the FOURCC 'AVI ' in a RIFF header identifies the file to be an AVI file. An AVI file has two mandatory LIST chunks, defining the format of the streams and the stream data, respectively. An AVI file might also include an index chunk, indicating the address of the data chunks of the file; it has the following form ( Table 2 ):
RIFF ('AVI '
LIST ('hdrl' ... )
LIST ('movi' ... )
['idx1' (
|
| Table 2. AVI RIFF Format |
If we expand 'hdrl' and 'movi' in Table 2, we shall get a form shown below ( Table 3 ):
RIFF ('AVI '
LIST ('hdrl' //header length
'avih'(
|
The 'hdrl' list begins with the main AVI header, which is contained in an 'avih' chunk. The main header contains global information for the entire AVI file, such as the number of streams within the file and the width and height of the AVI sequence. This main header structure is shown below.
typedef struct _avimainheader {
FOURCC fcc; //'avih'
DWORD cb; //size of structure, not including first 8 bytes
DWORD dwMicroSecPerFrame;
DWORD dwMaxBytesPerSec;
DWORD dwPaddingGranularity;
DWORD dwFlags;
DWORD dwTotalFrames;
DWORD dwInitialFrames;
DWORD dwStreams;
DWORD dwSuggestedBufferSize;
DWORD dwWidth;
DWORD dwHeight;
DWORD dwReserved[4];
} AVIMAINHEADER;
|
One or more 'strl' lists follow the main header. A 'strl' list is required for each data stream. Each 'strl' list contains information about one stream in the file, and must contain a stream header chunk ('strh') and a stream format chunk ('strf'). In addition, a 'strl' list might contain a stream-header data chunk ('strd') and a stream name chunk ('strn'). The stream header chunk ('strh') consists of an AVISTREAMHEADER structure shown below.
typedef struct _avistreamheader {
FOURCC fcc;
DWORD cb;
FOURCC fccType; //'vids' - video, 'auds' - audio, 'txts' - subtitle
FOURCC fccHandler;
DWORD dwFlags;
WORD wPriority;
WORD wLanguage;
DWORD dwInitialFrames;
DWORD dwScale;
DWORD dwRate;
DWORD dwStart;
DWORD dwLength;
DWORD dwSuggestedBufferSize;
DWORD dwQuality;
DWORD dwSampleSize;
struct {
short int left;
short int top;
short int right;
short int bottom;
} rcFrame;
} AVISTREAMHEADER;
|
| Table 3. AVI Main Stream Header |
One can also express Digital Video ( DV ) data in the AVI file format. The following example shows the AIFF RIFF form for an AVI file with one DV data stream, expanded with completed header chunks.
00000000 RIFF (0FAE35D4) 'AVI '
0000000C LIST (00000106) 'hdrl'
00000018 avih (00000038)
dwMicroSecPerFrame : 33367
dwMaxBytesPerSec : 3728000
dwPaddingGranularity : 0
dwFlags : 0x810 HASINDEX | TRUSTCKTYPE
dwTotalFrames : 2192
dwInitialFrames : 0
dwStreams : 1
dwSuggestedBufferSize : 120000
dwWidth : 720
dwHeight : 480
dwReserved : 0x0
00000058 LIST (0000006C) 'strl'
00000064 strh (00000038)
fccType : 'iavs'
fccHandler : 'dvsd'
dwFlags : 0x0
wPriority : 0
wLanguage : 0x0 undefined
dwInitialFrames : 0
dwScale : 100 (29.970 Frames/Sec)
dwRate : 2997
dwStart : 0
dwLength : 2192
dwSuggestedBufferSize : 120000
dwQuality : 0
dwSampleSize : 0
rcFrame : 0,0,720,480
000000A4 strf (00000020)
dwDVAAuxSrc : 0x........
dwDVAAuxCtl : 0x........
dwDVAAuxSrc1 : 0x........
dwDVAAuxCtl1 : 0x........
dwDVVAuxSrc : 0x........
dwDVVAuxCtl : 0x........
dwDVReserved[2] : 0,0
000000CC LIST (0FADAC00) 'movi'
0FADACD4 idx1 (00008900)
|
In order to extract the data from an AVI file, we need to write a program that can understand the AVI format. However, rather than reinventing the wheel from the ground up, we make use of some existing open-source libraries and codes to help us do the job. We shall utilize the codes from MPEG4IP, which is an open-source package consisting of tools for streaming video and audio that is standards-oriented and free from proprietary protocols and extensions. MPEG4IP's development is focused on the Linux platform, and has been ported to Windows, Solaris, FreeBSD, BSD/OS and Mac OS X. It can handle MPEG-4, H.261, MPEG-2, H.263, MP3, AAC, and AMR file formats. When you download the package from its site and unpack it, you will find the utility program "avilib.c" and its header file "avilib.h" in the subdirectory ./lib/avi. You also need the header file "mpeg4ip.h" in the subdirectory ./include to compile "avilib.c".
We shall utilize the functions provided by "avilib.c" to read an AVI file. The main AVI structure is defined in "avilib.h" and is named avi_t. As shown below, avi_t basically covers all the crucial features of an AVI file.
typedef struct
{
long fdes; /* File descriptor of AVI file */
long mode; /* 0 for reading, 1 for writing */
long width; /* Width of a video frame */
long height; /* Height of a video frame */
double fps; /* Frames per second */
char compressor[8]; /* Type of compressor, 4 bytes + padding for 0 byte */
long video_strn; /* Video stream number */
long video_frames; /* Number of video frames */
char video_tag[4]; /* Tag of video data */
long video_pos; /* Number of next frame to be read
(if index present) */
long a_fmt; /* Audio format, see #defines below */
long a_chans; /* Audio channels, 0 for no audio */
long a_rate; /* Rate in Hz */
long a_bits; /* bits per audio sample */
long audio_strn; /* Audio stream number */
long audio_bytes; /* Total number of bytes of audio data */
long audio_chunks; /* Chunks of audio data in the file */
char audio_tag[4]; /* Tag of audio data */
long audio_posc; /* Audio position: chunk */
long audio_posb; /* Audio position: byte within chunk */
long pos; /* position in file */
long n_idx; /* number of index entries actually filled */
long max_idx; /* number of index entries actually allocated */
unsigned char (*idx)[16]; /* index entries (AVI idx1 tag) */
video_index_entry * video_index;
audio_index_entry * audio_index;
long last_pos; /* Position of last frame written */
long last_len; /* Length of last frame written */
int must_use_index; /* Flag if frames are duplicated */
long movi_start;
} avi_t;
|
To read an AVI file, we can first use the function AVI_open_input_file() of "avilib.c" to open the AVI file:
avi_t *AVI_open_input_file(const char *filename, int getIndex); |
The following program, avi_to_raw.cpp is a sample program that makes use of the avilib functions to extract the video and audio from an AVI file. It is simple and straightforward and the code is self-explained. However, it has to be compiled with the avilib.c program which is also listed below. For example, you can use the following command to compile the executable avi_to_raw:
$g++ -o avi_to_raw avi_to_raw.cpp avilib.c
$./avi_to_raw avi_file [raw_video_output_file raw_audio_output_file]
--------------------------------------------------------- Video Frame Rate: 20 fps Number of Frames: 680 File Duration: 34 sec Video width: 320 pixels Video height: 240 pixels Compressor type: Audio channels: 0 Audio sample rate: 0 Hz Bits per audio sample: 0 Video starts at: 17420 ( 0x440c ) --------------------------------------------------------- |
The following lists the programs avi_to_raw.cpp; the links to avilib.c, and the required header files avilib.h and mpeg4ip.h are also provided. If you want to try it, simply copy-and-paste the code into your own file and download the other three files ( avilib.c, avilib.h, mpeg4ip.h ) to the same directory and compile them using the command mentioned above.
/*
avi_to_raw.cpp
Need to compile together with MPEG4IP avilib.c
Compile: g++ -o avi_to_raw avi_to_raw.cpp avilib.c
Execute: ./avi_to_raw infile [video_output_file audio_output_file]
http://www.webkinesia.com/games/index.php
*/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <math.h>
#include "avilib.h"
using namespace std;
int main( int argc, char *argv[] )
{
FILE *fpo = NULL;
avi_t *avi = NULL; //points to opened avi file
char infilename[30], outfilename[30], *pc;
double videoFrameRate;
unsigned numVideoFrames;
unsigned long fileDuration;
long numBytes = 0, totalBytes = 0;
unsigned long videoFramesRead = 0;
unsigned long emptyFramesRead = 0;
unsigned long short_frames_len = 4;
if ( argc < 2 ) {
printf("\nUsage: avi_to_raw infile [outfile1 outfile2]\n");
exit ( 1 );
}
strcpy ( infilename, argv[1] );
avi = AVI_open_input_file( infilename, 1 );
if (avi == NULL) {
fprintf(stderr, "error %s: %s\n", infilename, AVI_strerror());
exit( 2 );
}
videoFrameRate = AVI_video_frame_rate( avi ); //frame per second
numVideoFrames = AVI_video_frames( avi );
fileDuration = ( unsigned long ) ceil( numVideoFrames / videoFrameRate);
printf("\n---------------------------------------------------------");
printf("\nVideo Frame Rate:\t%5.0f \tfps", videoFrameRate );
printf("\nNumber of Frames:\t%5d", numVideoFrames );
printf("\nFile Duration: \t%5ld \tsec", fileDuration );
printf("\nVideo width: \t%5d \tpixels", avi->width );
printf("\nVideo height: \t%5d \tpixels", avi->height );
printf("\nCompressor type: \t%5s", avi->compressor );
printf("\nAudio channels: \t%5d", avi->a_chans );
printf("\nAudio sample rate:\t%5d \tHz", avi->a_rate );
printf("\nBits per audio sample:\t%5d", avi->a_bits );
printf("\nVideo starts at: \t%5ld ( 0x%lx )", avi->movi_start, avi->movi_start );
printf("\n---------------------------------------------------------\n");
//extract video
if ( argc < 3 ) { //construct raw video output file name
strcpy ( outfilename, infilename );
pc = strrchr ( outfilename, '.' ); //pc points to '.'
if ( pc != NULL ) *pc = 0;
strcat ( outfilename, ".rawv" ); //extenstion .rawv
} else
strcpy ( outfilename, argv[2] );
if ( ( fpo = fopen ( outfilename, "wb" ) ) == NULL ) {
fprintf(stderr, "error opening file %s for output\n", outfilename );
exit ( 3 );
}
// get a buffer large enough to handle a frame of raw SDTV
u_char *buf = ( u_char * ) malloc ( 768 * 576 * 4 );
if ( buf == NULL ) {
fprintf(stderr, "Error allocating memory\n");
exit ( 4 );
}
//move file pointer pointing to beginning of video data
if (AVI_seek_start( avi )) {
fprintf(stderr, "bad seek 0: %s\n", AVI_strerror());
exit( 5 );
}
if (AVI_set_video_position( avi, 0, NULL)) {
fprintf(stderr, "bad seek 1: %s\n", AVI_strerror());
exit( 6 );
}
videoFramesRead = 0;
while ( ( numBytes = AVI_read_frame( avi, (char *)buf ) ) > -1 ) {
totalBytes += numBytes;
videoFramesRead++;
printf("frame %d - length %u total %u\r", videoFramesRead, numBytes, totalBytes);
//eliminate short frames
if ( numBytes > short_frames_len ) {
if ( fwrite ( buf, 1, numBytes, fpo ) != numBytes ) {
fprintf ( stderr, "Error writing %s\n", outfilename );
break;
}
} else {
emptyFramesRead++;
}
if ( videoFramesRead >= numVideoFrames ) break;
} //while
// read error
if (numBytes < 0)
fprintf(stderr, "read error\n" );
if (emptyFramesRead) {
fprintf(stderr, "warning: %u zero length frames ignored\n", emptyFramesRead);
}
fclose ( fpo ); //done with video
if ( avi->a_chans == 0 ) {
printf("\nno audio channel\n" );
exit ( 7 );
}
//extract audio
if ( argc < 4 ) { //construct raw audio output file name
strcpy ( outfilename, infilename );
pc = strrchr ( outfilename, '.' ); //pc points to '.'
if ( pc != NULL ) *pc = 0;
strcat ( outfilename, ".rawa" ); //extenstion .rawa
} else
strcpy ( outfilename, argv[3] );
if ( ( fpo = fopen ( outfilename, "wb" ) ) == NULL ) {
fprintf(stderr, "error opening file %s for output\n", outfilename );
exit ( 9 );
}
unsigned audioFrames = 0;
totalBytes = 0;
if (AVI_seek_start( avi )) {
fprintf(stderr, "Audio bad seek 2: %s\n", AVI_strerror());
exit( 10 );
}
if (AVI_set_audio_position( avi, 0)) {
fprintf(stderr, "Audio bad seek 3: %s\n", AVI_strerror());
exit( 11 );
}
while ((numBytes = AVI_read_audio( avi, (char *)buf, 8192)) > 0) {
totalBytes += numBytes;
if (fwrite ( buf, 1, numBytes, fpo ) != numBytes) {
fprintf(stderr, "error writing %s\n", outfilename );
break;
}
audioFrames++;
printf("frame %d - length %u total %u\r", audioFrames, numBytes, totalBytes);
}
printf("\n");
fclose ( fpo );
return 0;
}
|