Raw Video Player

1. Introduction

   In more advanced games, to enhance the atmosphere of gaming you may need to play a video clip to tell players a story related to the game or to illustrate some game features. The video clip could be obtained from shooting some real sceneries or from computer graphics animation. In this Chapter, we shall discuss how to play a video using data saved in the raw format of a file. We shall discuss the handling of data with compression in a later Chapter.

2. A Simple Raw Video Player

   Actually, to play a raw video clip ( one without compression ) using SDL may be a lot simpler than you thought. All you need to do is to read in the raw data from a file and use SDL_Flip() or SDL_UpdateRect() to blit them on the screen in a controlled time period. For the sake of illustrating the principles and avoiding the burden of handling overheads, let us for instance consider that we already have a raw data file called "sample_video.raw" that saves the raw data of a video clip. Suppose we already know the width and height of the image of each frame. Let us assume that the image size is 320 x 240 and we want to play the video at a rate of 20 fps ( frames per second ). We also assume that an image is saved in RGB format with each color represented by an eight bit number (i.e. 24 bits / pixel ).

   The following program, player.cpp shows how to play such a video clip. It is particularly simple ( everything has been hard-coded for clarity and simplicity to illustrate the main concepts ). It reads one frame of video data from the file sample_video.raw into a buffer at one time. It waits for 50 ms before using SDL_UpdateRect() to blit the data on the screen. The 50 ms delay gives us a frame rate of 20 fps ( 1/20 s = 50 ms ); of course this is a very rough estimate as the operation of reading data also consumes time and a more practical delay should be less than 50 ms in order to achieve a frame rate of 20 fps. Interested readers may click here to obtain a sample file of sample_video.raw' to test the program. ( Simply ignore the .zip extension and save the file with file name "sample_video.raw'. )

   /* player.cpp A simple program to demonstrate playing raw video using SDL. Assume that you have a raw video clip named "sample_video.raw" in the directory and assume that the image size is 320 x 240 and is 24 bits/pixel. compile: g++ -o player player.cpp -I/usr/include -L/usr/local/lib -lSDL execute: ./player see 'http://www.webkinesia.com/games/index.php' */ #include <SDL/SDL.h> #include <stdio.h> #include <stdlib.h> #include <assert.h> int main() { SDL_Surface *screen; FILE *fp; char *buf; int n; //hard-code file name and parameters just for demo purposes if ( ( fp = fopen ( "sample_video.raw", "rb" ) ) == NULL ) { printf("\nError opening file \n"); exit ( 0 ); } //allocate memory to hold one frame of data n = 320 * 240 * 3; buf = ( char *) malloc ( n ); assert ( buf ); //make sure memory has been allocated successfully //initialize video system if ( SDL_Init( SDL_INIT_VIDEO ) < 0 ) { fprintf(stderr, "Unable to init SDL: %s\n", SDL_GetError()); exit(1); } //ensure SDL_Quit is called when the program exits atexit(SDL_Quit); //set video mode of 320 x 240 with 24-bit pixels screen = SDL_SetVideoMode( 320, 240, 24, SDL_SWSURFACE); if ( screen == NULL ) { fprintf(stderr, "Unable to set 320x240 video: %s\n", SDL_GetError()); exit(1); } screen->pixels = buf; //point framebuffer to data buffer while ( 1 ) { //read data ( one frame of n bytes ) into buffer if ( fread ( buf, 1, n, fp ) <= 0 ) break; SDL_Delay ( 50 ); //50 ms per frame SDL_UpdateRect ( screen, 0, 0, 0, 0 ); //blit data to screen } fclose ( fp ); SDL_Delay ( 2000 ); //delay 2 seconds before exit printf("Playing video successful!\n"); return 0; }

3. A Multi-threaded Raw Video Player

   Though the above program, player.cpp can play raw video data, it does not handle data in an effective way. It sits in a main loop to read in the video data and blit them to the screen. In practice, a player has to handle various tasks besides blitting data on the screen. It may have to decode the data or to process the audio; programs like player.cpp tangle all the tasks together and one needs to worry about the coordination between various tasks. A better way to do this is to use multi-threads that we have discussed in an earlier chapter to synchronize the tasks. In this way, playing data ( sending data to screen ) can be cleanly separated from other tasks. The following section describes how to develop a multi-threaded program named tplayer.cpp to play video data.

   In its simplest form, besides the main(), our multi-threaded player needs two threads, one for sending data to the screen and one for 'decoding' data ( at this moment, 'decoding' is simply reading reading from the file ); suppose we name these threads player() and decoder() respectively. This becomes a classical producer-consumer problem. Here, decoder() is the producer which produces resources ( video data ) and player() is the consumer which consumes resources ( video data ).

   Producer-Consumer Problem

   In the producer-consumer problem, to allow producer and consumer threads to run concurrently, we must have available a buffer of items that can be filled by the producer and emptied by the consumer. A producer can produce zero or more items while the consumer is consuming an item; the number of items that can be produced or consumed depend on the production rate as well as the consumption rate and other factors that may influence the production and consumption operations. The producer and consumer must be synchronized, so that the consumer does not try to consume an item that has not yet been produced and the producer suspends production when the buffer is full as there will not be any space to hold the produced item. Therefore, the producer ( decoder() ) must wait when the buffer is full and the consumer ( player() ) must wait when the buffer is empty.

   Typically, when the buffer is empty, the consumer goes to sleep and when the producer has finished producing an item and put it in the buffer, it is responsible to wake up the consumer; on the other hand, when the buffer is full, the producer goes to sleep and the consumer is responsible to wake up the producer after it has consumed an item. These can be implemented using the SDL semaphores that we have discussed in Chapter SDL Threads. However, to slightly simplify our implementation of tplayer.cpp, each thread is responsible to wake up by itself after sleeping ( SDL_Delay() ) for a certain period time and check the buffer condition again. In our implementation, we create a buffer that can hold four frames of video data:

   char *buf[4]; for ( int i = 0; i < 4; ++i ) { buf[i] = ( char * ) malloc ( frameSize ); assert ( buf[i] ); }

   This buffer buf acts as a circular queue. We maintain two pointers, head and tail. Data are entered into the queue at the 'rear' where the tail points at and are deleted ( consumed ) at the front where the head points at. Each time, an item is produced, tail is advanced by 1 and when an item is produced, head is advanced. When head equals tail, indicating that the buffer is empty, the player() thread must wait:

   while ( !quit ) { if ( head == tail ) { //buffer empty (data not available yet ) SDL_Delay ( 30 ); //sleep for 30 ms continue; } .... }

   When tail is 4 ahead of head, it implies that the buffer is full and the decoder() must wait:

   while ( !quit ) { if ( tail >= head + 4 ) { //buffer full SDL_Delay ( 30 ); continue; } .... }

   Modulo 4 is taken on head and tail to point to the actual buffer location.

   Another improvement made in tplayer.cpp over player.cpp is that it uses SDL_GetTicks() to give a much better estimate of the time delay between two frames:

   Uint32 prev_time, current_time; current_time = SDL_GetTicks(); //ms since library starts if ( current_time - prev_time < 50 ) //20 fps ~ 50 ms / frame SDL_Delay ( 50 - ( current_time - prev_time ) ); prev_time = current_time;

   The complete code of tplayer.cpp is listed below. Again, you can try it with sample_video.raw'. You may quit the program by pressing the 'ESC' key.

   /* tplayer.cpp A simple multi-threaded program to demonstrate playing raw video using SDL. Assume that you have a raw video clip named "sample_video.raw" in directory and assume that the image size is 320 x 240 and is 24 bits/pixel. Compile: g++ -o tplayer tplayer.cpp -I/usr/include -L/usr/local/lib -lSDL execute: ./tplayer see 'http://www.webkinesia.com/games/index.php' */ #include <SDL/SDL.h> #include <SDL/SDL_thread.h> #include <stdio.h> #include <malloc.h> #include <stdlib.h> #include <assert.h> //shared variables bool quit = false; char *buf[4]; unsigned long head = 0, tail = 0; unsigned int frameSize; //Consumer int player ( void *scr ) { SDL_Surface *screen = ( SDL_Surface * ) scr; Uint32 prev_time, current_time; current_time = SDL_GetTicks();//ms since library starts prev_time = SDL_GetTicks(); //ms since library starts while ( !quit || head > tail ) { if ( head == tail ) { //buffer empty (data not available yet ) SDL_Delay ( 30 ); //sleep for 30 ms continue; } //consumes the data screen->pixels = buf[head%4]; current_time = SDL_GetTicks(); //ms since library starts if ( current_time - prev_time < 50 ) //20 fps ~ 50 ms / frame SDL_Delay ( 50 - ( current_time - prev_time ) ); prev_time = current_time; SDL_UpdateRect ( screen, 0, 0, 0, 0 ); //update whole screen head++; } //while return 0; } //Producer int decoder ( void *data ) { static FILE *fp = NULL; long n; if ( fp == NULL ) fp = fopen ( "sample_video.raw", "rb" ); if ( fp == NULL ) { printf("\nError opeing file\n" ); quit = true; return 1; } while ( !quit ) { if ( tail >= head + 4 ) { //buffer full SDL_Delay ( 30 ); continue; } //produce data n = fread ( buf[tail%4], frameSize, 1, fp ); if ( n <= 0 ) quit = true; else tail++; } //while return 0; } int main() { SDL_Surface *screen; frameSize = 320 * 240 * 3; SDL_Thread *producer, *consumer; SDL_Event event; int status; char *key; //initialize video system if ( SDL_Init( SDL_INIT_VIDEO ) < 0 ) { fprintf(stderr, "Unable to init SDL: %s\n", SDL_GetError()); exit(1); } //ensure SDL_Quit is called when the program exits atexit(SDL_Quit); //set video mode of 320 x 240 with 24-bit pixels screen = SDL_SetVideoMode(320, 240, 24, SDL_SWSURFACE); if ( screen == NULL ) { fprintf(stderr, "Unable to set 320x240 video: %s\n", SDL_GetError()); exit(1); } //create buffers to hold at most 4 frames of video data for ( int i = 0; i < 4; ++i ) { buf[i] = ( char * ) malloc ( frameSize ); assert ( buf[i] ); } consumer = SDL_CreateThread ( player, screen ); producer = SDL_CreateThread ( decoder, ( void * ) "decdoing" ); while (!quit) { status = SDL_WaitEvent(&event); //wait indefinitely for an event to occur //event will be removed from event queue if ( !status ) { //Error has occured while waiting printf("SDL_WaitEvent error: %s\n", SDL_GetError()); quit = true; return false; } switch (event.type) { //check the event type case SDL_KEYDOWN: //if a key has been pressed key = SDL_GetKeyName(event.key.keysym.sym); printf("The %s key was pressed!\n", key ); if ( event.key.keysym.sym == SDLK_ESCAPE ) //quit if 'ESC' pressed quit = true; else if ( key[0] == 'q' ) //quit if 'q' pressed quit = true; //same as "if ( event.key.keysym.sym == SDLK_q )" } SDL_Delay ( 100 ); //give up some CPU time } SDL_WaitThread ( consumer, NULL ); SDL_WaitThread ( producer, NULL ); printf("Video play successful!\n"); return 0; }