CS335 Principles of Multimedia Systems

Final Project Report

Marc Brouillette and Patrick Burke

December 20, 2007

Abstract

This report describes a motion based video poker game designed by Patrick Burke and Marc Brouillette as a final project for Principles of Multimedia Systems. The goal was to implement a video poker game controlled entirely by user gestures. This was accomplished by running reacTIVision (an open-source program capable of tracking fiducial markers) alongside a video poker program built to receive the necessary information. The user is able to play with a specialized glove that contains fiducial markers, thus allowing the system to track his or her gestures.

Introduction

The goal of this project was to create a video poker game controlled by user hand gestures rather than a mouse. Video poker is a five card poker game in which the user places a bet, is dealt a hand, and has the option of keeping any or all of their cards, and replacing the ones they do not wish to keep with another draw. After this draw, there is a payout depending on the final hand. Generally, this game is controlled by mouse actions.

To establish hand gesture control, a system of recognition must be implemented. To accomplish this, the open source program reacTIVision was used. This program is able to track a specialized set of fiducial markers with faster than real time performance. Since the program is able to differentiate between fiducials, different fiducials could be used to represent scrolling and clicking performed by a mouse in standard video poker games.

Methods

Video Poker is composed of three classes: Poker, Evaluate, and Card. Poker is the main part of the program. This is where the user interface is created, and it manages the running of the game. Evaluate is used to evaluate the player’s hand to see if he has a winning hand or not. Evaluate.evaluate() is called, with the hand represented as an array of 5 Cards passed in as an argument, and Evaluate then runs through its private methods to see whether or not there is a win. Card is an object that has a rank, suit, a state variable indicating whether or not the card is being held, and a Boolean indicating whether the card’s back or front is showing. Card has a number of accessor methods to tell Poker about itself, such as what its suit and rank are. These methods are used to evaluate the hands, which are made up of 5 Card objects.

To allow the user to interact with the game, a special one-finger glove was created. At the tip of the finger were two fiducials, back to back, which the reacTIVision program could detect. One fiducial is used to move the cursor on the screen, while the other is used to click on a card or button.

When the game is started, the player has 100 credits, with an initial bet of 1 credit. This is the minimum bet. Both of these numbers are displayed in labels at the bottom of the game window. The user can bet up to 5 coins before the hand is dealt by pressing the Bet+ button, and they can decrease their bet by pressing the Bet- button. At the start of the game, 5 card backs are shown. When deal is pressed, the player’s credits are decremented by the bet amount, and the cards are “flipped over” one by one by setting their card back Booleans to false. At this point, the text on the Deal button is changed to say “Draw.” The user can then click on any of their cards to hold them, and then press Draw to get replacements for the cards they did not hold. These replacement cards are turned over one by one like when dealing. Once the cards are all showing, the hand is evaluated using Evaluate.evaluate(), and if there is a win, its text is changed to red on the pay table, and the player’s credits are incremented accordingly (with a cash register sound effect). The player can then change their bet amount and play another hand by pressing Deal. When the player runs out of money, they are given the option of getting 100 more credits.

The program was dependent upon the reacTIVision software. reacTIVision uses the computer's camera to recognize and track fiducial images. The algorithm used first converts the captured frame into an adaptive threshold black and white image. This image is then converted into a specialized tree of black and white areas. This tree is then scanned for particularly left heavy regions, as each fiducial marker is represented by a specific left heavy tree. The left heavy regions are then compared to the fiducial tree dictionary. This allows the fiducial ID, position, and orientation to be determined.

As a template for interacting with the reacTIVision software, the program TUIO Demo, available on reacTIVision's website, was used. This program works by using OSC classes to receive packets directly from the reacTIVision software. The TUIO Client class communicates with the OSC classes to get the information from reacTIVision and creates and deletes TUIO Objects as necessary, passing them their necessary data. Fiducial information is stored in the TUIO Object class. This object stores the fiducial ID, position, orientation, and also has a method for drawing a representation of the fiducial in TUIO Demo. For our purposes, we used this method to draw a gold circle, which would represent the cursor in the poker game:

The TUIO Demo class contains a hash table of TUIO Objects. It draws the interface, and then calls the paint method of its TUIO Objects. For the purposes of this project, an instance of TUIO Demo was created in the Poker class. In this way, fiducial information could be passed into Poker to control the game. The classes' draw functions were used to pass said information. Each time the window is redrawn, the fiducial ID and position are extracted to allow the Poker class to determine the proper action. A series of ingenious if statements check which actions can be performed at that time. For example, if the cards have already been dealt, the user cannot change his or her bet.

This code recognizes the two aforementioned fiducials used for game control. If the scrolling fiducial is recognized, the boolean variable "clicked" is set to false. The instant the click fiducial is recognized it "clicks" the first clickable object over which the player scrolls. This then sets "clicked" to true, preventing further clicking until the scroll fiducial is recognized. The if statements determine if the region is clickable (a button or a card), and carries out the appropriate action.

Results

The result of these methods is a video poker game controlled entirely by gestures, rather than through mouse use. The player is able to move the cursor (a gold spot) by moving a glove with a fiducial attached to it. The player is able to “click” on buttons and cards simply by flipping the glove over and showing the “click” fiducial.

One of the shortcomings of using this method is that the actual action of the player flipping his or her hand over will usually change the location of the cursor, making clicking difficult. A possible solution to this in future work could be changing the code so that the “click” fiducial does not cause the cursor to scroll. In this case, the game will simply lock the cursor in the place it last “saw” the scroll fiducial and “click” there. Another shortcoming was that the reacTIVision software running in conjunction with the poker game did not allow the cursor to move smoothly. To prevent the jumpy cursor, future work could implement a smoothing algorithm to allow the cursor to move in a continuous fashion.

Creation of this game also results in a great deal of practice for the creators. It has been determined that this practice is sufficient to give the creators a great deal of skill and prowess at video poker.

ROYAL FLUSH…

…and the esteemed creators.