Hand Gesture Interface for Windows OS

SIMILAR | BUMM
subglobal1 link | subglobal1 link | subglobal1 link | subglobal1 link | subglobal1 link | subglobal1 link | subglobal1 link
subglobal2 link | subglobal2 link | subglobal2 link | subglobal2 link | subglobal2 link | subglobal2 link | subglobal2 link
subglobal3 link | subglobal3 link | subglobal3 link | subglobal3 link | subglobal3 link | subglobal3 link | subglobal3 link
subglobal4 link | subglobal4 link | subglobal4 link | subglobal4 link | subglobal4 link | subglobal4 link | subglobal4 link
subglobal5 link | subglobal5 link | subglobal5 link | subglobal5 link | subglobal5 link | subglobal5 link | subglobal5 link
subglobal6 link | subglobal6 link | subglobal6 link | subglobal6 link | subglobal6 link | subglobal6 link | subglobal6 link
subglobal7 link | subglobal7 link | subglobal7 link | subglobal7 link | subglobal7 link | subglobal7 link | subglobal7 link
subglobal8 link | subglobal8 link | subglobal8 link | subglobal8 link | subglobal8 link | subglobal8 link | subglobal8 link

 

Hand Gesture Interface for Windows Operating Systems

Introduction

The Hand Gesture Interface is a real time gestural interface based on 3D dynamic hand gesture recognition using simple webcams. The application captures and recognizes hand gestures of the user wearing colored gloves, where the hand coordinates are obtained via 3D reconstruction from stereo.

The developed system aims to be a generic interface for Windows based applications, and provides supplementary features such as an interactive training and gesture defining system, a gesture tutor, a self-calibration utility for the cameras and a tool for linking the interface to different applications. The interface can be used by people with disabilities for various tasks, as well as by common users for controlling of desktop applications. The system can be linked to any third party program through generation of windows based mouse and keyboard events.

Requirements

The system is tested on Windows XP and Windows Me operating systems. In theory it should work on all newer Windows systems, although not confirmed to do so. The system also relies on MFC dll's, which are statically linked to the executable to avoid compatibility problems.

The application can be run in 2D mode or 3D mode (requires two webcams to be present).

Requirements for 2D mode:

  • A camera accesible via VFW drivers. All webcams fall into this category. The camera must support the 24 bit RGB image format. (USB 2.0 cameras recommended)
  • A fast CPU ( at least 1.6 GHz recommended, even though the system will still work on much slower computers)
  • A smoothly colored glove. The tracking works best if the hue of the glove is not close to the skin hue. also gray (including black and white) gloves cannot be used.

Requirements for 3D mode:

  • At least two cameras accesible via VFW drivers. All webcams fall into this category. The cameras must support the 24 bit RGB image format, and must be configured to have the same resolutions beforehand. (USB 2.0 cameras recommended)
  • If the cameras are USB, the mainboard must be able to handle enough bandwidth to allow processing of both cameras. Since USB ports are usually coupled, at least 4 USB ports might be necessary.
  • A fast CPU ( at least 1.6 GHz recommended, even though the system will still work on much slower computers)
  • A smoothly colored glove. The tracking works best if the hue of the glove is not close to the skin hue. also gray (including black and white) gloves cannot be used.

There are several parameters, which help smooth tracking of the hand. These usually don't need to be touched if the lighting conditions are good enough. Avoid illuminating the scene with only directional light, since it will cause too strong shadows, which is not the optimal condition for hue based tracking.

Application Explanation and Screenshots

Marker Registration

The marker, a colored glove, must be registered to the system. The registration is done by pressing the corresponding button and waving the hand to the camera(s).

Camera Calibration

The camera calibration is done using a tool included in the system that is designed for a special calibration object. There is also another tool, which allows calibration and angle correction by hand tracking.

Kalman Filtering

To eliminate measurement noise Kalman filter is applied on the trajectories. Here the blue lines are the measured trajectories, whereas the red ones are the filtered ones. Filtering can be turned on and off from the options menu.

Hand Modes

When the hand is found in an image, its elongation is calculated. The vertical aligned hand is considered in neutral mode and gesture recognition is allowed. The horizontal aligned mode is considered in left-click mode, and it is used to simulate the left click of a mouse.

Gesture Training

The interface comes with eight pre-trained gestures. The user can define and train new gestures from the interface. The user usually needs to perform the gesture for 30-40 times for it to be properly recognized by the system. The trained gestures can be linked to keyboard or omuse events by manually editing the configuration file "/configuration/appconfig.cfg". The syntax is simple and is given in the default configuration files.

Gesture Recognition

There is no indicator for the starting and ending of gestures. The interface automatically detects the starting point of gestures using a threshold model. When the gesture is recognized, its correspoinding event is raised.

SIMILAR | BUMM | Top | ©2006