gems project

Gesture Based Human Computer Interaction in Emergency Management Systems (GEMS)

Emergency Management relies on timely and sufficient information: During an emergency, the efficient display of large amounts of information in a central command room is a vital requirement. In this project, we propose to build a gesture-based human-computer interaction (HCI) prototype. Hand gestures may be used together with head and body posture to convey information: They can help convey predefined meanings; direct attention to certain points, or enhance the meaning of other messages. In an emergency management smart room, they may be used to control information displays; highlight information by means of pointing at it; zoom a certain portion; advance or backtrack in a presentation; or facilitate turn-taking in a meeting environment.

In this project, our goal is to define emergency management interaction scenarios by talking to experts, and collect appropriate data representative of these scenarios in a smart room environment. For each scenario, we will define some gestures and interaction modes, and develop techniques to track humans, track hands, head and torso, detect head pose, torso posture, and segment gestures in time; spot predefined gestures and recognize them, and fuse information from different modalities. We will develop real-time or near-real-time techniques for each portion, and combine these in a demonstrator.

We aim to define sample scenarios that reflect typical emergency management interaction cases. These may include:

  1. A group of people discussing in front of a large screen display
  2. A presenter and an audience
  3. A group of people sitting on a round table

These scenarios will be held in a model room and data will be collected for each of these scenarios. This model room will also be a test bed for the prototype demonstrator of the tools developed in GEMS.

As a kick-off meeting of GEMS, there will be a 2-day workshop in Istanbul, Turkey on 26-27 March 2009. The workshop will involve experts from disaster management centers, who will be invited to give talks and participate in scenario building. Several centers will be visited and example interaction scenarios will be solidified.