Ramazan S. Aygun
University of Alabama in Huntsville, USA
Title: Query-by-Gaming: Interactive spatio-temporal querying and retrieval using gaming controller
Biography
Biography: Ramazan S. Aygun
Abstract
Inputting spatio-temporal queries is challenging as it requires a sequence of both spatial and temporal information in queries. Despite development of powerful spatio-temporal index structures and querying languages, forming spatio-temporal queries is still difficult for many users. Inspired from games where a user provides a sequence of actions on a spatial layout, we propose the methodology of query-by-gaming for interactive spatio-temporal querying using a gaming controller. We have applied query-by-gaming for spatio-temporal retrieval of content for tennis videos. Queries are built using a gaming controller rather than a mouse or keyboard. First, for a database of tennis videos, the players, ball, and shot types of players are indexed with respect to their positions using our semantic sequence state graph (S3G). The user is allowed to select the initial positions of objects on the court and a relevant clip is retrieved from the database to build the query. The user starts with this video clip to select the positions of players and ball including where the ball should be sent after a shot using the controller. The system searches the database and determines if there is a relevant clip and the partial clip is displayed if found. Then the user is permitted to control the other player with an option of skipping events and the query is a built as a sequence of interactions.
Meanwhile, the system searches for videos that satisfy the complete query as a conjunction of query segments. At the end of query, the user is provided the list of videos satisfying his or her query.
We use linear temporal logic to formally represent our queries. We have also performed user interface evaluation for comparing gamepad-based interface and mouse interface for forming spatio-temporal queries.