Software engine enabling users to interact directly with a screen using a camera

Abstract

The technology of this application relates to a software engine that enables a user to interact with a display (e.g., television) by using an image capture device (e.g., camera). In one non-limiting example, a user can operate a device (such as a mobile phone) to run an application that interfaces with the software engine, so that the device's camera can operate in unison with a separate display to enable unique types of user interactions. For example, the user may use such an application to tap a user interface button or draw a shape with their hand, directly on the surface of a larger display (e.g., television), in view of the mobile phone's camera; the application's visual state would update in response to the user's captured gestures and to the position and orientation of the mobile phone. In another non-limiting example, a user, while playing a game utilizing the software engine, swings a device (such as a mobile phone) in free space, to control the position and orientation of a paddle within the three dimensional scene of the game, presented on the larger display.

CPC Classifications