Dynamic Insertion of Virtual Objects in Photographs

Introducing virtual objects in photographs or video sequences presents several challenges, such as the pose estimation and the visually correct interaction boundaries of such objects. In this article a framework for the introduction of virtual objects in user-captured photos is discussed. Furthermore, the introduced virtual objects should be interactive and respond to real physical environments. The proposed detection system is semi-automatic and thus depends on the user to obtain the elements it needs. This operation should be significantly simple to accommodate the needs of a non-expert user. The system analyses a photo taken by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create virtual mixed and augmented reality applications where the user takes one or more photos of a certain place and interactively introduces virtual objects or elements that blend with the picture in real time. This article discusses the techniques required to acquire images and information about the scenario involving the user. To demonstrate the framework, a proof-of-concept implementation is presented. This implementation was used to conduct a user study regarding the evaluation of the reliability of the concept. The presented results show a high reliability in the scene detection and that users are able and motivated to use this type of systems.