Touching Augmented Reality: Direct Object Manipulation for Marker-less AR on Handheld Mobile Devices

Abstract

Supporting augmented reality (AR) on handheld mobile devices is interesting due to the low costs, low power consumptions, portability, and mobility. Until recently, AR has been accomplished on handheld devices through the aid of distinguishable visual markers in which computer generated virtual objects are displayed over these visual markers, this is referred to as marker AR. A major restriction with marker AR is that the entire visual marker must be visible through a camera of a handheld device at all times. Recent advancements in technology has overcome this limitation, and mobile devices now support marker-less AR on handheld devices in which visual markers are no-longer required. While there are significant work done and results available in addressing virtual object manipulation with marker AR, it is unclear how these findings will apply in marker-less AR applications. With the involved technology on mobile devices being a recent development, there currently is a lack of results addressing virtual object manipulations in marker-less AR settings. The goals of this study are to analyze the distinct differences of virtual object manipulation in marker AR and marker-less AR, understand the implications of these differences, observe the results of these differences in actual user tests, and identify relevant results from marker AR virtual object manipulations to predict and verify the subset that can transition into marker-less AR. This study conducts user testing on five marker AR virtual object manipulation methods. Four methods are the latest results from marker AR research on handheld devices, and a fifth method is a proposed novel method attempting to address the distinctions between the two AR settings. User testing of these methods shows that no single method applies well in all marker-less AR test cases where the requirements of manipulating small scale objects are distinct, and orthogonal from those of interacting with relatively large scale objects. The gathered results suggests that for mark-less AR applications, methods for virtual object manipulations should be adaptive to the relative sizes of the target AR objects.