# US 2014/0191947 A1 · Using Natural Movements of a Hand-Held Device to Manipulate Digital Content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0191947 A1 - Document ID: US20140191947A1 - Status: Published application - Kind: A1 - Application number: 14/144,254 - Filed: 2013-12-30 - Published: 2014-07-10 - Earliest claimed: 2013-01-04 - Assignees: Texas Instruments Incorporated - Applicants: Texas Instruments Incorporated - Inventors: Vinay Sharma - Web page: https://patentgrep.com/patent/US20140191947A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0317: {in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 7/73: using feature-based methods - G06T 15/04: Texture mapping - G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence) - H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - H04N 9/3194: {including sensor feedback} - G06F 3/005: {Input arrangements through a video camera} - G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation - G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen - G09G 2340/0464: Positioning - G09G 2354/00: Aspects of interface with display user - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A mobile device, such as a smart phone, is provided with a camera. Digital content displayed on display screen of the mobile device may be manipulated in response to natural movements of the mobile device by a user. Motion of the mobile device is detected relative to a nearby textured surface by analyzing images of the textured surface. The displayed digital content is manipulated in response to the detected motion of the mobile device. ## Claims ### Claim 1 A method for displaying an image by a mobile device, the method comprising: displaying digital content on a display screen of the mobile device; detecting motion of the mobile device relative to a nearby textured surface; and manipulating the displayed digital content in response to the detected motion of the mobile device. ### Claim 2 (depends on 1) The method of claim 1, wherein detecting motion of the mobile device comprises: receiving an image of the textured surface from a camera in the mobile device; defining a virtual world coordinate system that is referenced to the image of the textured surface; transforming digital content to the virtual world coordinate system; and determining a camera pose of the camera in relationship to the textured surface; and tracking the camera pose periodically as the camera is moved relative to the textured surface. ### Claim 3 (depends on 2) The method of claim 2, wherein manipulation of the displayed digital content comprises displaying on a display screen of the mobile device a rendered version of the transformed digital content responsive to the camera pose, such that the digital content displayed on the display screen is manipulated in response to motion of the mobile device. ### Claim 4 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises zooming inward in response to the mobile device moving closer to the textured surface and zooming outward in response to mobile device moving further from the textured surface. ### Claim 5 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises zooming inward in response to the mobile device being rotated in a first direction relative to the textured surface and zooming outward in response to mobile device being rotated in a different direction relative to the textured surface. ### Claim 6 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises extending a window that holds the digital content, such that the window is extended in a direction corresponding to motion of the mobile device relative to the textured surface. ### Claim 7 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises warping the perspective view of the digital content in response to rotating the mobile device relative to the textured surface. ### Claim 8 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises selecting a window for display from a plurality of windows in response to pointing the mobile device towards a particular region of the textured surface. ### Claim 9 (depends on 8) The method of claim 8, further comprising indentifying the particular region based on surface parameters of the textured surface. ### Claim 10 (depends on 1) The method of claim 1, wherein manipulating the digital content comprises selecting a particular object within the digital content by hovering the motion of the mobile device in alignment with the particular object. ### Claim 11 A mobile device comprising: processing logic encased in a housing; a user input device coupled to the processing logic; a display screen on a surface of the housing coupled to the processing logic; and wherein the processing logic is operable to: display digital content on the display screen of the mobile device; detect motion of the mobile device relative to a nearby textured surface; and manipulate the displayed digital content in response to the detected motion of the mobile device. ### Claim 12 (depends on 11) The mobile device of claim 11, further comprising a camera mounted to the housing, wherein the camera is coupled to the processing logic, and wherein the camera is operable to receive an image of a textured surface located nearby the mobile device; wherein detecting motion of the mobile device comprises: receiving an image of the textured surface from the camera; defining a virtual world coordinate system that is referenced to the image of the textured surface; transforming digital content to the virtual world coordinate system; and determining a camera pose of the camera in relationship to the textured surface and tracking the camera pose periodically as the camera is moved relative to the textured surface; and wherein manipulation of the displayed digital content comprises displaying on a display screen of the mobile device a rendered version of the transformed digital content responsive to the camera pose, such that the digital content displayed on the display screen is manipulated in response to motion of the mobile device. ### Claim 13 (depends on 11) The mobile device of claim 11, wherein manipulating the digital content comprises zooming inward in response to the mobile device moving closer to the textured surface and zooming outward in response to mobile device moving further from the textured surface. ### Claim 14 (depends on 11) The mobile device of claim 11, wherein manipulating the digital content comprises zooming inward in response to the mobile device being rotated in a first direction relative to the textured surface and zooming outward in response to mobile device being rotated in a different direction relative to the textured surface. ### Claim 15 (depends on 11) The mobile device of claim 11, wherein manipulating the digital content comprises extending a window that holds the digital content, such that the window is extended in a direction corresponding to motion of the mobile device relative to the textured surface. ### Claim 16 (depends on 11) The mobile device of claim 11, wherein manipulating the digital content comprises warping the perspective view of the digital content in response to rotating the mobile device relative to the textured surface. ### Claim 17 (depends on 11) The mobile device of claim 11, wherein manipulating the digital content comprises selecting a window for display from a plurality of windows in response to pointing the mobile device at a particular region of the textured surface. ### Claim 18 (depends on 17) The mobile device of claim 17, further comprising indentifying the particular region based on surface parameters of the textured surface. ### Claim 19 A non-transitory computer readable medium storing software instructions that, when executed by at least one processor in a mobile device, perform a method of displaying an image by the mobile device, the method comprising: displaying digital content on a display screen of the mobile device; detecting motion of the mobile device relative to a nearby textured surface; and manipulating the displayed digital content in response to the detected motion of the mobile device. ### Claim 20 (depends on 19) The method of claim 19, wherein detecting motion of the mobile device comprises: receiving an image of the textured surface from a camera in the mobile device; defining a virtual world coordinate system that is referenced to the image of the textured surface; transforming digital content to the virtual world coordinate system; and determining a camera pose of the camera in relationship to the textured surface and tracking the camera pose periodically as the camera is moved relative to the textured surface; and wherein manipulating the displayed digital content comprises displaying on a display screen of the mobile device a rendered version of the transformed digital content responsive to the camera pose, such that the digital content displayed on the display screen is manipulated in response to motion of the mobile device.