# US 2018/0322701 A1 · Syndication of direct and indirect interactions in a computer-mediated reality environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322701 A1 - Document ID: US20180322701A1 - Status: Published application - Kind: A1 - Application number: 15/586,570 - Filed: 2017-05-04 - Published: 2018-11-08 - Earliest claimed: 2017-05-04 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Michel PAHUD; Nathalie RICHE; Eyal OFEK; Christophe HURTER - Web page: https://patentgrep.com/patent/US20180322701A1 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 27/0172: {characterised by optical features} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G02B 2027/0118: {comprising devices for improving the contrast of the display / brillance control visibility} - G02B 2027/0127: {comprising devices increasing the depth of field} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0141: {characterised by the informative content of the display} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves} - G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041) - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer - G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation - G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06T 3/60: Rotation of whole images or parts thereof - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In various embodiments, computerized methods and systems for syndicating direct and indirect interactions with objects in a computer-mediated environment to facilitate precise interactions with the objects in the computer-mediated environment are provided. The system detects a direct interaction with an object in the computer-mediated reality environment. The direct interaction may be a natural or hypernatural interaction. Subsequently, the system may determine various options of indirect interaction with the object related to the direct interaction. The indirect interaction may be generated by a controller. Upon receiving an indirect interaction, the system may modify the object based on the syndication of the direct interaction and the indirect interaction. ## Claims ### Claim 1 A computing system, comprising: a direct interaction component to receive a first input signal that is generated based on a detected direct interaction that corresponds to a computer-generated object presented in a computer-mediated reality environment, wherein the direct interaction is one of a natural or hypernatural interaction; an indirect interaction component, operationally coupled to the direct interaction component, to receive a second input signal that is generated based on a detected indirect interaction that also corresponds to the presented computer-generated object, wherein the indirect interaction is a controller-based interaction; and an interaction syndication component, operationally coupled to the direct interaction component and the indirect interaction component, to determine a modification of the presented computer-generated object based at least in part on a combination of the first input signal and the second input signal. ### Claim 2 (depends on 1) The computing system of claim 1, further comprising: a sensing component, operationally coupled to the direct interaction component, to generate the first input signal based at least in part on one of a detected hand or finger position in the computer-mediated reality environment or a detected gaze directed to the presented computer-generated object. ### Claim 3 (depends on 1) The computing system of claim 1, further comprising: an input device, coupled to the indirect interaction component, to generate the second input signal based at least in part on the detected indirect interaction. ### Claim 4 (depends on 3) The computing system of claim 3, wherein the input device comprises a controlling component to generate a measurement signal to be a part of the second input signal, the measurement signal being indicative of a magnitude of the detected indirect interaction. ### Claim 5 (depends on 1) The computing system of claim 1, wherein the direct interaction component is further to determine a characteristic of the detected direct interaction. ### Claim 6 (depends on 5) The computing system of claim 5, wherein the direct interaction component is further to generate an indication for haptic feedback based at least in part on the determined characteristic of the detected direct interaction. ### Claim 7 (depends on 5) The computing system of claim 5, wherein the indirect interaction component is further to determine a plurality of options for indirect interaction based at least in part on the determined characteristic of the detected direct interaction. ### Claim 8 (depends on 7) The computing system of claim 7, further comprising: a head-mounted display device, operationally coupled to the indirect interaction component, to cause at least one of the plurality of options to be presented in the computer-mediated reality environment. ### Claim 9 (depends on 1) The computing system of claim 1, wherein the interaction syndication component is further to determine a form of the modification based on the first input signal, and a control parameter of the form of the modification based on the second input signal. ### Claim 10 (depends on 1) The computing system of claim 1, wherein the direct interaction component is further to determine an interaction point of the object related to the detected direct interaction, and the indirect interaction component is further to determine a magnification setting related to the interaction point or a rotation setting of the object associated with the interaction point. ### Claim 11 A computer-implemented method, comprising: detecting a direct interaction with an object that is visible within a computer-mediated reality environment; determining one or more options of indirect interaction with the object based at least in part on a characteristic of the direct interaction; receiving a signal corresponding to one of the one or more options of indirect interaction; and modifying the computer-mediated reality environment in relation to the object based at least in part on the signal. ### Claim 12 (depends on 11) The method of claim 11, wherein detecting the direct interaction comprises determining that a distance between a body part of a user and the object is within a first predetermined threshold distance, and that the direct interaction is detected for a duration exceeding a second predetermined threshold duration. ### Claim 13 (depends on 11) The method of claim 11, wherein the object is a physical object, and wherein modifying the computer-mediated reality environment comprises augmenting the physical object with a virtual object based at least in part on the signal. ### Claim 14 (depends on 11) The method of claim 11, wherein the object is a virtual object, and wherein modifying the computer-mediated reality environment comprises transforming the virtual object based at least in part on the signal. ### Claim 15 (depends on 11) The method of claim 11, wherein modifying the computer-mediated reality environment comprises transforming the object based at least in part on a form of a transformation indicated by the characteristic of the direct interaction, and a degree of the transformation indicated by the signal. ### Claim 16 One or more computer storage media having computer-executable instructions embodied thereon that, when executed by one or more processors, causes the one or more processors to: detect a direct interaction with an object in a computer-mediated reality environment; receive a signal indicating an indirect interaction with the object; determine a form of a transformation of the object based at least in part on the direct interaction, and a degree of the transformation based at least in part on the indirect interaction; and modify rendering parameters of the object in the computer-mediated reality environment based on the form and the degree of the transformation. ### Claim 17 (depends on 16) The media of claim 16, further causes the one or more processors to: determine the form of the transformation to be a linear movement and the degree of the transformation to be a distance of the linear movement. ### Claim 18 (depends on 16) The media of claim 16, further causes the one or more processors to: determine the form of the transformation to be a rotation and the degree of the transformation to be an angle of the rotation. ### Claim 19 (depends on 16) The media of claim 16, further causes the one or more processors to: determine the form of the transformation to be a magnification and the degree of the transformation to be a ratio of the magnification. ### Claim 20 (depends on 16) The media of claim 16, further causes the one or more processors to: determine the form of the transformation to be a deformation and the degree of the transformation to be a degree of the deformation.