# US 9,436,437 B2 · Creation, editing and navigation of diagrams > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,437 B2 - Document ID: US9436437B2 - Status: Granted - Kind: B2 - Application number: 12/972,060 - Filed: 2010-12-17 - Published: 2012-06-21 - Granted: 2016-09-06 - Earliest claimed: 2010-12-17 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Jens K. Jacobsen; Sadi Khan; Gareth Alun Jones; Tyler Edward Gibson; Steven Douglas De Mar; Andrew Jude Byrne - Inventors: Jens K. Jacobsen; Sadi Khan; Gareth Alun Jones; Tyler Edward Gibson; Steven Douglas De Mar; Andrew Jude Byrne - Web page: https://patentgrep.com/patent/US9436437B2 ## CPC - G06F 8/10: Requirements analysis; Specification techniques - G06F 9/453: {Help systems} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Organizing data where the data has spatial significance. A method includes, at a user interface, displaying to a user a representation of spatially structured data. User input is received at a computer implemented user interface through one or more hardware user interface devices. The user input is domain agnostic, but has a spatial connotation. Based on pre-existing structured data displayed at the user interface, a domain is determined for the user input. Based on the determined domain, the user input is interpreted into a domain specific response. The domain specific response is consistent with the spatial connotation across a plurality of domains. ## Claims ### Claim 1 A method, implemented at a computer system that includes one or more processors, of organizing data wherein the data has spatial significance, the method comprising: at a user interface, displaying to a user a representation of spatially structured data which includes one or more displayed objects; at the user interface, indicating that a particular object included in the spatially structured data has been selected, including displaying a plurality of directional user interface controls, including (i) a first user interface control above the particular object indicating that an up directional user input associated with the particular object can be received, (ii) a second user interface control below the particular object indicating that a down directional user input associated with the particular object can be received, (iii) a third user interface control to the left of the particular object indicating that a left directional user input associated with the particular object can be received, and (iv) a fourth user interface control to the right of the particular object indicating that a right directional user input associated with the particular object can be received; receiving a directional user input through one or more hardware input devices that is associated with the particular object, the directional user input being selected from among an up direction, a down direction, a left direction, and a right direction; based on receiving the directional user input, determining a domain type of the particular object; and based on the determined domain type of the particular object, interpreting the directional user input to add a new object to the spatially structured data in association with the particular object, wherein: if the domain type is a first domain type, the type of the new object is selected from within a first plurality of object types that are associated with the first domain type, and the type of the new object is further selected from among the first plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the first plurality of object types; and if the domain type is a second domain type, the type of the new object is selected from within a second plurality of object types that are different from the first plurality of object types that are associated with the second domain type, and the type of the new object is further selected from among the second plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the second plurality of object types. ### Claim 2 (depends on 1) The method of claim 1, further comprising additional receiving user input generated with the assistance of a hinting module that provides one or more constrained hints. ### Claim 3 (depends on 2) The method of claim 2, wherein the hints are constrained based on the type of the particular object. ### Claim 4 (depends on 2) The method of claim 2, wherein the hints are constrained based on context. ### Claim 5 (depends on 1) The method of claim 1, further comprising receiving additional user input generated with the assistance of a mnemonics module that provides one or more constrained hints. ### Claim 6 (depends on 1) The method of claim 1, further comprising changing a default type of data within the new object to a new type using a mouse scroll mechanism, wherein activation of the mouse scroll mechanism causes a filtered list of types to be displayed and receiving a user selection of the new type from the displayed types in the filtered list. ### Claim 7 (depends on 1) The method of claim 1, further comprising changing a default type of data within the new object to a new type using user input generated with the assistance of a mnemonics module that causes the new type to be selected from a filtered list of types in response to detecting a typed character. ### Claim 8 (depends on 1) The method of claim 1, further comprising changing a default type of data within the new object to a new type using user input generated with the assistance of a hinting module that causes a filtered list of types to be displayed in response to detecting a typed character and receiving a user selection of the new type from the filtered list of types. ### Claim 9 (depends on 1) The method of claim 1, further comprising creating one or more links between the particular object and the new object using user input generated with the assistance of a hinting module that provides one or more constrained hints. ### Claim 10 (depends on 1) The method of claim 1, wherein the spatially structured data comprises data having a visually observable spatial layout. ### Claim 11 (depends on 1) The method of claim 1, further comprising receiving additional user input for navigating links between data objects in the representation of spatially structured data. ### Claim 12 (depends on 11) The method of claim 11, further comprising displaying a constrained list of possible navigation destinations based on the domain type the particular object in response to detecting a plurality of different navigation destinations in a particular direction identified by received navigation input. ### Claim 13 A computer system, comprising: one or more hardware processors; a display device; one or more hardware input devices; and one or more computer readable hardware storage devices comprising computer executable instructions that are executable by the one or more hardware processors to configure the computer system to organize data wherein the data has spatial significance, including computer executable instructions that are executable to configure the computer system to perform at least the following: display, at a user interface on the display device, a representation of spatially structured data which includes one or more displayed objects; at the user interface, indicate that a particular object included in the spatially structured data has been selected, including displaying a plurality of directional user interface controls, including (i) a first user interface control above the particular object indicating that an up directional user input associated with the particular object can be received, (ii) a second user interface control below the particular object indicating that a down directional user input associated with the particular object can be received, (iii) a third user interface control to the left of the particular object indicating that a left directional user input associated with the particular object can be received, and (iv) a fourth user interface control to the right of the particular object indicating that a right directional user input associated with the particular object can be received; receive a directional user input through the one or more hardware input devices that is associated with the particular object, the directional user input being selected from among an up direction, a down direction, a left direction, and a right direction; based on receiving the directional user input, determine a domain type of the particular object; and based on the determined domain type of the particular object, interpret the directional user input to add a new object to the spatially structured data in association with the particular object, wherein: if the domain type is a first domain type, the type of the new object is selected from within a first plurality of object types that are associated with the first domain type, and the type of the new object is further selected by the one or more hardware processors from among the first plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the first plurality of object types; and if the domain type is a second domain type, the type of the new object is selected from within a second plurality of object types that are different from the first plurality of object types that are associated with the second domain type, and the type of the new object is further selected by the one or more hardware processors from among the second plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the second plurality of object types. ### Claim 14 (depends on 13) The computer system of claim 13, also including computer executable instructions that are executable to configure the computer system to receive additional user input generated with the assistance of a hinting module that provides one or more constrained hints. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the hints are constrained based on the type of the particular object. ### Claim 16 (depends on 14) The computer system of claim 14, wherein the hints are constrained based on context. ### Claim 17 (depends on 13) The computer system of claim 13, also including computer executable instructions that are executable to configure the computer system to receive additional user input generated with the assistance of a mnemonics module that provides one or more constrained hints. ### Claim 18 A computer program product comprising one or more computer readable hardware storage devices comprising computer executable instructions that are executable by one or more processors to configure a computer system to organize data wherein the data has spatial significance, including computer executable instructions that are executable to configure the computer system to perform at least the following: at a user interface, display to a user a representation of spatially structured data structured in a visually observable spatial layout which includes one or more displayed objects; at the user interface, indicate that a particular object included in the spatially structured data has been selected, including displaying a plurality of directional user interface controls, including (i) a first user interface control above the particular object indicating that an up directional user input associated with the particular object can be received, (ii) a second user interface control below the particular object indicating that a down directional user input associated with the particular object can be received, (iii) a third user interface control to the left of the particular object indicating that a left directional user input associated with the particular object can be received, and (iv) a fourth user interface control to the right of the particular object indicating that a right directional user input associated with the particular object can be received; receive a directional user input through one or more hardware input devices that is associated with the particular object, the directional user input being selected from among an up direction, a down direction, a left direction, and a right direction; based on receiving the directional user input, determine a domain type of the particular object; and based on the determined domain type, interpret the directional user input to add a new object to the spatially structured data in association with the particular object, wherein: if the domain type is a first domain type, the type of the new object is selected from within a first plurality of object types that are associated with the first domain type, and the type of the new object is further selected from among the first plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the first plurality of object types; and if the domain type is a second domain type, the type of the new object is selected from within a second plurality of object types that are different from the first plurality of object types that are associated with the second domain type, and the type of the new object is further selected from among the second plurality of object types based on the particular direction of the directional user input, at least two different directions being associated with different object types in the second plurality of object types. ### Claim 19 (depends on 13) The computer system of claim 13, wherein the new object inherits at least one property from the particular object. ### Claim 20 (depends on 13) The computer system of claim 13, wherein the new object is a parent or a child of the particular object.