# US 8,752,010 B1 · Dynamic interface synthesizer > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,752,010 B1 - Document ID: US8752010B1 - Status: Granted - Kind: B1 - Application number: 09/001,854 - Filed: 1997-12-31 - Granted: 2014-06-10 - Earliest claimed: 1997-12-31 - Assignees: Honeywell International Inc. - Applicants: Robin R. Penner - Inventors: Robin R. Penner - Web page: https://patentgrep.com/patent/US8752010B1 ## CPC - G06F 8/38: for implementing user interfaces - G06F 8/34: Graphical or visual programming - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A user interaction synthesizer is described in the context of an application program running on a computer system having diverse user interaction devices attached to it. The user interface for the application is dynamically designed and synthesized based on the states of individual application objects that are used by an interaction facilitator to traverse a framework of interaction elements and match situation variables to appropriate elements by a specialization process. Interaction elements are selected by the interaction facilitator to create an interaction space designed to satisfy the information needs of the current active objects in the application. Finally, a number of modules are used to convert the elements in the interaction space to generate a user interface as a function of the resources and devices available for the user. ## Claims ### Claim 1 An interaction synthesizing system for a computer running an application program formed of multiple objects, the system comprising: an interaction space including: a top level user interface object; an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable object representative of a user interface element, wherein the instantiable object inherits capabilities from the to level user interface object or from the intermediate level abstract object; and an interface synthesizer which generates the user interaction representations for a current state of an application and modifies the interaction space dynamically in response to changes in the application objects. ### Claim 2 (depends on 1) The system of claim 1 wherein the user interaction representations are representative of sensed parameters. ### Claim 3 A method of creating a user interface for an application program running on a computer system having devices available for communicating with a user, comprising the computer-executed steps of: retrieving an interaction model of interaction requirements for general situational components of the application program, the interaction model including: a top level user interface object; an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable object representative of a user interface element, wherein the instantiable object inherits capabilities from the to level user interface object or from the intermediate level abstract object; creating an interaction space representative of current interaction requirements of the application program using the interaction model; dynamically creating a user interface for the application program based on the interaction space and available user interaction devices; and modifying the interaction space in response to changes in the application program. ### Claim 4 (depends on 3) The method of claim 3 wherein the available interaction devices have different capabilities for interacting with a user, and wherein the user interface is dynamically created responsive to those different capabilities. ### Claim 5 (depends on 4) The method of claim 4 wherein one of the devices comprises a telephone, and the other device comprises a computer, and wherein the interface created for the telephone provides information to the user in the form of voice, and wherein the interface created for the computer comprises a visual representation of the interaction space. ### Claim 6 A method of creating an interaction space having components representative of user interaction needs with an application program, comprising the computerized steps of: perceiving an interaction need; entering an interaction class model at a point dependent on the perceived interaction need, the interaction class model including: a top level user interface object; an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable node representative of user interface elements, wherein the instantiable node inherits capabilities from the to level user interface object or from the intermediate level abstract object; specializing the class model to arrive at the instantiable node in the interaction class model; adding the instantiable node to the interaction space; and modifying the interaction space dynamically in response to changes in the application program. ### Claim 7 (depends on 6) The method of claim 6 and further comprising the step of: converting the interaction space into a user interface. ### Claim 8 (depends on 7) The method of claim 7 wherein the conversion of the interaction space into a user interface step is performed as a function of capabilities of a user interaction device. ### Claim 9 A non-transitory machine readable medium comprising instructions that when executed by a processor execute a process comprising: inheriting capabilities from a top level user interface object to an intermediate level abstract object representing an abstract user interaction type, wherein the top level abstract user interface object has a create operation, and wherein the intermediate level abstract object comprises an object selected from the group consisting of tasks, subtasks, components, and primitives; inheriting capabilities from the top level object or intermediate level object to an instantiable object representative of a user interface element; automatically dynamically generating the user interaction representation for a current state of an application; and modifying the user interaction representation in response to changes in the application. ### Claim 10 A machine readable medium having instructions stored thereon for causing a suitably programmed computer to perform the computerized steps comprising: retrieving an interaction model of interaction requirements for general situational components of the application program, the interaction model including: a top level user interface object; an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable object representative of a user interface element, wherein the instantiable object inherits capabilities from the to level user interface object or from the intermediate level abstract object; constructing a representation of current interaction requirements of the application program using the interaction model; creating an interaction space from the representation, from one or more abstract compositional constraints, and from one or more selected properties of the application program; dynamically creating a user interface for the application program based on the interaction space and available devices; and modifying the interaction space in response to changes in the application program. ### Claim 11 A machine readable medium having instructions stored thereon for causing a suitably programmed computer to perform the steps comprising: perceiving a change in an object in an application having an interaction need; entering an interaction class model at a point dependent on the perceived interaction need, the interaction class model including: a top level user interface object; an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable object representative of user interface elements, wherein the instantiable object inherits capabilities from the to level user interface object or from the intermediate level abstract object; specializing the class model to arrive at the instantiable object in the interaction class model; and adding the instantiable object to the interaction space. ### Claim 12 A computer system comprising: an application comprised of multiple objects, including to level objects for interacting with users; an interaction class model including: an intermediate level abstract object representing an abstract user interaction type selected from the group consisting of tasks, subtasks, components, and primitives, wherein the intermediate level abstract object inherits capabilities from the to level user interface object; and an instantiable class model object representative of a user interface element, wherein the instantiable object inherits capabilities from the to level user interface object or from the intermediate level abstract object; and an interaction facilitator coupled to the application and to the interaction class model for creating an interaction space comprised of the instantiated class model objects responsive to currently active application objects and for modifying the interaction space dynamically in response to changes in the application objects. ### Claim 13 (depends on 12) The system of claim 12 wherein the application comprises multiple dynamically varying networked objects. ### Claim 14 (depends on 12) The system of claim 12 wherein the application is selected from the group consisting of search and rescue applications, building management applications, and process control applications. ### Claim 15 (depends on 12) The system of claim 12 and further comprising a module coupled to the interaction space for converting interaction elements to a user interface on a selected device based on the capabilities of the selected device. ### Claim 16 (depends on 15) The system of claim 15 wherein the module is selected from the group consisting of a JavaTech, and a SpeechTech.