System and method for an integrated, multi-modal, multi-device natural language voice services environment

A method to provide an integrated, multi-modal, natural language voice services environment having an input device, a central device, and one or more secondary devices, wherein the method comprises:

• receiving, at the central device, a multi-modal natural language input from the input device, wherein the input device initially received the multi-modal natural language input;
• maintaining, on the input device, the central device, and the one or more secondary devices, a constellation model that describes natural language resources, dynamic states, and intent determination capabilities associated with the input device, the central device, and the one or more secondary devices;
• aggregating the natural language resources, the dynamic states, and the intent determination capabilities associated with the input device and the one or more secondary devices on the central device to converge the natural language resources, the dynamic states, and the intent determination capabilities held across the natural language voice services environment on the central device;
• determining, on the central device, a preliminary intent associated with the multi-modal natural language input using the converged natural language resources, dynamic states, and intent determination capabilities held across the natural language voice services environment;
• sending the multi-modal natural language input from the central device to the one or more secondary devices to invoke the intent determination capabilities associated with the one or more secondary devices;
• collating, at the central device, intent determination responses received from the one or more secondary devices with the preliminary intent determined on the central device to generate an intent hypothesis associated with the multi-modal natural language input on the central device;
• and
• returning the intent hypothesis associated with the multi-modal natural language input and information relating to one or more requests associated with the multi-modal natural language input to the input device, wherein the input device invokes one or more actions based on the returned intent hypothesis and the information relating to one or more requests associated with the multi-modal natural language input.