Alternative modalities generation for digital content based on presentation context

#The present invention relates to computing systems, and more specifically, to processes for enhancing human and computer interaction.

#A modality is a classification of a single independent channel of sensory input/output between a computer and a human. For instance, a computer may receive and interactively use audio, tactile, or visual input, each comprising a modality. Multiple modalities can be used in combination to provide complementary methods to convey information more effectively. For example, artificial intelligence may combine speech understanding with lip-reading and hand gestures to add context. A single modality can thus be combined with others to glean a more accurate or a deeper meaning. When combinations are possible, they can enhance computer-human interaction because certain modalities are more effective at expressing specific types of information than others.

#Despite the promise of such technologies, however, implementations can fall short of their potential when the nature of digital content limits the modality of the information that is being conveyed. For instance, users can be limited to consume content in the modality it was created, making it difficult to experience depending on their context and needs. For example, some content cannot be exploited on an available device. For instance, a scent modality may not be communicated via a cell phone. In another scenario, content may not be designed to support a user's needs, such as for a vision or a hearing impaired consumer.

#FIG. 1 is a block diagram of a system configured to create alternative modalities in real-time;

#FIG. 2 is a block diagram showing a module of the system of FIG. 1 in more detail according to an embodiment;

#FIG. 3 is a flowchart of an embodiment of a method of creating alternative modality content;

#An embodiment of a system automatically enriches content with alternative modalities according to user, device, and environmental characteristics using artificial intelligence and content semantics. The system may provide high production and availability of traditional and nontraditional digital content that does not limit experiences to common modalities of visual and audible sensations. The system may enhance a source of the content by learning about it using artificial intelligence. The automatically gleaned understanding may be used to create alternative modalities according to determined user needs, device characteristics, and environmental parameters.

#Instead of only trying to combine already existing modalities (and being constrained to traditional modalities), an embodiment of the system may automatically enrich content with alternative modalities according to user, device, and environmental characteristics using artificial intelligence and content semantics. An implementation may receive, generate, and output real-time content generation along with user, device, and environment aspects in real-time.

#An embodiment of a system may explore all communicative channels and modalities available to digital content. The system may process in real-time all content being transmitted or presented to generate alternative modalities. To this end, the system may employ artificial intelligence to learn source content semantics. The system may automatically create proper complimentary modalities that were not previously considered. For example, the system may process the source content and apply generative adversarial networks. In a traditional generative adversarial network, two neural networks may contest with one another to generate new data.