Back to results
US 12,489,946 B2Granted
Online show rendition system, laughter analysis device, and laughter analysis method
AssigneeLollol Co., Ltd.
InventorHiroyuki Kyan
Life of this patent
Estimating the term…
Abstract
A management terminal device, on the basis of a first emotion analysis value, a second emotion analysis value, a third emotion analysis value, and a user attribute analysis value which are numerical values representing users' emotions, calculates a comprehensive user's emotion analysis value for each user, and, on the basis of a plurality of comprehensive user's emotion analysis values of at least some of the users, generates image information and audio information. When a user terminal device outputs online-delivered information, the management terminal device causes an image output unit and an audio output unit of the user terminal device to output the information with the image information and the audio information superimposed thereon.
Claims22
An online show rendition system comprising:
- a management terminal device for delivering a show online;
- and
- a plurality of user terminal devices, each comprising an image output unit, an audio output unit, an imaging unit, and a sound recording unit, being configured to be capable of mutually communicating information with the management terminal device, and outputting information delivered online by the management terminal device to the image output unit and the audio output unit to allow a user to view the show,
- wherein at least one of the user terminal device and the management terminal device being configured to:
- analyze an emotion of the user based on a motion of a facial expression of the user imaged by the imaging unit of the user terminal device to calculate a first emotion analysis value,
- wherein, when the first emotion analysis value is calculated, at least a first facial surface position corresponding to a root portion, on an eye side of the user, of a zygomaticus major muscle of the user, and a plurality of second facial surface positions corresponding to portions other than the root portion of the zygomaticus major muscle on at least one of left and right sides of a face of the user have been acquired by the imaging unit of the user terminal device;
- based on setting the first facial surface position as a first reference position and setting each of the second facial surface positions as a second reference position, calculate each of displacements between the second reference position and the first reference position in a predetermined time cycle;
- sum up the calculated displacements for every predetermined time cycle to calculate a first feature quantity;
- calculate a laughter-related emotion analysis value, of the first emotion analysis value, of the user based on the first feature quantity;
- analyze a motion of the user based on a motion of a body of the user imaged by the imaging unit of the user terminal device to calculate a second emotion analysis value;
- analyze a reaction of the user based on type and/or volume of an utterance of the user recorded by the sound recording unit of the user terminal device to calculate a third emotion analysis value;
- analyze an attribute of the user based on user information and terminal information acquired through the user terminal device to calculate a user attribute analysis value;
- calculate a comprehensive user's emotion analysis value based on the first emotion analysis value, the second emotion analysis value, the third emotion analysis value and the user attribute analysis value for each of the users;
- generate image information and/or audio information based on a plurality of the comprehensive user's emotion analysis values in at least some of all the users;
- and
- cause the image output unit and/or the audio output unit of the user terminal device to output online-delivered information with the image information and/or the audio information superimposed on the online-delivered information when the user terminal device outputs the online-delivered information.
13 more elements
A program stored on a non-transitory computer-readable medium for causing a computer to implement functions in order to allow a user to view a show, the computer being associated with a management terminal device for delivering the show online and/or a plurality of user terminal devices each comprising an image output unit, an audio output unit, an imaging unit and a sound recording unit, the management terminal device and the plurality of user terminal devices being configured to be capable of mutually communicating information, information online-delivered by the management terminal device being output to the image output unit and the audio output unit of each of the user terminal devices to allow the user to view the show, the functions comprising:
- analyzing an emotion of the user based on a motion of a facial expression of the user imaged by the imaging unit of the user terminal device to calculate a first emotion analysis value,
- wherein, when the first emotion analysis value is calculated, at least a first facial surface position corresponding to a root portion, on an eye side of the user, of a zygomaticus major muscle of the user, and a plurality of second facial surface positions corresponding to portions other than the root portion of the zygomaticus major muscle on at least one of left and right sides of a face of the user have been acquired by the imaging unit of the user terminal device;
- based on setting the first facial surface position as a first reference position and setting each of the second facial surface positions as a second reference position, calculating each of displacements between the second reference positions and the first reference position in a predetermined time cycle;
- summing up the calculated displacements for every predetermined time cycle to calculate a first feature quantity;
- calculating a laughter-related emotion analysis value, of the first emotion analysis value, of the user based on the first feature quantity;
- analyzing a motion of the user based on a motion of a body of the user imaged by the imaging unit of the user terminal device to calculate a second emotion analysis value;
- analyzing a reaction of the user based on type and/or volume of an utterance of the user recorded by the sound recording unit of the user terminal device to calculate a third emotion analysis value;
- analyzing an attribute of the user based on user information and terminal information acquired through the user terminal device to calculate a user attribute analysis value;
- calculating a comprehensive user's emotion analysis value based on the first emotion analysis value, the second emotion analysis value, the third emotion analysis value and the user attribute analysis value for each of the users;
- generating image information and/or audio information based on a plurality of the comprehensive user's emotion analysis values in at least some of all the users;
- and
- causing the image output unit and/or the audio output unit of the user terminal device to output online-delivered information with the image information and/or the audio information superimposed on the online-delivered information when the user terminal device outputs the online-delivered information.
9 more elements
A laughter analysis device configured to:
- acquire, by an imaging unit, at least a first facial surface position corresponding to a root portion, on an eye side of a subject, of a zygomaticus major muscle of the subject, and a plurality of second facial surface positions corresponding to portions other than the root portion of the zygomaticus major muscle on at least one of left and right sides of a face of the subject;
- based on setting the first facial surface position as a first reference position and setting each of the second facial surface positions as a second reference position, calculate each of displacements between the second reference position and the first reference position in a predetermined time cycle, and sum up the calculated displacements for every predetermined time cycle to calculate a first feature quantity;
- and
- calculate a laughter-related emotion analysis value of the subject based on the first feature quantity.
1 more element
A laughter analysis method comprising:
- acquiring, by an imaging unit, at least a first facial surface position corresponding to a root portion, on an eye side of a subject, of a zygomaticus major muscle of the subject, and a plurality of second facial surface positions corresponding to portions other than the root portion of the zygomaticus major muscle on a facial surface of the subject;
- based on setting the first facial surface position as a first reference position and setting each of the second facial surface positions as a second reference position, calculating each of displacements between the second reference position and the first reference position in a predetermined time cycle, and summing up the calculated displacements for every predetermined time cycle to calculate a first feature quantity;
- and
- calculating a laughter-related emotion analysis value of the subject based on the first feature quantity.
1 more element
A non transitory computer-readable medium storing a program causing a computer to implement functions comprising:
- acquiring, at least a first facial surface position corresponding to a root portion, on an eye side of the user, of a zygomaticus major muscle of the subject, and a plurality of second facial surface positions corresponding to portions other than the root portion of the zygomaticus major muscle on a facial surface of the subject acquired by an imaging unit;
- based on setting the first facial surface positions as first reference positions and setting each of the second facial surface positions as second reference positions, calculating each of displacements between the second reference positions and the first reference positions in a predetermined time cycle, and summing up the calculated displacements for every predetermined time cycle to calculate a first feature quantity;
- and calculating a laughter-related emotion analysis value of the subject based on the first feature quantity.
Dependencies are inferred from the claim text.