# US 7,609,853 B2 · Detecting a composition of an audience > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,609,853 B2 - Document ID: US7609853B2 - Status: Granted - Kind: B2 - Application number: 10/538,692 - Filed: 2002-12-11 - Published: 2006-09-07 - Granted: 2009-10-27 - Earliest claimed: 2002-12-11 - Assignees: The Nielsen Company (US), LLC - Inventors: Arun Ramaswamy; Daniel J. Nelson; Venugopal Srinivasan - Web page: https://patentgrep.com/patent/US7609853B2 ## CPC - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - H04H 60/33: Arrangements for monitoring the users' behaviour or opinions - H04H 60/45: for identifying users - H04N 7/17309: {Transmission or handling of upstream communications} - H04N 21/258: Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data {(arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30)} - H04N 21/42201: {biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015)} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/433: Content storage operation, e.g. storage operation in response to a pause request, caching operations - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/44222: {Analytics of user selections, e.g. selection of programmes or purchase activity (monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33)} - H04H 60/43: for identifying broadcast channels - H04N 21/4335: Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions {(storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90)} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and apparatus for detecting a composition of an audience of an information presenting device are disclosed. One disclosed method captures images of the audience, determines a number of people within the images and prompts the audience to identify its members if a change in the number of people in the audience is visually detected. This disclosed method avoids both under sampling of the audience and excessive prompting of the audience by only requesting entry of identification data when an audience change occurs. Methods and apparatus to determine the number of persons in an image or series of image are also disclosed. ## Claims ### Claim 1 A method of automatically detecting a composition of an audience of an information presenting device comprising: capturing at least one image of the audience; determining a number of people within the at least one image; and prompting the audience to identify its members if a change in the number of people is detected based on the number of people determined to be within the at least one image, wherein determining the number of people within the at least one image comprises: (a) determining at least one difference image based on at least two captured images; (b) developing at least one shape from the at least one difference image; (c) identifying at least one center of the at least one shape; (d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram; (e) repeating (a)-(d); and (f) if any symbol in the histogram grows beyond a predetermined threshold, counting that symbol as a person. ### Claim 2 (depends on 1) A method as defined in claim 1 wherein capturing the at least one image of the audience comprises digitizing the at least one image. ### Claim 3 (depends on 1) A method as defined in claim 1 wherein determining the at least one difference image based on the at least two of the captured images comprises: (a) determining a first difference image between a first image and a second image; and (b) determining a second difference image between the second image and a third image. ### Claim 4 (depends on 3) A method as defined in claim 3 further comprising developing an amalgamated image from the first and second difference images. ### Claim 5 (depends on 4) A method as defined in claim 4 wherein developing the at least one shape from the at least one difference image comprises developing the at least one shape from the amalgamated image. ### Claim 6 (depends on 1) A method as defined in claim 1 wherein identifying the at least one center of the at least one shape comprises identifying a center of a first shape and a center of a second shape. ### Claim 7 (depends on 6) A method as defined in claim 6 wherein adding the symbol having the predetermined size and indicative of the at least one center of the at least one shape to the histogram comprises adding a first symbol indicative of the center of the first shape to the histogram and adding a second symbol indicative of the center of the second shape to the histogram. ### Claim 8 (depends on 7) A method as defined in claim 7 wherein repeating (a)-(d) comprises stacking a third symbol on the first symbol if a second difference image contains a shape having a center that substantially corresponds to the center of the first shape. ### Claim 9 (depends on 1) A method as defined in claim 1 further comprising excluding a shape from a group of possible human shapes based on a test. ### Claim 10 (depends on 9) A method as defined in claim 9 wherein the test comprises at least one of a location test and a size test. ### Claim 11 (depends on 1) A method as defined in claim 1 further comprising identifying an energy value associated with the at least one difference image, and, performing (f) if the energy value exceeds a predetermined threshold. ### Claim 12 (depends on 1) A method as defined in claim 1 wherein developing the at least one shape from the at least one difference image comprises executing a convex hull process. ### Claim 13 (depends on 1) A method as defined in claim 1 wherein, if any symbol in the histogram does not grow within a predetermined length of time, that symbol is eliminated from the histogram. ### Claim 14 (depends on 1) A method as defined in claim 1 further comprising: if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, adjusting a value to avoid excessive prompting of the audience. ### Claim 15 (depends on 14) A method as defined in claim 14 wherein adjusting the value comprises increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image. ### Claim 16 (depends on 15) A method as defined in claim 15 further comprising recording the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members. ### Claim 17 (depends on 1) A method as defined in claim 1 further comprising periodically exporting recorded data to a device to identify audiences and programs being consumed by the audiences. ### Claim 18 (depends on 1) A method as defined in claim 1 further comprising identifying a program being consumed by the audience. ### Claim 19 (depends on 18) A method as defined in claim 18 wherein identifying a program being consumed by the audience further comprises: identifying a consumption time; and identifying a source of the program being consumed by the audience. ### Claim 20 (depends on 19) A method as defined in claim 19 further comprising recording the source and the consumption time. ### Claim 21 (depends on 19) A method as defined in claim 19 wherein identifying the source of the program comprises identifying a tuned channel. ### Claim 22 (depends on 19) A method as defined in claim 19 further comprising using the consumption time and the source to identify the program from a program guide. ### Claim 23 An apparatus to detect a composition of an audience of an information presenting device comprising: a camera to capture at least one image of the audience; a processor; a memory storing computer readable instructions which, when executed, cause the processor to: determine a number of people within the at least one image; and develop a prompt signal requesting the audience to identify its members if a change in the number of people is visually detected; and an output device responsive to the prompt signal to output an indication requesting the audience to identify its members, wherein the processor determines the number of people within the at least one image by: (a) determining at least one difference image between at least two images captured by the camera; (b) developing at least one shape from the at least one difference image; (c) identifying at least one center of the at least one shape; (d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram; (e) repeating (a)-(d); and (f) if any symbol in the histogram grows beyond a predetermined threshold, counting the symbol as a person. ### Claim 24 (depends on 23) An apparatus as defined in claim 23 wherein the output device is a visual display and the prompt signal is a visual signal. ### Claim 25 (depends on 23) An apparatus as defined in claim 23 wherein the camera digitizes the at least one image. ### Claim 26 (depends on 23) An apparatus as defined in claim 23 wherein, if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, the computer readable instructions cause the processor to adjust a value to avoid excessive prompting of the audience. ### Claim 27 (depends on 26) An apparatus as defined in claim 26 wherein the processor adjusts the value by increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image. ### Claim 28 (depends on 27) An apparatus as defined in claim 27 wherein the computer readable instructions cause the processor to record the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members. ### Claim 29 (depends on 23) An apparatus as defined in claim 23 wherein the computer readable instructions cause the processor to periodically export recorded data to a device to identify audiences and programs being consumed by the audiences. ### Claim 30 (depends on 23) An apparatus as defined in claim 23 wherein the computer readable instructions cause the processor to identify the program being consumed by the audience. ### Claim 31 (depends on 30) An apparatus as defined in claim 30 wherein identifying a program being consumed by the audience further comprises: identifying a consumption time; and identifying a source of the program being consumed by the audience. ### Claim 32 (depends on 31) An apparatus as defined in claim 31 wherein the computer readable instructions cause the processor to record the source and the consumption time. ### Claim 33 (depends on 31) An apparatus as defined in claim 31 wherein identifying the source of the program further comprises identifying a tuned channel. ### Claim 34 (depends on 31) An apparatus as defined in claim 31 further comprising using the consumption time and the source to identify the program from a program guide. ### Claim 35 A computer readable medium storing computer readable instructions which, when executed, cause at least one computer to: capture at least one image of an audience consuming a program; determine a number of people within the at least one image; and output a prompt signal requesting the audience to identify its members if a change in the number of people is detected from the at least one image, wherein the at least one computer determines the number of people within the at least one image by: (a) determining at least one difference image between at least two captured images; (b) developing at least one shape from the at least one difference image; (c) identifying at least one center of the at least one shape; (d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram; (e) repeating (a)-(d); and (f) if any symbol in the histogram grows beyond a predetermined threshold, counting the symbol as a person. ### Claim 36 (depends on 35) A computer readable medium as defined in claim 35 wherein, if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, the computer readable instructions cause the at least one computer to adjust a value to avoid excessive prompting of the audience. ### Claim 37 (depends on 36) A computer readable medium as defined in claim 36 wherein the at least one computer adjusts the value by increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image. ### Claim 38 (depends on 37) A computer readable medium as defined in claim 37 wherein the computer readable instructions cause the at least one computer to record the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members. ### Claim 39 (depends on 35) A computer readable medium as defined in claim 35 wherein the computer readable instructions cause the at least one computer to periodically export recorded data to a device to identify audiences and programs being consumed by the audiences. ### Claim 40 (depends on 35) A computer readable medium as defined in claim 35 wherein the computer readable instructions cause the at least one computer to identify the program being consumed by the audience. ### Claim 41 (depends on 40) A computer readable medium as defined in claim 40 wherein identifying a program being consumed by the audience further comprises: identifying a consumption time; and identifying a source of the program being consumed by the audience. ### Claim 42 (depends on 41) A computer readable medium as defined in claim 41 wherein the computer readable instructions cause the at least one computer to record the source and the consumption time. ### Claim 43 (depends on 41) A computer readable medium as defined in claim 41 wherein identifying the source of the program comprises identifying a tuned channel. ### Claim 44 (depends on 41) A computer readable medium as defined in claim 41 further comprising using the consumption time and the source to identify the program from a program guide.