# US 9,519,913 B2 · Providing social impact information associated with identified products or businesses > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,913 B2 - Document ID: US9519913B2 - Status: Granted - Kind: B2 - Application number: 15/159,507 - Filed: 2016-05-19 - Published: 2016-09-15 - Granted: 2016-12-13 - Earliest claimed: 2011-03-08 - Assignees: Bank of America Corporation - Applicants: Bank of America Corporation - Inventors: Matthew A. Calman; Erik Stephen Ross; Alfred Hamilton - Web page: https://patentgrep.com/patent/US9519913B2 ## CPC - G06Q 30/0203: Market surveys; Market polls - G06F 16/951: Indexing; Web crawling techniques - G06Q 10/00: Administration; Management - G06Q 30/02: Marketing; Price estimation or determination; Fundraising - G06Q 30/0201: Market modelling; Market analysis; Collecting market data - G06Q 30/0625: {by formulating product or service queries, e.g. using keywords or predefined options} - G06Q 30/0631: {Recommending goods or services} - G06Q 30/0643: {graphically representing goods, e.g. 3D product representation} - H04L 67/306: User profiles - H04W 4/02: Services making use of location information - H04W 4/029: Location-based management or tracking services - G06Q 10/40: Business processes related to social networking or social networking services - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Embodiments of the invention are directed to methods and apparatuses for capturing a real-time video stream using a mobile computing device, determining, using a computing device processor, which images from the real-time video stream are associated with goods and businesses consistent with a predetermined social impact, and presenting on a display of the real-time video stream, one or more indicators, each indicator being associated with an image determined to be consistent with a predetermined social impact. ## Claims ### Claim 1 A computer implemented method of providing social impact information of goods and services providers that are consistent with a predetermined social impact profile, the computer implemented method comprising: generating a predetermined social impact profile based on input from a user regarding the user's self-defined social impact values; capturing a real-time video stream of a surrounding environment using a video capture device associated with a mobile device of the user, wherein the real-time video stream comprises images of one or more goods or service providers; identifying, via a processor, one or more goods or service providers proximate in location to the mobile device, based on analyzing the captured real-time video stream for one or more identifying markers associated with the one or more goods or service providers; determining, via the processor, which goods or service providers proximate in location to the mobile device are consistent with the predetermined social impact profile; presenting, via a display device, one or more indicators within a presentation of the real-time video stream of the surrounding environment, each indicator of the one or more indicators being associated with a good or service provider proximate in location to the mobile device determined to be consistent with the predetermined social impact profile, the indicator comprising a virtual image superimposed on the image of the good or service provider presented in the real-time video stream; presenting, via the display device, a checking account statement of the user of the mobile device; determining, via the processor, which of one or more transactions conducted between the user and the one or more identified goods or service providers are consistent with the predetermined social impact profile; generating, via the processor, a social impact score of the user based on the determined one or more transactions conducted between the user and the one or more identified goods or service providers that are consistent with the predetermined social impact profile and based in part on one or more transactions that are inconsistent with the predetermined social impact profile; determining, via the processor, which of the determined transactions were used to generate the social impact score, resulting in a first set of transactions; determining, via the processor, which of the transactions that are inconsistent with the predetermined social impact profile were used to generate the social impact score, resulting in a second set of transactions, wherein the first set of transactions and the second set of transactions together make up a third set of transactions; identifying, via the processor, any purchases associated with any of the third set of transactions; presenting, via the display device to the user, the identified purchases, wherein the user may use the identified purchases to adjust the user's behavior to move the determined social impact score in a direction consistent with the user's self-defined social impact values; transmitting, via the processor, the generated social impact score of the user to one or more social network accounts; determining that the mobile device is associated with at least one trigger event, wherein the trigger event comprises: (i) the mobile device being exposed to predetermined low levels of illumination for a first predetermined extended period of time, (ii) the mobile device being stationary for a second predetermined extended period of time or (iii) a power source of the mobile device being associated with predetermined low power levels; disengaging the continuous capture of the real-time video stream of the surrounding environment, based on the at least one trigger event; and re-engaging, automatically, the continuous capture of the real-time video stream of the surrounding environment, based on determining that the mobile device is not associated with the at least one trigger event. ### Claim 2 (depends on 1) The computer implemented method of claim 1, wherein capturing the real-time video stream further comprises implementing, via the processor, object recognition processing to identify the goods or service providers within the real-time video stream. ### Claim 3 (depends on 1) The computer implemented method of claim 1, wherein identifying one or more goods or service providers further comprises capturing a code located on or proximate to one or more of the products or service providers, wherein the code identifies the product or service provider. ### Claim 4 (depends on 1) The computer implemented method of claim 1, wherein identifying one or more goods or service providers further comprises receiving, via the mobile device, wireless communication from a product identifying tag or a service provider identifying tag associated with the goods or service provider. ### Claim 5 (depends on 1) The computer implemented method of claim 1, wherein identifying one or more goods or service providers further comprises determining, via the processor, a location of the mobile device and determining, via the processor, the one or more goods or service providers based on the determined location. ### Claim 6 (depends on 1) The computer implemented method of claim 1, wherein the mobile device is a mobile phone. ### Claim 7 (depends on 1) The computer implemented method of claim 1, wherein the mobile device comprises the display device that presents the one or more indicators. ### Claim 8 (depends on 1) The computer implemented method of claim 1, wherein determining which goods or service providers proximate in location to the mobile device are consistent with the predetermined social impact profile further comprises: collecting data concerning social values of importance to the user; and generating the social impact profile for the user. ### Claim 9 (depends on 1) The computer implemented method of claim 1, wherein determining which images from the real-time video stream are associated with goods or service providers consistent with the predetermined social impact profile further comprises: identifying the goods and service providers captured in the real-time video stream; correlating the goods and service providers identified in the video stream to information regarding the social impact of the goods or service providers; and comparing the information regarding the social impact of the goods and service providers to the social impact profile of the user. ### Claim 10 (depends on 8) The computer implemented method of claim 8 wherein collecting the data concerning the social values of importance to the user comprises collecting user defined data concerning the social values of importance to the user. ### Claim 11 (depends on 8) The computer implemented method of claim 8 wherein collecting the data concerning the social values of importance to the user comprises: analyzing purchasing behavior of the user; and identifying the social values of importance to the user based on the purchasing behavior of the user. ### Claim 12 (depends on 9) The computer implemented method of claim 9 wherein the social impact of the goods or service providers comprises one or more of the following: an environmental impact, nature of production, product attributes, business attributes or charitable activities. ### Claim 13 (depends on 10) The computer implemented method of claim 10 wherein the user defined data comprises responses to survey questions. ### Claim 14 (depends on 10) The computer implemented method of claim 10 wherein the user defined data comprises search terms inputted by the user and the user's desired correlation between the inputted search terms and the goods or service providers. ### Claim 15 (depends on 11) The computer implemented method of claim 11 wherein the purchasing behavior comprises financial records available to a financial institution. ### Claim 16 (depends on 1) The computer implemented method of claim 1, wherein the indicators are selectable by the user. ### Claim 17 (depends on 1) The computer implemented method of claim 1, wherein the indicators, upon being selected present the user with additional information about the goods or service provider. ### Claim 18 (depends on 1) The computer implemented method of claim 1, wherein the indicators, upon being selected present the user with an option to purchase one or more of the goods or a service provided by the service provider via the mobile device. ### Claim 19 A computer program product for providing social impact information of goods and services providers that are consistent with a predetermined social impact profile, the computer program product comprising: a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising: a computer readable program code configured for generating a predetermined social impact profile based on input from a user regarding the user's self-defined social impact values; a computer readable program code configured for capturing a real-time video stream of a surrounding environment using a video capture device associated with a mobile device of the user, wherein the real-time video stream comprises images of one or more goods or service providers; a computer readable program code configured for identifying, via a processor, one or more goods or service providers proximate in location to the mobile device, based on analyzing the captured real-time video stream for one or more identifying markers associated with the one or more goods or service providers; a computer readable program code configured for determining, via the processor, which goods or service providers proximate in location to the mobile device are consistent with the predetermined social impact profile; a computer readable program code configured for presenting, via a display device, one or more indicators within a presentation of the real-time video stream of the surrounding environment, each indicator of the one or more indicators being associated with a good or service provider proximate in location to the mobile device determined to be consistent with the predetermined social impact profile, the indicator comprising a virtual image superimposed on the image of the good or service provider presented in the real-time video stream; a computer readable program code configured for presenting, via the display device, a checking account statement of the user of the mobile device; a computer readable program code configured for determining, via the processor, which of one or more transactions conducted between the user and the one or more identified goods or service providers are consistent with the predetermined social impact profile; a computer readable program code configured for generating, via the processor, a social impact score of the user based on the determined one or more transactions conducted between the user and the one or more identified goods or service providers that are consistent with the predetermined social impact profile and based in part on one or more transactions that are inconsistent with the predetermined social impact profile; a computer readable program code configured for determining, via the processor, which of the determined transactions were used to generate the social impact score, resulting in a first set of transactions; a computer readable program code configured for determining, via the processor, which of the transactions that are inconsistent with the predetermined social impact profile were used to generate the social impact score, resulting in a second set of transactions, wherein the first set of transactions and the second set of transactions together make up a third set of transactions; a computer readable program code configured for identifying, via the processor, any purchases associated with any of the third set of transactions; a computer readable program code configured for presenting, via the display device to the user, the identified purchases, wherein the user may use the identified purchases to adjust the user's behavior to move the determined social impact score in a direction consistent with the user's self-defined social impact values; a computer readable program code configured for transmitting, via the processor, the generated social impact score of the user to one or more social network accounts; a computer readable program code configured for determining that the mobile device is associated with at least one trigger event, wherein the trigger event comprises: (i) the mobile device being exposed to predetermined low levels of illumination for a first predetermined extended period of time, (ii) the mobile device being stationary for a second predetermined extended period of time or (iii) a power source of the mobile device being associated with predetermined low power levels; a computer readable program code configured for disengaging the continuous capture of the real-time video stream of the surrounding environment, based on the at least one trigger event; and a computer readable program code configured for re-engaging, automatically, the continuous capture of the real-time video stream of the surrounding environment, based on determining that the mobile device is not associated with the at least one trigger event. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for capturing the real-time video stream further comprises software code for implementing, via the processor, object recognition processing to identify the goods or service providers within the real-time video stream. ### Claim 21 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for identifying one or more goods or service providers further comprises software code for capturing a code located on or proximate to one or more of the products or service providers, wherein the code identifies the product or service provider. ### Claim 22 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for identifying one or more goods or service providers further comprises software code for receiving, via the mobile device, wireless communication from a product identifying tag or a service provider identifying tag associated with the goods or service provider. ### Claim 23 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for identifying one or more goods or service providers further comprises software code for determining, via the processor, a location of the mobile device and determining, via the processor, the one or more goods or service providers based on the determined location. ### Claim 24 (depends on 19) The computer program product of claim 19, wherein the mobile device is a mobile phone. ### Claim 25 (depends on 19) The computer program product of claim 19, wherein the mobile device comprises the display device that presents the one or more indicators. ### Claim 26 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for determining which goods or service providers proximate in location to the mobile device are consistent with the predetermined social impact profile further comprises software code for: collecting data concerning social values of importance to the user; and generating the social impact profile for the user. ### Claim 27 (depends on 19) The computer program product of claim 19, wherein the computer readable program code for determining which images from the real-time video stream are associated with goods or service providers consistent with the predetermined social impact profile further comprises software code for: identifying the goods and service providers captured in the real-time video stream; correlating the goods and service providers identified in the video stream to information regarding the social impact of the goods or service providers; and comparing the information regarding the social impact of the goods and service providers to the social impact profile of the user. ### Claim 28 (depends on 26) The computer program product of claim 26 wherein the computer readable program code for collecting the data concerning the social values of importance to the user comprises software code for collecting user defined data concerning the social values of importance to the user. ### Claim 29 (depends on 26) The computer program product of claim 26 wherein the computer readable program code for collecting the data concerning the social values of importance to the user comprises software code for: analyzing purchasing behavior of the user; and identifying the social values of importance to the user based on the purchasing behavior of the user. ### Claim 30 (depends on 27) The computer program product of claim 27 wherein the social impact of the goods or service providers comprises one or more of the following: an environmental impact, nature of production, product attributes, business attributes or charitable activities. ### Claim 31 (depends on 28) The computer program product of claim 28 wherein the user defined data comprises responses to survey questions. ### Claim 32 (depends on 28) The computer program product of claim 28 wherein the user defined data comprises search terms inputted by the user and the user's desired correlation between the inputted search terms and the goods or service providers. ### Claim 33 (depends on 29) The computer program product of claim 29 wherein the purchasing behavior comprises financial records available to a financial institution. ### Claim 34 (depends on 29) The computer program product of claim 29, wherein the indicators are selectable by the user. ### Claim 35 (depends on 29) The computer program product of claim 29, wherein the indicators, upon being selected present the user with additional information about the goods or service provider. ### Claim 36 (depends on 29) The computer program product of claim 29, wherein the indicators, upon being selected present the user with an option to purchase one or more of the goods or a service provided by the service provider via the mobile device.