# US 2019/0318404 A1 · Systems and methods for presenting information related to products or services being shown on a second display device on a first display device using augmented reality technology > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318404 A1 - Document ID: US20190318404A1 - Status: Published application - Kind: A1 - Application number: 15/950,716 - Filed: 2018-04-11 - Published: 2019-10-17 - Earliest claimed: 2018-04-11 - Assignees: Trivver, Inc. - Applicants: Trivver, Inc. - Inventors: Joel LaMontagne - Web page: https://patentgrep.com/patent/US20190318404A1 ## CPC - G06Q 30/0623: {by investigating goods or services} - G06Q 30/0643: {graphically representing goods, e.g. 3D product representation} - G06T 7/73: using feature-based methods - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04L 67/131: Protocols for games, networked simulations or virtual reality - G06T 2200/28: involving image processing hardware - G06T 2207/10004: Still image; Photographic image - G06T 2207/30204: Marker - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems and methods for presenting information, related to products or services being shown on a second display device, on a first display device using augmented reality technology are disclosed. Exemplary implementations may: receive a request, by a server, from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image generated or captured in a viewpoint of an image capturing system; determine the product or service being shown on the second display device; and transmit information related to the product or service. ## Claims ### Claim 1 A system comprising: one or more hardware processors configured by machine-readable instructions to: receive a request, by a server, from a processing system coupled to a first display device to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm; determine the product or service being shown on a show or channel being displayed on a second display device during a specific period of time; and transmit information related to the product or service to the first display device; wherein the processing system augments the real-time or near real-time image comprising the augmented reality marker with information related to the product or service. ### Claim 2 (depends on 1) The system of claim 1, wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device. ### Claim 3 (depends on 2) The system of claim 2, wherein displaying the 3D artifact on the first display device includes: receive the real-time or near real-time image from the image capturing system; determine the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and receive the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker; wherein the 3D artifact is augmented at the augmented reality marker. ### Claim 4 (depends on 3) The system of claim 3, wherein the center point of the 3D artifact is determined using a two dimensional grid of coordinates at the location of the 3D artifact. ### Claim 5 (depends on 3) The system of claim 3, wherein the rendering location can be expressed in a coordinate system of the first display device, and wherein the metadata further comprises a position and orientation of the 3D artifact in the coordinate system of the first display device using matrix transformation. ### Claim 6 (depends on 1) The system of claim 1, wherein to determine the product or service being shown on the second display device, the one or more hardware processors are further configured by machine-readable instructions to: determine a timestamp value at which the request was received; retrieve a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device during the specific time period; and identify the product or service being displayed on the second display device. ### Claim 7 (depends on 6) The system of claim 6, wherein to determine the product or service being shown on the second display device, the one or more hardware processors are further configured by machine-readable instructions to: determine another timestamp value related to each product or service scheduled to be displayed on the second displaying device during the specific time period; and identify the another timestamp value that is near or equal to the timestamp value at which the request was received. ### Claim 8 A method, comprising: receiving a request, by a server, from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm; determining the product or service being shown on a show or channel being displayed on a second display device during a specific period of time; and transmitting information related to the product or service to the first display device; wherein the processing system augments the image comprising the augmented reality marker with information related to the product or service and displays the information related to the product or service. ### Claim 9 (depends on 8) The method of claim 8, wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device. ### Claim 10 (depends on 9) The method of claim 9, wherein displaying the 3D artifact on the first display device includes: receiving the real-time or near real-time image from the image capturing system; determining the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and receiving the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker; wherein the 3D artifact is augmented at the augmented reality marker. ### Claim 11 (depends on 10) The method of claim 10, wherein the center point of the 3D artifact is determined using a two dimensional grid of coordinates at the location of the 3D artifact. ### Claim 12 (depends on 10) The method of claim 10, wherein the rendering location can be expressed in a coordinate system of the first display device, and wherein the metadata further comprises a position and orientation of the 3D artifact in the coordinate system of the first display device using matrix transformation. ### Claim 13 (depends on 8) The method of claim 8, wherein determining the product or service being shown on the second display device comprises: determining a timestamp value at which the request was received; retrieving a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device during the specific time period; and identifying the product or service being displayed on the second display device. ### Claim 14 (depends on 13) The method of claim 13, wherein determining the product or service being displayed on the second display device further comprises: determining another timestamp value related to each product or service scheduled to be displayed on the second displaying device during the specific time period; and identifying the another timestamp value that is near or equal to the timestamp value at which the request was received. ### Claim 15 A non-transitory computer-readable storage medium comprising instructions which when executed by one or more processors performs a method comprising: receiving a request, by a server, from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm; determining the product or service being shown on a show or channel being displayed on a second display device during a specific period of time; and transmitting information related to the product or service to the first display device; wherein the processing system augments the image comprising the augmented reality marker with information related to the product or service and displays the information related to the product or service. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein displaying the 3D artifact on the first display device includes: receiving the real-time or near real-time image from the image capturing system; determining the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and receiving the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker, wherein the 3D artifact is augmented at the augmented reality marker. ### Claim 18 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the image capturing system is capable of capturing a color image data to an image sensor using at least one of a CCD or CMOS, and a filter for obtaining a color image. ### Claim 19 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein determining the product or service being shown on the second display device comprises: determining a timestamp value at which the request was received; retrieving a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device during the specific time period; and identifying the product or service being displayed on the second display device. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium of claim 19, wherein determining the product or service being displayed on the second display device further comprises: determining another timestamp value related to each product or service scheduled to be displayed on the second displaying device during the specific time period; and identifying the another timestamp value that is near or equal to the timestamp value at which the request was received.