# US 11,310,301 B2 · Detecting sensor-based interactions with client device in conjunction with presentation of content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,301 B2 - Document ID: US11310301B2 - Status: Granted - Kind: B2 - Application number: 17/219,596 - Filed: 2021-03-31 - Published: 2021-07-15 - Granted: 2022-04-19 - Earliest claimed: 2016-08-01 - Assignees: Adswizz Inc. - Applicants: AdsWizz Inc. - Inventors: Bruno Nieuwenhuys; Bogdan Bocse; Mihai Ciurea; Caba Ciprian; Guzik Dumitru Florin - Web page: https://patentgrep.com/patent/US11310301B2 ## CPC - H04L 65/70: Media network packetisation - G06F 3/00: Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A content replacement system sends a content item with one or more content replacement parts to be consumed by a user of a client device. The content item is further associated with an interaction pattern that corresponds to a particular user interaction and a defined action that is to be executed by the client device when the particular user interaction is performed on the client device. While the content replacement part is played on the client device, one or more sensors of the client device acquire data regarding user interactions performed by the user on the client device. The acquired data is processed to determine whether the user has performed a user interaction that matches the interaction pattern. Upon detecting a match, the client device sends a notification to the content replacement system indicating that the match has occurred and further executes the defined action. ## Claims ### Claim 1 A method comprising: accessing, by a content replacement system, an interaction pattern corresponding to a content replacement part, the interaction pattern received separately from a third-party system, the interaction pattern corresponding to a physical interaction with a client device while the client device is locked; accessing, by the content replacement system, a defined action associated with the content replacement part that is executed by the client device responsive to a match with the interaction pattern occurring, the match occurring responsive to a similarity score representing a similarity between the interaction pattern and a signal acquired by at least one sensor of the client device being above a score threshold; transmitting the content replacement part, the interaction pattern, and the defined action in a message to the client device, wherein the content replacement part is presented by the client device to a user of the client device responsive to a break marker of a content item being reached, the break marker specifying a time point in the content item, wherein at least a portion of the content item is replaced with the content replacement part, the portion indicated by the time point specified by the break marker, wherein the content replacement part instructs the user to perform a user interaction to demonstrate interest in content of the content replacement part; and receiving, from the client device, an indication of the match between the transmitted interaction pattern and a signal acquired by at least one sensor during presentation of the content replacement part, and wherein the client device executes the defined action upon determining the match between the transmitted interaction pattern and the signal. ### Claim 2 (depends on 1) The method of claim 1 further comprising: receiving, from the client device, a request associated with the defined action. ### Claim 3 (depends on 2) The method of claim 2, wherein the request is sent by the client device and is for at least one of retrieving additional information related to the content replacement part corresponding to the interaction pattern. ### Claim 4 (depends on 2) The method of claim 2, wherein the request is sent by the client device and is for at least one of retrieving additional information related to contacting the user. ### Claim 5 (depends on 1) The method of claim 1, wherein the signal is acquired during the presentation of the content replacement part. ### Claim 6 (depends on 1) The method of claim 1, wherein the interaction pattern specifies a signal pattern received by the at least one sensor that corresponds to a user interaction provided by the user. ### Claim 7 (depends on 6) The method of claim 6, wherein the user interaction is one or more of: shaking the client device, tapping on the client device, tilting the client device in a particular orientation, drawing a gesture on the client device, holding an object in proximity to a proximity sensor of the client device, providing a noise input to the client device, providing a temperature change to the client device, providing a visual image, and providing a fingerprint. ### Claim 8 (depends on 1) The method of claim 1, wherein the at least one sensor of the client device is one or more of an accelerometer, a gyroscope, a capacitive screen, a resistive screen, a temperature sensor, a microphone, a camera, a button, and a fingerprint sensor. ### Claim 9 A non-transitory computer-readable medium comprising memory with instructions encoded thereon, the instructions, when executed by one or more processors, causing the one or more processors to perform operations, the instructions comprising instructions to: access, by a content replacement system, an interaction pattern corresponding to a content replacement part, the interaction pattern received separately from a third-party system, the interaction pattern corresponding to a physical interaction with a client device while the client device is locked; access, by the content replacement system, a defined action associated with the content replacement part that is executed by the client device responsive to a match with the interaction pattern occurring, the match occurring responsive to a similarity score representing a similarity between the interaction pattern and a signal acquired by at least one sensor of the client device being above a score threshold; transmit the content replacement part, the interaction pattern, and the defined action in a message to the client device, wherein the content replacement part is presented by the client device to a user of the client device responsive to a break marker of a content item being reached, the break marker specifying a time point in the content item, wherein at least a portion of the content item is replaced with the content replacement part, the portion indicated by the time point specified by the break marker, wherein the content replacement part instructs the user to perform a user interaction to demonstrate interest in content of the content replacement part; and receive, from the client device, an indication of the match between the transmitted interaction pattern and a signal acquired by at least one sensor during presentation of the content replacement part, and wherein the client device executes the defined action upon determining the match between the transmitted interaction pattern and the signal. ### Claim 10 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the instructions further comprise instructions to: receive, from the client device, a request associated with the defined action. ### Claim 11 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein the request is sent by the client device and is for at least one of retrieving additional information related to the content replacement part corresponding to the interaction pattern. ### Claim 12 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein the request is sent by the client device and is for at least one of retrieving additional information related to contacting the user. ### Claim 13 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the signal is acquired during the presentation of the content replacement part. ### Claim 14 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the interaction pattern specifies a signal pattern received by the at least one sensor that corresponds to a user interaction provided by the user. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the user interaction is one or more of: shaking the client device, tapping on the client device, tilting the client device in a particular orientation, drawing a gesture on the client device, holding an object in proximity to a proximity sensor of the client device, providing a noise input to the client device, providing a temperature change to the client device, providing a visual image, and providing a fingerprint. ### Claim 16 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the at least one sensor of the client device is one or more of an accelerometer, a gyroscope, a capacitive screen, a resistive screen, a temperature sensor, a microphone, a camera, a button, and a fingerprint sensor. ### Claim 17 A content replacement system comprising: memory comprising instructions encoded thereon; and one or more processors that, when executing the instructions, are caused to perform operations comprising: accessing, by the content replacement system, an interaction pattern corresponding to a content replacement part, the interaction pattern received separately from a third-party system, the interaction pattern corresponding to a physical interaction with a client device while the client device is locked; accessing, by the content replacement system, a defined action associated with the content replacement part that is executed by the client device responsive to a match with the interaction pattern occurring, the match occurring responsive to a similarity score representing a similarity between the interaction pattern and a signal acquired by at least one sensor of the client device being above a score threshold; transmitting the content replacement part, the interaction pattern, and the defined action in a message to the client device, wherein the content replacement part is presented by the client device to a user of the client device responsive to a break marker of a content item being reached, the break marker specifying a time point in the content item, wherein at least a portion of the content item is replaced with the content replacement part, the portion indicated by the time point specified by the break marker, wherein the content replacement part instructs the user to perform a user interaction to demonstrate interest in content of the content replacement part; and receiving, from the client device, an indication of the match between the transmitted interaction pattern and a signal acquired by at least one sensor during presentation of the content replacement part, and wherein the client device executes the defined action upon determining the match between the transmitted interaction pattern and the signal. ### Claim 18 (depends on 17) The content replacement system of claim 17, the operations further comprising: receiving, from the client device, a request associated with the defined action. ### Claim 19 (depends on 18) The content replacement system of claim 18, wherein the request is sent by the client device and is for at least one of retrieving additional information related to the content replacement part corresponding to the interaction pattern. ### Claim 20 (depends on 18) The content replacement system of claim 18, wherein the request is sent by the client device and is for at least one of retrieving additional information related to contacting the user.