# US 12,179,119 B2 · Cheat detection by comparing mouse input and frame buffer > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,179,119 B2 - Document ID: US12179119B2 - Status: Granted - Kind: B2 - Application number: 17/828,734 - Filed: 2022-05-31 - Published: 2023-08-03 - Granted: 2024-12-31 - Earliest claimed: 2022-02-02 - Assignees: NVIDIA Corporation - Applicants: Nvidia Corporation - Inventors: Joohwan Kim; Josef Spjut - Web page: https://patentgrep.com/patent/US12179119B2 ## CPC - A63F 13/75: Enforcing rules, e.g. detecting foul play or generating lists of cheating players - A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console - A63F 13/25: Output arrangements for video game devices - G06F 3/03544: {having dual sensing arrangement, e.g. two balls or two coils used to track rotation of the pointing device} - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G06T 7/215: Motion-based segmentation - G08B 21/182: {Level alarms, e.g. alarms responsive to variables exceeding a threshold} - G06T 2207/10016: Video; Image sequence - G06T 2207/30244: Camera pose - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS ## Abstract The disclosure provides a cheating detection strategy for interactive programs, which detects programmatically-generated motion from actual human-generated motion based on a comparison of actual motion data to inferred motion data. The cheating detection strategy uses visual and input information to ensure that the input matches the output to detect and avoid cheating tools positioned in between the input and the output. In one example, the disclosure provide a method of monitoring cheating in interactive programs that includes: (1) receiving actual motion data from a user input device, wherein the actual motion data corresponds to interacting with the interactive program, (2) receiving image data of the interactive program that includes image sequences of the interactive program to display on a screen, (3) comparing the actual motion data to inferred motion data determined from the image sequences, and (4) determining possible cheating based on the comparing. ## Claims ### Claim 1 A method of monitoring cheating in interactive programs, comprising: receiving, at a first computing device, actual motion data from a user input device, wherein the actual motion data corresponds to interacting with an interactive program executing on a second computing device; receiving, from the second computing device, image data of the interactive program that includes an image sequence of the interactive program to display on a screen integrated with the first computing device; determining per frame optic flow change from the image sequence; estimating inferred motion data from the image sequence using the per frame optic flow change; comparing the actual motion data to the inferred motion data; and determining possible cheating based on the comparing, wherein the first and second computing devices are different devices. ### Claim 2 (depends on 1) The method as recited in claim 1, further comprising estimating optic flow from the image sequence and determining the per frame optic flow change using the optic flow estimation. ### Claim 3 (depends on 2) The method as recited in claim 2, further comprising estimating an orientation change for the image sequence from the per frame optic flow change and estimating, at the monitor, the inferred motion data using the orientation change for the image sequence. ### Claim 4 (depends on 3) The method as recited in claim 3, further comprising adjusting the inferred motion data using mouse sensitivity or the inverse of mouse sensitivity. ### Claim 5 (depends on 3) The method as recited in claim 3, further comprising adjusting the inferred motion data according to mouse acceleration. ### Claim 6 (depends on 1) The method as recited in claim 1, wherein the comparing includes synching the inferred motion data with the actual motion data. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein the interactive program is a video game. ### Claim 8 (depends on 7) The method as recited in claim 7, wherein the video game is a cursor motion game and the inferred motion data is cursor motion. ### Claim 9 (depends on 1) The method as recited in claim 1, further comprising sending the actual motion data to the second computing device executing the interactive program and sending the image data from the second computing device to the first computing device. ### Claim 10 (depends on 9) The method as recited in claim 9, further comprising storing a sequence of the actual motion data in an input buffer of the monitor. ### Claim 11 (depends on 9) The method as recited in claim 9, further comprising storing the image sequence in a frame buffer of the monitor. ### Claim 12 (depends on 1) The method as recited in claim 1, further comprising sending a cheating alert based on the determining the possible cheating. ### Claim 13 (depends on 12) The method as recited in claim 12, wherein the cheating alert indicates a percentage of a likelihood of cheating or a likelihood of no cheating based on the comparing. ### Claim 14 A computer program product having a series of operating instructions stored on a non-transitory computer readable medium that direct the operation of one or more processors to perform operations including: determining inferred motion data from image sequences received from a computing device, wherein the image sequences are from an interactive program executing on the computing device and the determining includes estimating per-frame camera pose of the image sequences, determining camera pose coordinates for each frame of the image sequence, and calculating an orientation change for the image sequence; comparing actual motion data to the inferred motion data determined from the image sequences, wherein the actual motion data is from a user input device and corresponds to interacting with the interactive program executing on the computing device; and determining possible cheating based on the comparing. ### Claim 15 A monitor, comprising: a screen; an input port configured to receive actual motion data from a user input device, wherein the actual motion data corresponds to interacting with an interactive program on the screen; and one or more processors to perform operations including: receiving image data of the interactive program that includes image sequences of the interactive program to display on the screen; estimating inferred motion data from the image sequences determining per frame optic flow change using motion vectors, and estimating orientation of change using the per frame optic flow changes; comparing the actual motion data to the inferred motion data determined from the image sequences; and determining possible cheating based on the comparing. ### Claim 16 (depends on 15) The monitor as recited in claim 15, wherein the operations further include determining the motion vectors from the image sequences using optic flow. ### Claim 17 A computing device for monitoring cheating, comprising: a screen; one or more interfaces to receive actual motion data from a user input device and image data, wherein the actual motion data corresponds to interacting with an interactive program, and the image data is from the interactive program and includes image sequences of the interactive program to display on a screen; and one or more processors to perform operations including: determining inferred motion data from the image sequences by estimating per-frame camera pose of the image sequences, determining camera pose coordinates for each frame of the image sequence, calculating an orientation change for the image sequence, and adjusting using mouse sensitivity; comparing the actual motion data to the inferred motion data determined from the image sequences; and determining possible cheating based on the comparing. ### Claim 18 (depends on 17) The computing device for monitoring as recited in claim 17, wherein the operations further include adjusting the inferred motion data according to mouse acceleration. ### Claim 19 A cheat detection system for video game play, comprising: a computing device including an integrated screen and one or more processors that generates a cheat alert by comparing actual motion data from an input device used by a user playing a video game to inferred motion data determined from received image sequences of the video game displayed on the screen; and a streaming processor having one or more processors to perform operations including: receiving cheating alerts associated with a user executing the video game, wherein one type of the cheating alerts is the cheat alert generated by the computing device; and determining a likelihood of cheating by the user based on the cheating alerts received over a designated amount of time. ### Claim 20 (depends on 19) The system as recited in claim 19, wherein the computing device is a monitor.