# US 2024/0005658 A1 · Method for detecting display screen boundaries

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005658 A1
- Document ID: US20240005658A1
- Status: Published application
- Kind: A1
- Application number: 18/344,328
- Filed: 2023-06-29
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Sony Interactive Entertainment Europe Limited
- Applicants: Sony Interactive Entertainment Europe Limited
- Inventors: Pedro Federico Quijada Leyton; David Erwan Damien Uberti; Lloyd Preston Stemple; Aron Giuseppe Visciglia
- Web page: https://patentgrep.com/patent/US20240005658A1

## CPC

- G06V 10/248: {by interactive preprocessing or interactive shape modelling, e.g. feature points assigned by a user}
- G06V 20/20: in augmented reality scenes
- G06T 7/73: using feature-based methods
- G06V 10/225: {based on a marking or identifier characterising the area}
- G06V 10/245: {by locating a pattern; Special marks for positioning}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06V 2201/02: Recognising information on displays, dials, clocks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A computer-implemented method for detecting a display screen using an extended reality display device, comprising receiving, at the extended reality display device, an image, or data defining an image, corresponding to a predetermined frame of a video stream, the predetermined frame comprising a plurality of specified feature points, monitoring a display screen that is displaying the video stream, detecting at least some of the plurality of specified feature points in the predetermined frame of the video stream as displayed on the display screen, and determining, based on the detected specified feature points, the physical boundaries of the display screen.

## Claims

### Claim 1

A computer-implemented method for detecting a display screen using an extended reality display device, comprising:
receiving, at the extended reality display device, an image corresponding to a predetermined frame of a video stream, the predetermined frame comprising a plurality of specified feature points;
monitoring a display screen that is displaying the video stream;
detecting at least some of the plurality of specified feature points in the predetermined frame of the video stream as displayed on the display screen; and
determining, based on the detected specified feature points, the physical boundaries of the display screen.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining, based on the detected specified feature points, the orientation and location of the display screen.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, at the extended reality display device, information comprising the location of each specified feature point relative to at least one physical boundary of the display screen.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining, from the image, a location of each specified feature point relative to at least one physical boundary of the display screen.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein determining, based on the detected specified feature points, the physical boundaries of the display screen, comprises:
comparing the detected specified feature points to the image corresponding to the predetermined frame of the video stream.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein, in response to the extended reality display device failing to detect a sufficient number of the specified feature points to enable the physical boundaries of the display screen to be detected, the method comprises outputting a notification from the extended reality display device.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, comprising receiving, at the extended reality display device, a plurality of images, each image corresponding to one of a plurality of predetermined frames of the video stream, each predetermined frame comprising a respective plurality of specified feature points.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein in response to the extended reality display device failing to detect a sufficient number of the specified feature points in a first one of the plurality of predetermined frames to enable the physical boundaries of the display screen to be determined, the method comprises receiving a second image corresponding to a second predetermined frame of the video stream.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, comprising:
detecting at least some of the plurality of specified feature points in the second predetermined frame of the video stream as displayed on the display screen; and
determining, based on the detected specified feature points of the second predetermined frame, the physical boundaries of the display screen.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein in response to the extended reality display device failing to determine the physical boundaries of the display screen, the method comprises:
receiving at the extended reality display device a test image, the test image comprising a plurality of specified feature points;
detecting at least some of the plurality of specified feature points in the test image as displayed on the display screen; and
determining, based on the detected specified feature points in the test image, the physical boundaries of the display screen.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the video stream is a video game or a computer game.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, further comprising:
providing the determined physical boundaries of the display screen to one or more electronic devices operably connected to the extended reality display device via a communication network.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein the extended reality display device and the display screen are in communication with a plurality of electronic devices via a communication network, the method further comprising the initial step of:
communicating with the plurality of electronic devices to determine if the physical boundaries of the display screen are known by any of the plurality of electronic devices; and
in response to determining that the physical boundaries of the display screen are not known by any of the plurality of electronic devices, the method continues according to any of the preceding claims.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, further comprising:
generating one or more virtual boundaries aligned with or overlapping with the determined physical boundaries of the display screen.

### Claim 15 (depends on 14)

The computer-implemented method of claim 14, wherein:
the one or more virtual boundaries are fixed relative to the respective physical boundaries of the display screen; and/or
the one or more virtual boundaries provide a virtual representation of the physical boundaries of the display screen within a virtual environment created by the extended reality display device.

### Claim 16 (depends on 1)

An extended reality display device comprising:
a processor; and
memory, the memory including executable instructions that, as a result of execution by the processor, causes the extended reality display device to perform the computer-implemented method of claim 1.

### Claim 17 (depends on 16)

The extended reality display device of claim 16, wherein the extended reality display device is an augmented reality, AR, headset.

### Claim 18 (depends on 16)

A system comprising:
a display screen;
a computing device; and
an extended reality display device according to claim 16, wherein the display screen, the computing device and the extended reality display device are operably connected by a communication network.
