# US 12,223,219 B2 · Screen combination method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,219 B2
- Document ID: US12223219B2
- Status: Granted
- Kind: B2
- Application number: 18/264,517
- Filed: 2021-12-09
- Published: 2024-02-08
- Granted: 2025-02-11
- Earliest claimed: 2021-02-08
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Zhiqiang Xie
- Web page: https://patentgrep.com/patent/US12223219B2

## CPC

- G06F 3/1446: {display composed of modules, e.g. video walls}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56)
- G06V 10/757: {Matching configurations of points or features}
- G09G 2300/026: Video wall, i.e. juxtaposition of a plurality of screens to create a display screen of bigger dimensions
- G09G 2356/00: Detection of the display position w.r.t. other display screens
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A method, implemented by a host of a screen splicing system, comprises sending, to a first screen of the screen splicing system, a first indication indicating the first screen to photograph a first image, sending, to a second screen of the screen splicing system, a second indication indicating the second screen to photograph a second image, and determining first orientation information of the first screen according to the first image and second orientation information of the second screen according to the second image.

## Claims

### Claim 1

A method comprising:
sending, to a first screen of a screen splicing system, a first indication instructing the first screen to photograph a first image;
sending, to a second screen of the screen splicing system, a second indication instructing the second screen to photograph a second image, wherein the first screen and the second screen are in communication connection; and
determining first orientation information of the first screen according to the first image and second orientation information of the second screen according to the second image,
wherein the first screen is located in a first position when a first area that is in the first image and that comprises a first gesture or a first action is greater than or equal to a preset threshold, and
wherein the second screen is located in a second position when a second area that is in the second image and that comprises a second gesture or a second action is greater than or equal to the preset threshold.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the first orientation information and the second orientation information comprises:
receiving, from the first screen, the first image;
receiving, from the second screen, the second image; and
determining the first orientation information according to the first image and the second orientation information according to the second image.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the first orientation information and the second orientation information comprises:
performing, according to an image matching algorithm, image matching on the first image and the second image to determine an overlapping area between the first image and the second image; and
determining an orientation of the first screen relative to the second screen according to orientations of the overlapping area in the first image and the second image.

### Claim 4 (depends on 3)

The method of claim 3, wherein determining the orientation comprises:
determining that the first screen is located above the second screen when the overlapping area is located in a lower half area of the first image and is located in an upper half area of the second image;
determining that the first screen is located to an upper right of the second screen when the overlapping area is located at a lower left corner of the first image and is located at an upper right corner of the second image;
determining that the first screen is located to a right of the second screen when the overlapping area is located in a left half area of the first image and is located in a right half area of the second image;
determining that the first screen is located to a lower right of the second screen when the overlapping area is located at an upper left corner of the first image and is located at a lower right corner of the second image;
determining that the first screen is located below the second screen when the overlapping area is located in an upper half area of the first image and is located in a lower half area of the second image;
determining that the first screen is located to a lower left of the second screen when the overlapping area is located at an upper right corner of the first image and is located at a lower left corner of the second image;
determining that the first screen is located to a left of the second screen when the overlapping area is located in a right half area of the first image and is located in a left half area of the second image; and
determining that the first screen is located to an upper left of the second screen when the overlapping area is located in a lower right area of the first image and is located in an upper left area of the second image.

### Claim 5 (depends on 3)

The method of claim 3, wherein the image matching algorithm comprises a scale-invariant feature transform (SIFT) algorithm.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the first orientation information and the second orientation information comprises determining an orientation of the first screen relative to the second screen according to orientations of a target object in the first image and the second image when determining that the first image and the second image comprise the target object.

### Claim 7 (depends on 1)

The method of claim 1, wherein before photographing the first image, the method further comprises:
sending, to the first screen and the second screen, layout information comprising at least one combination mode; and
sending, to the first screen and the second screen, operation information in response to an operation of selecting one combination mode from the at least one combination mode by a user, wherein the operation information instructs the user to perform the first gesture or the first action in the first position and perform the second gesture or the second action in the second position.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
determining first display information corresponding to the first screen and second display information corresponding to the second screen respectively according to the first orientation information and the second orientation information;
sending, to the first screen, the first display information; and
sending, to the second screen, the second display information.

### Claim 9 (depends on 3)

The method of claim 3, wherein the image matching algorithm comprises a speeded up robust features (SURF) algorithm or a fast nearest neighbor search algorithm.

### Claim 10

A screen splicing system, comprising:
a first screen;
a second screen communicatively coupled to the first screen; and
a host coupled to the first screen and the second screen and configured to:
send, to the first screen, a first indication instructing the first screen to photograph a first image;
send, to the second screen, a second indication instructing the second screen to photograph a second image; and
determine first orientation information of the first screen according to the first image and second orientation information of the second screen according to the second image,
wherein the first screen is located in a first position when a first area that is in the first image and that comprises a first gesture or a first action is greater than or equal to a preset threshold, and
wherein the second screen is located in a second position when a second area that is in the second image and that comprises a second gesture or a second action is greater than or equal to the preset threshold.

### Claim 11 (depends on 10)

The screen splicing system of claim 10, wherein the host is further configured to:
receive, from the first screen, the first image;
receive, from the second screen, the second image; and
determine first orientation information according to the first image and second orientation information according to the second image.

### Claim 12 (depends on 10)

The screen splicing system of claim 10, wherein the host is further configured to:
perform, according to an image matching algorithm, image matching on the first image and the second image to determine an overlapping area between the first image and the second image; and
determine an orientation of the first screen relative to the second screen according to orientations of the overlapping area in the first image and the second image.

### Claim 13 (depends on 12)

The screen splicing system of claim 12, wherein the host is further configured to:
determine that the first screen is located above the second screen when the overlapping area is located in a lower half area of the first image and is located in an upper half area of the second image;
determine that the first screen is located to an upper right of the second screen when the overlapping area is located at a lower left corner of the first image and is located at an upper right corner of the second image;
determine that the first screen is located to a right of the second screen when the overlapping area is located in a left half area of the first image and is located in a right half area of the second image;
determine that the first screen is located to a lower right of the second screen when the overlapping area is located at an upper left corner of the first image and is located at a lower right corner of the second image;
determine that the first screen is located below the second screen when the overlapping area is located in an upper half area of the first image and is located in a lower half area of the second image;
determine that the first screen is located to a lower left of the second screen when the overlapping area is located at an upper right corner of the first image and is located at a lower left corner of the second image;
determine that the first screen is located to a left of the second screen when the overlapping area is located in a right half area of the first image and is located in a left half area of the second image; and
determine that the first screen is located to an upper left of the second screen when the overlapping area is located in a lower right area of the first image and is located in an upper left area of the second image.

### Claim 14 (depends on 12)

The screen splicing system of claim 12, wherein the image matching algorithm comprises a scale-invariant feature transform (SIFT) algorithm.

### Claim 15 (depends on 10)

The screen splicing system of claim 10, wherein the host is further configured to determine an orientation of the first screen relative to the second screen according to orientations of a target object in the first image and the second image when determining that the first image and the second image comprise the target object.

### Claim 16 (depends on 10)

The screen splicing system of claim 10, wherein the host is further configured to:
send, to the first screen and the second screen, layout information comprising at least one combination mode; and
send, to the first screen and the second screen, operation information in response to an operation of selecting one combination mode from the at least one combination mode by a user, wherein the operation information instructs the user to perform the first gesture or the first action in the first position and perform the second gesture or the second action in the second position.

### Claim 17 (depends on 10)

The screen splicing system of claim 10, wherein the host is further configured to:
determine first display information corresponding to the first screen and second display information corresponding to the second screen respectively according to the first orientation information and the second orientation information;
send, to the first screen, the first display information; and
send, to the second screen, the second display information.

### Claim 18 (depends on 12)

The screen splicing system of claim 12, wherein the image matching algorithm comprises a speeded up robust features (SURF) algorithm.

### Claim 19 (depends on 12)

The screen splicing system of claim 12, wherein the image matching algorithm comprises a fast nearest neighbor search algorithm.

### Claim 20

A computer program product comprising computer-executable instructions that are stored on a non-transitory computer storage medium and that, when executed by one or more processors, cause a host of a screen splicing system to:
send, to a first screen of the screen splicing system, a first indication instructing the first screen to photograph a first image;
send, to a second screen of the screen splicing system, a second indication instructing the second screen to photograph a second image, wherein the first screen and the second screen are in communication connection; and
determine first orientation information of the first screen according to the first image and second orientation information of the second screen according to the second image,
wherein the first screen is located in a first position when a first area that is in the first image and that comprises a first gesture or a first action is greater than or equal to a preset threshold, and
wherein the second screen is located in a second position when a second area that is in the second image and that comprises a second gesture or a second action is greater than or equal to the preset threshold.