# US 2025/0094040 A1 · Interactive projection input and output device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094040 A1
- Document ID: US20250094040A1
- Status: Published application
- Kind: A1
- Application number: 18/926,814
- Filed: 2024-10-25
- Published: 2025-03-20
- Earliest claimed: 2020-12-16
- Assignees: Yong GAO
- Inventors: Yong GAO
- Web page: https://patentgrep.com/patent/US20250094040A1

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure relates to an interactive projection input and output device, which includes: a projecting module, configured to display a screen mirroring content from an external device in a projection region of a projection picture corresponding to the external device; a coordinated operation capturing module, configured to: determine a first coordinate of a user coordinated operation point corresponding to a user coordinated operation behavior in a first coordinate system; a coordinated instruction generating module, configured to, generate a position information instruction according to the first coordinate of the user coordinated operation point and regional position information of the projection region in the first coordinate system; and a wireless transmission module, configured to receive screen mirroring information from the external device, and send the position information instruction to the external device, so as to operate the external device.

## Claims

### Claim 1

An interactive projection input and output device, comprising:
a projecting module, configured to display a screen mirroring content from an external device in a projection region of a projection picture corresponding to the external device;
a coordinated operation capturing module, configured to:
form a coordinated operation capture scope corresponding to the projection region;
establish a first coordinate system within the coordinated operation capture scope;
capture a user coordinated operation behavior occurring within the coordinated operation capture scope; and
determine a first coordinate of a user coordinated operation point corresponding to the user coordinated operation behavior in the first coordinate system;
a coordinated instruction generating module, configured to:
obtain a relative position relationship between the user coordinated operation point and the projection region according to the first coordinate of the user coordinated operation point and regional position information of the projection region in the first coordinate system; and
generate a position information instruction according to the relative position relationship; and
a wireless transmission module, configured to:
receive screen mirroring information from the external device through a downlink wireless connection, wherein the screen mirroring information comprises the screen mirroring content; and,
send the position information instruction to the external device through an uplink wireless connection, such that an operating system of the external device calls an underlying drive program to execute the position information instruction, so as to achieve a control operation in response to the user coordinated operation behavior and thus operate the external device.

### Claim 2 (depends on 1)

The device of claim 1, wherein the projecting module comprises:
a projection function assembly, configured to project the screen mirroring content; and
a projection controller, configured to:
adjust a position, an angle and/or a size of the projection region in response to receiving a projection adjustment instruction from the coordinated instruction generating module; and
perform spectral correction favorable for protection of eye sight according to optical electronic information contained in the screen mirroring content.

### Claim 3 (depends on 1)

The device of claim 1, wherein the coordinated operation capturing module comprises a coordinated operation detector which is disposed to form the coordinated operation capture scope corresponding to the projection region, and the coordinated operation detector is configured to:
capture the user coordinated operation behavior occurring within the coordinated operation capture scope according to detected environmental physical signals;
determine the first coordinate of the user coordinated operation point corresponding to the user coordinated operation behavior in the first coordinate system; and
identify and capture the regional position information of the projection region in the first coordinate system.

### Claim 4 (depends on 1)

The device of claim 1, wherein the coordinated operation capturing module comprises:
a detection signal generator, configured to form the coordinated operation capture scope corresponding to the projection region by sending detection signals; and
a coordinated operation detector, configured to:
capture the user coordinated operation behavior occurring within the coordinated operation capture scope by detecting the detection signals;
determine the first coordinate of the user coordinated operation point corresponding to the user coordinated operation behavior in the first coordinate system; and
identify and capture the regional position information of the projection region in the first coordinate system.

### Claim 5 (depends on 1)

The device of claim 1, wherein the coordinated instruction generating module comprises an application specific integrated circuit chip having a hardware computing circuit formed thereon by burning.

### Claim 6 (depends on 1)

The device of claim 1, wherein the screen mirroring information further comprises at least one of picture vertex coordinates, an eccentric distance, a screen resolution, a horizontal and vertical screen orientation, a data rate, a frame rate or an operating system type; wherein,
the horizontal and vertical screen orientation is used to instruct the coordinated instruction generating module to adjust the relative position relationship;
the picture vertex coordinates or the screen resolution is used to instruct the coordinated instruction generating module to:
generate a second coordinate of a virtual coordinated operation point to be displayed by the external device according to the relative position relationship, and any one of the picture vertex coordinate and the screen resolution, such that the relative position relationship is satisfied between the virtual coordinated operation point and a screen of the external device; and
generate the position information instruction for performing the control operation at the virtual coordinated operation point according to the second coordinate;
the data rate and the frame rate are used jointly to calculate the screen resolution and the horizontal and vertical screen orientation;
the eccentric distance is used to correct the second coordinate of the virtual coordinated operation point; and
the operating system type is used to instruct the coordinated instruction generating module to generate the position information instruction matched with the operating system type according to the relative position relationship.

### Claim 7 (depends on 1)

The device of claim 1, wherein,
the external device is plural in number,
the projecting module is specifically configured to respectively project screen mirroring contents from a plurality of external devices to projection regions of the projection picture corresponding to the plurality of external devices;
the coordinated instruction generating module is specifically configured to:
determine a specific projection region where the user coordinated operation point is located, according to the first coordinate of the user coordinated operation point and regional position information of each of the projection regions in the first coordinate system; and
generate a position information instruction for a specific external device corresponding to the specific projection region according to the relative position relationship between the user coordinated operation point and the specific projection region;
the wireless transmission module is specifically configured to, through wireless connections respectively established with the plurality of external devices,
receive screen mirroring information of the plurality of external devices, wherein the screen mirroring information comprises the screen mirroring contents; and
send the position information instruction for the specific external device from the coordinated instruction generating module to the specific external device.

### Claim 8 (depends on 7)

The device of claim 7, wherein in response to identifying a gesture operation corresponding to a file sharing function, the coordinated instruction generating module is further configured to:
determine an external device as a sharing end and position information of a shared file to be displayed in the external device as the sharing end, based on a start point of the gesture operation,
determine an external device as a receiving end based on a swipe direction or endpoint of the gesture operation, and
generate a file sharing position information instruction, wherein the file sharing position information instruction is used to share the shared file from the external device as the sharing end to the external device as the receiving end.

### Claim 9 (depends on 8)

The device of claim 8, wherein the wireless transmission module is further configured to:
send the file sharing position information instruction to the external device as the sharing end, such that the external device as the sharing end sends the shared file to the external device as the receiving end through a wireless connection established with the external device as the receiving end.

### Claim 10 (depends on 8)

The device of claim 8, wherein the wireless transmission module is further configured to:
send the file sharing position information instruction to the external device as the sharing end to cause the external device as the sharing end to send the shared file to the wireless transmission module through a wireless connection established with the wireless transmission module, such that the wireless transmission module sends the shared file to the external device as the receiving end through a wireless connection established with the external device as the receiving end.

### Claim 11 (depends on 1)

The device of claim 1, wherein the downlink wireless connection established by the wireless transmission module with the external device comprises one or more of router, Bluetooth, NFC, UWB, ZigBee, LiFi, WiFi, NB-IoT, eMTC, LoRa, tuner, GNSS, LNB, cellular 3G, cellular 4G, cellular 5G, cellular 6G, or its own wireless hotspot.

### Claim 12 (depends on 1)

The device of claim 1, wherein the uplink wireless connection established by the wireless transmission module with the external device comprises one or more of Bluetooth, BLE, NFC, router, WiFi, NB-IoT, eMTC, LoRa, tuner, GNSS, LNB, Z-Wave, cellular 3G, cellular 4G, cellular 5G, cellular 6G or wireless USB.

### Claim 13 (depends on 1)

The device of claim 1, further comprising:
a camera module, configured to acquire image information; and
an audio module, configured to acquire audio information and play audio information received by the wireless transmission module from the external device.

### Claim 14 (depends on 1)

The device of claim 1, wherein,
the projecting module comprises an eye sight protection filter, and
the eye sight protection filter comprises an eye sight protection filter lens or an eye sight protection filter film to reduce density of high energy blue light and/or stroboscopic effect harmful to eye sight.

### Claim 15 (depends on 1)

The device of claim 1, wherein the device is integrated into glasses; and
the projecting module is configured to project the projection picture to retinas of human eyes or to lenses of the glasses or to air.

### Claim 16 (depends on 1)

The device of claim 1, wherein the projection picture comprises at least one of: a two-dimensional picture or a three-dimensional space picture, and the projection region comprises at least one of: a two-dimensional region or a three-dimensional space region.

### Claim 17 (depends on 6)

The device of claim 6, wherein the first coordinate comprises at least one of: a two-dimensional coordinate or a three-dimensional space coordinate, and the second coordinate comprises at least one of: a two-dimensional coordinate or a three-dimensional space coordinate.
