# US 2025/0165274 A1 · Interface interaction method and apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165274 A1
- Document ID: US20250165274A1
- Status: Published application
- Kind: A1
- Application number: 18/953,466
- Filed: 2024-11-20
- Published: 2025-05-22
- Earliest claimed: 2023-11-20
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Di Yu; Lingyun Chen; Yuchen Lian; Yuan Wang; Zhizhou Pan; Changbo Li; Zhanwei Liu
- Web page: https://patentgrep.com/patent/US20250165274A1

## CPC

- G06F 9/453: {Help systems}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G10L 13/00: Speech synthesis; Text to speech systems
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Embodiments of the present disclosure provide an interface interaction method and apparatus, an electronic device, and a storage medium. The method includes: displaying a target interface, where at least one interface object is displayed on the target interface, and the interface object includes an interface display resource and/or an interface display control; acquiring, in response to an object trigger operation being input on an interface object, the triggered interface object as a target object; and determining voice introduction information corresponding to the target object and playing the voice introduction information.

## Claims

### Claim 1

An interface interaction method, comprising:
displaying a target interface, wherein at least one interface object is displayed on the target interface, and the interface object comprises an interface display resource and/or an interface display control;
acquiring, in response to an object trigger operation being input on an interface object, the triggered interface object as a target object; and
determining voice introduction information corresponding to the target object; and
playing the voice introduction information.

### Claim 2 (depends on 1)

The interface interaction method according to claim 1, wherein determining the voice introduction information corresponding to the target object comprises:
generating, in response to the target object being the interface display resource, the voice introduction information corresponding to the target object based on resource content of the interface display resource.

### Claim 3 (depends on 2)

The interface interaction method according to claim 2, wherein generating the voice introduction information corresponding to the target object based on the resource content of the target object comprises:
generating an object keyword corresponding to the target object based on the resource content of the target object; and
generating, based on the object keyword and preset description prompt information, a content introduction text corresponding to the target object, and converting the content introduction text into the voice introduction information.

### Claim 4 (depends on 3)

The interface interaction method according to claim 3, wherein the interface display resource comprises an image resource, and generating the object keyword corresponding to the target object based on the resource content of the target object comprises:
inputting the image resource to a content recognition model for content recognition to obtain the object keyword corresponding to the image resource, wherein the content recognition model is obtained by training a neural network model based on a sample image and an expected keyword corresponding to the sample image, and the expected keyword is a keyword associated with image content of the sample image.

### Claim 5 (depends on 3)

The interface interaction method according to claim 3, wherein the interface display resource comprises a video resource, and generating the object keyword corresponding to the target object based on the resource content of the target object comprises:
acquiring a plurality of key frames in the video resource, and respectively performing content recognition on each key frame to obtain frame content keywords; and
determining the object keyword corresponding to the video resource based on association relationships among the plurality of key frames and the frame content keywords corresponding to the key frames.

### Claim 6 (depends on 3)

The interface interaction method according to claim 3, wherein generating, based on the object keyword and the preset description prompt information, the content introduction text corresponding to the target object comprises:
inputting the object keyword and the preset description prompt information to a text generation model to generate the content introduction text corresponding to the target object, wherein the text generation model is obtained by training a deep learning model based on sample keywords, sample prompt information, and an expected introduction text.

### Claim 7 (depends on 1)

The interface interaction method according to claim 1, wherein determining the voice introduction information corresponding to the target object comprises:
generating, in response to the target object being the interface display control, the voice introduction information corresponding to the interface display control based on function associated information corresponding to the interface display control.

### Claim 8 (depends on 7)

The interface interaction method according to claim 7, wherein the function associated information comprises an acting object corresponding to the interface display control and an acting result generated after the interface display control acts on the acting object; and
generating the voice introduction information corresponding to the interface display control based on the function associated information corresponding to the interface display control comprises:
determining the acting object corresponding to the interface display control and the acting result generated after the interface display control acts on the acting object;
generating a function description text corresponding to the interface display control based on the acting object and the acting result, and converting the function description text into the voice introduction information.

### Claim 9 (depends on 8)

The interface interaction method according to claim 8, wherein generating the function description text corresponding to the interface display control based on the acting object and the acting result comprises:
generating a control keyword corresponding to the interface display control based on the acting object and the acting result; and
generating the function description text corresponding to the interface display control based on the control keyword and the preset description prompt information.

### Claim 10 (depends on 1)

The interface interaction method according to claim 1, wherein acquiring, in response to the object trigger operation being input on the interface object, the triggered interface object as the target object comprises:
determining, in response to a touch selection operation being input on the interface object, the interface object selected based on the touch selection operation as the target object; and/or,
determining, in response to an object gaze operation being input on the interface object, a gaze fixation area, and determining the interface object corresponding to the gaze fixation area as the target object.

### Claim 11 (depends on 1)

The interface interaction method according to claim 1, wherein determining the voice introduction information corresponding to the target object comprises:
acquiring, in response to detecting text introduction information corresponding to the target object, the text introduction information, and converting the text introduction information into the voice introduction information; and
generating the voice introduction information based on the target object in response to not detecting the text introduction information corresponding to the target object.

### Claim 12

An electronic device, comprising
one or more processors; and
a storage, configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to:
display a target interface, wherein at least one interface object is displayed on the target interface, and the interface object comprises an interface display resource and/or an interface display control;
acquire, in response to an object trigger operation being input on an interface object, the triggered interface object as a target object; and
determine voice introduction information corresponding to the target object; and
play the voice introduction information.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein the one or more programs causing the one or more processors to determine the voice introduction information corresponding to the target object further cause the one or more processors to:
generate, in response to the target object being the interface display resource, the voice introduction information corresponding to the target object based on resource content of the interface display resource.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the one or more programs causing the one or more processors to generate the voice introduction information corresponding to the target object based on the resource content of the target object further cause the one or more processors to:
generate an object keyword corresponding to the target object based on the resource content of the target object; and
generate, based on the object keyword and preset description prompt information, a content introduction text corresponding to the target object, and convert the content introduction text into the voice introduction information.

### Claim 15 (depends on 14)

The electronic device according to claim 14, wherein the interface display resource comprises an image resource, and the one or more programs causing the one or more processors to generate the object keyword corresponding to the target object based on the resource content of the target object further cause the one or more processors to:
input the image resource to a content recognition model for content recognition to obtain the object keyword corresponding to the image resource, wherein the content recognition model is obtained by training a neural network model based on a sample image and an expected keyword corresponding to the sample image, and the expected keyword is a keyword associated with image content of the sample image.

### Claim 16 (depends on 14)

The electronic device according to claim 14, wherein the interface display resource comprises a video resource, and the one or more programs causing the one or more processors to generate the object keyword corresponding to the target object based on the resource content of the target object further cause the one or more processors to:
acquire a plurality of key frames in the video resource, and respectively perform content recognition on each key frame to obtain frame content keywords; and
determine the object keyword corresponding to the video resource based on association relationships among the plurality of key frames and the frame content keywords corresponding to the key frames.

### Claim 17 (depends on 14)

The electronic device according to claim 14, wherein the one or more programs causing the one or more processors to generate, based on the object keyword and the preset description prompt information, the content introduction text corresponding to the target object further cause the one or more processors to:
input the object keyword and the preset description prompt information to a text generation model to generate the content introduction text corresponding to the target object, wherein the text generation model is obtained by training a deep learning model based on sample keywords, sample prompt information, and an expected introduction text.

### Claim 18 (depends on 12)

The electronic device according to claim 12, wherein the one or more programs causing the one or more processors to determine the voice introduction information corresponding to the target object further cause the one or more processors to:
generate, in response to the target object being the interface display control, the voice introduction information corresponding to the interface display control based on function associated information corresponding to the interface display control.

### Claim 19 (depends on 18)

The electronic device according to claim 18, wherein the function associated information comprises an acting object corresponding to the interface display control and an acting result generated after the interface display control acts on the acting object; and
the one or more programs causing the one or more processors to generate the voice introduction information corresponding to the interface display control based on the function associated information corresponding to the interface display control further cause the one or more processors to:
determine the acting object corresponding to the interface display control and the acting result generated after the interface display control acts on the acting object;
generate a function description text corresponding to the interface display control based on the acting object and the acting result, and convert the function description text into the voice introduction information.

### Claim 20

A non-transitory storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to:
display a target interface, wherein at least one interface object is displayed on the target interface, and the interface object comprises an interface display resource and/or an interface display control;
acquire, in response to an object trigger operation being input on an interface object, the triggered interface object as a target object; and
determine voice introduction information corresponding to the target object; and
play the voice introduction information.
