# US 12,770,418 B2 · Digital data tagging apparatus, tagging method, program, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,770,418 B2
- Document ID: US12770418B2
- Status: Granted
- Kind: B2
- Application number: 18/468,410
- Filed: 2023-09-15
- Published: 2024-01-04
- Granted: 2026-10-20
- Earliest claimed: 2021-03-31
- Assignees: FUJIFILM Corporation
- Applicants: FUJIFILM Corporation
- Inventors: Mayuko Ikuta
- Web page: https://patentgrep.com/patent/US12770418B2

## CPC

- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2015/088: {Word spotting}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

It is possible for a user to easily assign a desired tag by using a voice, regardless of homonyms and synonyms having different expressions. In a digital data tagging apparatus, a tagging method, a program, and a recording medium according to an embodiment of the present invention, a digital data acquisition unit acquires digital data to which a tag is assigned, and an audio data acquisition unit acquires audio data related to the digital data. A word/phrase extraction unit extracts a word/phrase from the audio data, a tag candidate determination unit determines one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among a plurality of tag candidates, which are stored in advance in a tag candidate storage unit, as a first tag candidate, and a tag assignment unit assigns at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag.

## Claims

### Claim 1

A digital data tagging apparatus comprising:
a processor; and
a tag candidate memory that stores a plurality of tag candidates in advance,
wherein the processor is configured to:
acquire digital data to which a tag is assigned;
acquire audio data related to the digital data;
extract a word/phrase from the audio data;
determine one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among the plurality of tag candidates as a first tag candidate; and
assign at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag,
wherein the digital data is image data, and
wherein the processor is configured to:
recognize a subject included in an image corresponding to the image data;
determine a word/phrase, which represents a name of the subject corresponding to the word/phrase and is different from the word/phrase, as a second tag candidate; and
display the second tag candidate on the display by including the second tag candidate in the tag candidate group.

### Claim 2 (depends on 1)

The digital data tagging apparatus according to claim 1, further comprising:
a display,
wherein the processor is configured to:
convert the audio data into text data to extract one or more words/phrases from the text data;
display a text corresponding to the text data on the display;
determine the first tag candidate based on a word/phrase selected by a user from among the one or more words/phrases included in the text displayed on the display;
display the tag candidate group on the display; and
assign at least one selected by the user from the tag candidate group displayed on the display to the digital data as the tag.

### Claim 3 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to include a first synonym of which a degree of similarity in pronunciation to the word/phrase is equal to or more than the first threshold value among synonyms of the word/phrase in the first tag candidate.

### Claim 4 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to include a second synonym of which a degree of similarity in meaning to the word/phrase is equal to or more than the first threshold value among synonyms of the word/phrase in the first tag candidate.

### Claim 5 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to include a first synonym of which a degree of similarity in pronunciation to the word/phrase is equal to or more than the first threshold value and a second synonym of which a degree of similarity in meaning to the word/phrase is equal to or more than the first threshold value among synonyms of the word/phrase in the first tag candidate.

### Claim 6 (depends on 5)

The digital data tagging apparatus according to claim 5,
wherein the processor is configured to determine the number of the first synonyms and the number of the second synonyms to be included in the first tag candidate such that the number of the first synonyms is larger than the number of the second synonyms.

### Claim 7 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to include a homonym of the word/phrase in the first tag candidate.

### Claim 8 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to display a word/phrase or a tag candidate, which is previously selected by the user, from the tag candidate group with higher priority than a word/phrase or a tag candidate, which is not previously selected by the user.

### Claim 9 (depends on 8)

The digital data tagging apparatus according to claim 8,
wherein the processor is configured to display a word/phrase or a tag candidate, which is previously selected many times, among the words/phrases or the tag candidates, which are previously selected by the user, with higher priority than a word/phrase or a tag candidate, which is previously selected few times.

### Claim 10 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to:
acquire information on an imaging position of an image corresponding to the image data;
determine a tag candidate, which represents a place name that is located within a range equal to or less than a fourth threshold value from the imaging position of the image and has a degree of similarity of pronunciation to the word/phrase being equal to or more than a third threshold value, from among the plurality of tag candidates as a fifth tag candidate based on the information on the imaging position of the image; and
display the fifth tag candidate on the display by including the fifth tag candidate in the tag candidate group.

### Claim 11 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to:
extract a place name from audio data including the place name;
in a case in which there are a plurality of locations of the place name, determine a tag candidate consisting of a combination of the place name and each of the plurality of locations as an eighth tag candidate; and
display the eighth tag candidate on the display by including the eighth tag candidate in the tag candidate group.

### Claim 12 (depends on 2)

The digital data tagging apparatus according to claim 2,
wherein the processor is configured to:
extract at least one of a sound onomatopoeic word or a voice onomatopoeic word corresponding to an environmental sound included in the audio data from the audio data;
determine at least one of the sound onomatopoeic word or the voice onomatopoeia word as a ninth tag candidate; and
display the ninth tag candidate on the display by including the ninth tag candidate in the tag candidate group.

### Claim 13 (depends on 1)

The digital data tagging apparatus according to claim 1, further comprising:
an audio data memory that stores the audio data,
wherein the processor is configured to store the audio data having information on association with the digital data in the audio data memory.

### Claim 14 (depends on 1)

The digital data tagging apparatus according to claim 1,
wherein the digital data is moving image data, and
the processor is configured to extract the word/phrase from audio data included in the moving image data.

### Claim 15

A digital data tagging apparatus comprising:
a processor;
a display; and
a tag candidate memory that stores a plurality of tag candidates in advance,
wherein the processor is configured to:
acquire digital data to which a tag is assigned;
acquire audio data related to the digital data;
extract a word/phrase from the audio data;
determine one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among the plurality of tag candidates as a first tag candidate; and
assign at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag,
wherein the processor is configured to:
convert the audio data into text data to extract one or more words/phrases from the text data;
display a text corresponding to the text data on the display;
determine the first tag candidate based on a word/phrase selected by a user from among the one or more words/phrases included in the text displayed on the display;
display the tag candidate group on the display; and
assign at least one selected by the user from the tag candidate group displayed on the display to the digital data as the tag,
wherein the digital data is image data, and
wherein the processor is configured to:
recognize at least one of a subject or a scene included in an image corresponding to the image data; and
in a case in which there are a predetermined number or more of the tag candidates of which the degree of association with the word/phrase is equal to or more than the first threshold value among the plurality of tag candidates, determine only a tag candidate of which a degree of association with at least one of the subject or the scene is equal to or more than a second threshold value from among the predetermined number or more of the tag candidates as the first tag candidate.

### Claim 16

A digital data tagging apparatus comprising:
a processor;
a display; and
a tag candidate memory that stores a plurality of tag candidates in advance,
wherein the processor is configured to:
acquire digital data to which a tag is assigned;
acquire audio data related to the digital data;
extract a word/phrase from the audio data;
determine one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among the plurality of tag candidates as a first tag candidate; and
assign at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag,
wherein the processor is configured to:
convert the audio data into text data to extract one or more words/phrases from the text data;
display a text corresponding to the text data on the display;
determine the first tag candidate based on a word/phrase selected by a user from among the one or more words/phrases included in the text displayed on the display;
display the tag candidate group on the display; and
assign at least one selected by the user from the tag candidate group displayed on the display to the digital data as the tag,
wherein the digital data is image data, and
wherein the processor is configured to:
recognize at least one of a subject or a scene included in an image corresponding to the image data;
determine a tag candidate of which a degree of association with at least one of the subject or the scene is equal to or more than a second threshold value and a degree of similarity of pronunciation to the word/phrase is equal to or more than a third threshold value, from among the plurality of tag candidates as a third tag candidate; and
display the third tag candidate on the display by including the third tag candidate in the tag candidate group.

### Claim 17

A digital data tagging apparatus comprising:
a processor;
a display; and
a tag candidate memory that stores a plurality of tag candidates in advance,
wherein the processor is configured to:
acquire digital data to which a tag is assigned;
acquire audio data related to the digital data;
extract a word/phrase from the audio data;
determine one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among the plurality of tag candidates as a first tag candidate; and
assign at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag,
wherein the processor is configured to:
convert the audio data into text data to extract one or more words/phrases from the text data;
display a text corresponding to the text data on the display;
determine the first tag candidate based on a word/phrase selected by a user from among the one or more words/phrases included in the text displayed on the display;
display the tag candidate group on the display; and
assign at least one selected by the user from the tag candidate group displayed on the display to the digital data as the tag,
wherein the digital data is image data, and a person tag, which represents a name of a subject included in an image corresponding to the image data, is assigned to the image data by a first user, and
wherein the processor is configured to:
recognize the subject included in the image;
extract the name of the subject from audio data including a voice of a second user who is different from the first user and utters the name of the subject for the image;
determine one or more tag candidates of which a degree of association with the name of the subject is equal to or more than the first threshold value as the first tag candidate to determine the person tag as a fourth tag candidate in a case in which the first tag candidate and the person tag are different from each other; and
display the fourth tag candidate on the display by including the fourth tag candidate in the tag candidate group.

### Claim 18 (depends on 2)

A digital data tagging apparatus according to claim 2, comprising:
a processor;
a display; and
a tag candidate memory that stores a plurality of tag candidates in advance,
wherein the processor is configured to:
acquire digital data to which a tag is assigned;
acquire audio data related to the digital data;
extract a word/phrase from the audio data;
determine one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among the plurality of tag candidates as a first tag candidate; and
assign at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag,
wherein the processor is configured to:
convert the audio data into text data to extract one or more words/phrases from the text data;
display a text corresponding to the text data on the display;
determine the first tag candidate based on a word/phrase selected by a user from among the one or more words/phrases included in the text displayed on the display;
display the tag candidate group on the display; and
assign at least one selected by the user from the tag candidate group displayed on the display to the digital data as the tag,
wherein the digital data is image data, and
wherein the processor is configured to:
recognize a subject included in an image corresponding to the image data;
acquire information on an imaging position of the image;
extract a name of the subject from audio data including the name of the subject included in the image;
in a case in which the name of the subject and an actual name of the subject located within a range equal to or less than a fourth threshold value from the imaging position of the image are different from each other, determine the actual name of the subject as a sixth tag candidate based on the information on the imaging position of the image; and
display the sixth tag candidate on the display by including the sixth tag candidate in the tag candidate group.

### Claim 19 (depends on 18)

The digital data tagging apparatus according to claim 18,
wherein the processor is configured to:
in a case in which the sixth tag candidate is selected by the user from the tag candidate group including the sixth tag candidate displayed on the display for one image data, for each of a plurality of image data corresponding to a plurality of images captured within a predetermined period, determine an actual name corresponding to a subject included in each of the plurality of images as a seventh tag candidate; and
assign the seventh tag candidate corresponding to each of the plurality of image data to each of the plurality of image data as the tag.

### Claim 20

A digital data tagging method comprising:
acquiring digital data to which a tag is assigned via a processor;
acquiring audio data related to the digital data via the processor;
extracting a word/phrase from the audio data via the processor;
determining one or more tag candidates of which a degree of association with the word/phrase is equal to or more than a first threshold value from among a plurality of tag candidates, which are stored in advance in a tag candidate storage unit, as a first tag candidate via the processor; and
assigning at least one of a tag candidate group including the word/phrase and the first tag candidate to the digital data as the tag via the processor,
wherein the digital data is image data,
the digital data tagging method further comprising:
recognizing a subject included in an image corresponding to the image data via the processor;
determining a word/phrase, which represents a name of the subject corresponding to the word/phrase and is different from the word/phrase, as a second tag candidate via the processor; and
displaying the second tag candidate on the display by including the second tag candidate in the tag candidate group via the processor.

### Claim 21 (depends on 20)

A non-transitory computer-readable recording medium on which a program for causing a computer to execute each of the steps of the digital data tagging method according to claim 20 is recorded.