# US 2014/0373163 A1 · Filtering Confidential Information In Voice and Image Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0373163 A1
- Document ID: US20140373163A1
- Status: Published application
- Kind: A1
- Application number: 14/311,076
- Filed: 2014-06-20
- Published: 2014-12-18
- Earliest claimed: 2010-12-08
- Assignees: Nuance Communications, Inc.
- Applicants: Nuance Communications, Inc.
- Inventors: Seiji Abe; Mitsuru Shioya; Shigeki Takeuchi; Daisuke Tomoda
- Web page: https://patentgrep.com/patent/US20140373163A1

## CPC

- G06F 21/60: Protecting data
- G06F 16/7834: {using audio features}
- G06F 16/7844: {using original textual content or text extracted from visual content or transcript of audio data}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Confidential information included in image and voice data is filtered in an apparatus that includes an extraction unit for extracting a character string from an image frame, and a conversion unit for converting audio data to a character string. The apparatus also includes a determination unit for determining, in response to contents of a database, whether at least one of the image frame and the audio data include confidential information. The apparatus also includes a masking unit for concealing contents of the image frame by masking the image frame in response to determining that the image frame includes confidential information, and for making the audio data inaudible by masking the audio data in response to determining that the audio data includes confidential information. The playback unit included in the apparatus is for playing back the image frame and the audio data.

## Claims

### Claim 1

An information processing apparatus for filtering confidential information included in image and voice data, the information processing apparatus comprising:
an extraction unit configured for extracting a character string from an image frame;
a conversion unit configured for converting audio data to a character string; a database configured for registering determination information for determining confidential information;
a determination unit configured for determining whether at least one of the image frame and the audio data include confidential information by determining, responsive to the database, whether at least one of the character string extracted from the image frame and the character string converted from the audio data includes confidential information;
a masking unit configured for concealing contents of the image frame by masking the image frame, the concealing contents of the image frame responsive to determining that the image frame includes confidential information, the masking unit further configured for making the audio data inaudible by masking the audio data, the making the audio data inaudible responsive to determining that the audio data includes confidential information; and
a playback unit configured for playing back the image frame and the audio data.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1, wherein, in the database, character strings that may co-occur with confidential information and respective confidential information co-occurrence rates of the character strings are registered in association with each other as the determination information, and
the determination unit determines that at least one of the image frame and the audio data include confidential information responsive to determining that a confidential information co-occurrence rate of a character string matching the character string extracted from at least one of the image frame and the character string converted from the audio data exceeds a predetermined threshold value.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1, further comprising:
an updating unit configured for updating the determination information registered in the database,
wherein the updating unit is configured for changing a confidential information co-occurrence rate registered in the database in association with the character string determined as including confidential information to a confidential information co-occurrence rate indicating non-confidential information.

### Claim 4 (depends on 3)

The information processing apparatus according to claim 3, wherein the updating unit is further configured for registering a character string selected by a user in the database in association with a confidential information co-occurrence rate indicating confidential information.

### Claim 5 (depends on 3)

The information processing apparatus according to claim 3, wherein the updating unit is further configured for changing a confidential information co-occurrence rate registered in the database in association with the character string, which is determined as including no confidential information and selected by a user, to a confidential information co-occurrence rate indicating confidential information.

### Claim 6 (depends on 3)

The information processing apparatus according to claim 3, further comprising:
a storage unit configured for storing the image frame and the audio data having been played back,
wherein the updating unit is further configured for changing, when the playback unit plays back the image frame and the audio data stored in the storage unit, a confidential information co-occurrence rate registered in the database in association with the character string determined as including confidential information to a confidential information co-occurrence rate indicating non-confidential information.

### Claim 7 (depends on 6)

The information processing apparatus according to claim 6, wherein the updating unit is further configured for registering, when the playback unit plays back the image frame and the audio data stored in the storage unit, a character string selected by a user in the database in association with a confidential information co-occurrence rate indicating confidential information.

### Claim 8 (depends on 6)

The information processing apparatus according to claim 6, wherein the updating unit is configured for changing, when the playback unit plays back the image frame and the audio data stored in the storage unit, a confidential information co-occurrence rate registered in the database in association with the character string, which is determined as including no confidential information and selected by a user, to a confidential information co-occurrence rate indicating confidential information.

### Claim 9

A computer implemented method for filtering confidential information included in image and voice data, the method comprising:
extracting a character string from an image frame;
converting audio data to a character string;
determining whether at least one of the image frame and the audio data includes confidential information by determining, responsive to a database that registers determination information for determining confidential information, whether at least one of the character string extracted from the image frame and the character string converted from the audio data includes confidential information;
concealing contents of the image frame by masking the image frame, the concealing contents of the image frame responsive to determining that the image frame includes confidential information;
making the audio data inaudible by masking the audio data, the making the audio data inaudible responsive to determining that the audio data includes confidential information; and playing back the image frame and the audio data.

### Claim 10 (depends on 9)

The method according to claim 9, wherein, in the database, character strings that may co-occur with confidential information and respective confidential information co-occurrence rates of the character strings are registered in association with each other as the determination information, and
the determining includes, upon determining that a confidential information co-occurrence rate of a character string matching the character string extracted from the image frame exceeds a predetermined threshold that the image frame includes confidential information, and upon determining that a confidential co-occurrence rate of a character string matching the character string converted from the audio data exceeds the predetermined threshold value that the audio data includes confidential information.

### Claim 11 (depends on 9)

The method according to claim 9, further comprising:
updating the determination information registered in the database,
wherein the updating includes changing a confidential information co-occurrence rate registered in the database in association with the character string determined as including confidential information to a confidential information co-occurrence rate indicating non-confidential information.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the updating further includes registering a character string selected by a user in the database in association with a confidential information co-occurrence rate indicating confidential information.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the updating further includes changing a confidential information co-occurrence rate registered in the database in association with the character string, which is determined as including no confidential information and selected by a user, to a confidential information co-occurrence rate indicating confidential information.

### Claim 14

A computer program product for filtering confidential information included in image and voice data, the computer program product comprising:
a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:
computer readable program code configured for:
extracting a character string from an image frame;
converting audio data to a character string;
determining whether at least one of the image frame and the audio data includes confidential information by determining, responsive to a database that registers determination information for determining confidential information, whether at least one of the character string extracted from the image frame and the character string converted from the audio data includes confidential information;
concealing contents of the image frame by masking the image frame, the concealing contents of the image frame responsive to determining that the image frame includes confidential information;
making the audio data inaudible by masking the audio data, the making the audio data inaudible responsive to determining that the audio data includes confidential information; and
playing back the image frame and the audio data.

### Claim 15 (depends on 14)

The computer program product according to claim 14, wherein, in the database, character strings that may co-occur with confidential information and respective confidential information co-occurrence rates of the character strings are registered in association with each other as the determination information, and
the determining includes, upon determining that a confidential information co-occurrence rate of a character string matching the character string extracted from the image frame exceeds a predetermined threshold that the image frame includes confidential information, and upon determining that a confidential co-occurrence rate of a character string matching the character string converted from the audio data exceeds the predetermined threshold value that the audio data includes confidential information.

### Claim 16 (depends on 14)

The computer program product according to claim 14, wherein the computer readable program code is further configured for:
updating the determination information registered in the database, wherein the updating includes changing a confidential information co-occurrence rate registered in the database in association with the character string determined as including confidential information to a confidential information co-occurrence rate indicating non-confidential information.

### Claim 17 (depends on 16)

The computer program product according to claim 16, wherein the updating further includes registering a character string selected by a user in the database in association with a confidential information co-occurrence rate indicating confidential information.

### Claim 18 (depends on 16)

The computer program product according to claim 16, wherein the updating further includes changing a confidential information co-occurrence rate registered in the database in association with the character string, which is determined as including no confidential information and selected by a user, to a confidential information co-occurrence rate indicating confidential information.
