# US 6,295,391 B1 · Automatic data routing via voice command annotation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 6,295,391 B1
- Document ID: US6295391B1
- Status: Granted
- Kind: B1
- Application number: 09/219,972
- Filed: 1998-12-23
- Granted: 2001-09-25
- Earliest claimed: 1998-02-19
- Assignees: Hewlett-Packard Company
- Inventors: Michael L Rudd; Thomas C Oliver
- Web page: https://patentgrep.com/patent/US6295391B1

## CPC

- H04N 1/32128: {attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image (for information embedded in the image H04N1/32144)}
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G10L 19/00: Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis (in musical instruments G10H)
- H04M 3/42204: {Arrangements at the exchange for service or number selection by voice (at the terminal H04M1/27)}
- H04N 1/00127: {Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture}
- H04N 1/00204: {with a digital computer or a digital computer system, e.g. an internet server (programmed control between transmitter and receiver or between image input and image output device H04N1/32561)}
- H04N 1/00326: {with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus (arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00)}
- H04N 1/00352: {Input means (H04N1/00411 takes precedence; input arrangements for computers G06F3/00)}
- H04N 1/32122: {in a separate device, e.g. in a memory or on a display separate from image data}
- H04N 2201/0081: Image reader (H04N2201/0091 - H04N2201/0094 take precedence)
- H04N 2201/0084: Digital still camera
- H04N 2201/3222: of processing required or performed, e.g. forwarding, urgent or confidential handling
- H04N 2201/3261: of multimedia information, e.g. a sound signal
- H04N 2201/3264: of sound signals
- H04N 2201/3274: Storage or retrieval of prestored additional information
- H04N 2201/3278: Transmission
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

A system for automatically routing data captured by devices, such as digital cameras, personal digital assistants, laptop computers, or portable document scanners, through voice command annotation is disclosed. After capturing an image or document with the device, a voice disposition command is made by a user speaking into a voice pickup component in the device to control the disposition of the captured image or document. Commands such as “fax”, “e-mail”, “print and save” are typical. The voice input is stored in the device in a voice command annotation file and is associated with the image or document data file. Upon copying the image or document data file to a host computer system, the voice command annotation file is automatically copied with it. The host computer system processes the voice command annotation file and executes one or more instructions associated with the command or commands contained in the file.

## Claims

### Claim 1

A voice command annotation method for a device, said method comprising the steps of:
(a) capturing a voice disposition command with a voice pickup component in said device;
(b) storing said voice disposition command in a voice command annotation file;
(c) associating said voice command annotation file with a data file in said device;
(d) copying said data file and said voice command annotation file from said device to a host computer system;
(e) processing said voice command annotation file in said host computer system to identify at least one function associated with said voice disposition command, comprising the steps of
(e1) generating a first recognition pattern representative of said voice disposition command,
(e2) comparing said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory in said host computer system,
(e3) when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, retrieving a command number associated with said at least one recognition pattern, and
(e4) accessing, through said command number, said at least one instruction, wherein said at least one instruction is linked to said command number; and
(f) executing at least one instruction associated with said at least one function, wherein said at least one instruction acts upon said data file.

### Claim 2 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a1), step (b) further comprises the step (b1), step (e) replaces step (e), and step (f) further comprises the step (f1):
(a1) capturing a plurality of additional voice disposition commands with said voice pickup component in said device;
(b1) storing said plurality of additional voice disposition commands in said voice command annotation file;
(e) processing said voice command annotation file in said host computer system to identify a plurality of functions, wherein one of each of said plurality of functions is associated with one of each of said plurality of additional voice disposition commands, said processing comprising the steps of; and
(f1) executing a plurality of additional at least one instructions, wherein one of each of said plurality of additional at least one instruction is associated with said one of each of said plurality of additional functions, and further wherein each of said plurality of additional at least one instruction acts upon said data file.

### Claim 3 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the steps (a1) through (a3), step (b) further comprises the step (b1), and step (c) further comprises the step (c1):
(a1) outputting, from said voice pickup component, said voice disposition command as an analog signal to an analog-to-digital converter in said device;
(a2) converting said analog signal in said analog-to-digital converter to a digital signal;
(a3) outputting said digital signal from said analog-to-digital converter to said controller;
(b1) storing, by said controller, said digital signal in said voice command annotation file in a static memory in said device, wherein said voice command annotation file is an audio file format; and
(c1) associating said data file and said voice command annotation file by giving said data file and said voice command annotation file a same file name, and giving said data file a first file extension name, and giving said voice command annotation file a second file extension name.

### Claim 4 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein step (e2) further comprises the steps (e2a) through (e2c):
(e2a) when said first recognition pattern does not match said at least one recognition pattern for said at least one function stored in said function recognition table, outputting a first message in said host computer system indicating that no match was found, and outputting a second message in said host computer system prompting for input to listen to said voice command annotation file;
(e2b) when said input to listen from step (e2a) is received, calling audio file conversion software to process said voice command annotation file for audio output, and outputting a third message in said host computer system prompting for input for manual operations; and
(e2c) when said input for manual operations from step (e2b) is received, suspending said voice command annotation method until manual operations are performed.

### Claim 5 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein said command number is a command text.

### Claim 6 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein said device is a portable scanner device.

### Claim 7 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein said device is a digital camera.

### Claim 8 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a0) performed before step (a):
(a0) receiving a voice control command in said device to enable said device to capture said voice disposition command with said voice pickup component.

### Claim 9 (depends on 1)

The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a0) performed before step (a), and step (b) further comprises the step (b1):
(a0) receiving button down input in said controller from an annotation control button on said device, wherein said device is enabled to capture said voice disposition command with said voice pickup component;
(b1) receiving button up input in said controller from said annotation control button on said device, wherein said device is disabled from capturing said voice disposition command with said voice pickup component.

### Claim 10

A voice command annotation method for a device, said method comprising the steps of:
(a) capturing a voice disposition command with a voice pickup component in said device;
(b) processing said voice disposition command with a controller in said device comprising the steps of
(b1) calling, by said controller, voice analysis software from a memory in said device,
(b2) generating a first recognition pattern, wherein said first recognition pattern is representative of said voice disposition command,
(b3) comparing said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory in said device,
(b4) when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, retrieving a command number stored in said memory associated with said at least one recognition pattern;
(c) storing said voice disposition command processed with said controller in a voice command annotation file, comprising the step of
(c1) storing, by said controller, said command number in said voice command annotation file in said memory in said device;
(d) associating said voice command annotation file with a data file in said device;
(e) copying said data file and said voice command annotation file from said device to a host computer system;
(f) processing said voice command annotation file in said host computer system to identify at least one function associated with said command number; and
(g) executing at least one instruction associated with said at least one function, wherein said at least one instruction acts upon said data file.

### Claim 11 (depends on 10)

The voice command annotation method for a device according to claim 10 wherein step (b3) further comprises the steps (b3a) and (b3b):
(b3a) when said first recognition pattern does not match said at least one recognition pattern for said at least one function stored in said function recognition table, outputting a first message in said device indicating that no match was found; and
(b3b) outputting a second message in said device prompting for input to repeat said voice disposition command or to cancel.

### Claim 12 (depends on 10)

The voice command annotation method for a device according to claim 10 wherein said command number is a command text.

### Claim 13 (depends on 10)

The voice command annotation method for a device according to claim 10 wherein step (e) further comprises the steps (e0) performed before step (e) and step (e1) performed after step (e), and step (f) further comprises the steps (f1) through (f2):
(e0) loading file transfer software in a memory in said host computer system;
(e1) receiving, through said file transfer software, said data file and said voice command annotation file into said memory in said host computer system;
(f1) retrieving said command number from said voice command annotation file with said file transfer software; and
(f2) accessing said at least one instruction, stored in said function recognition table in said memory, through said command number, wherein said at least one instruction is linked to said command number.

### Claim 14 (depends on 10)

The voice command annotation method for a device according to claim 10 wherein step (a) further comprises the steps (a0a) through (a0j) performed before step (a):
(a0a) loading voice analysis software into a memory in a host computer system;
(a0b) connecting said device to said host computer system;
(a0c) selecting a predetermined function, with said voice analysis software, for training and voice analysis of at least one word for invoking said predetermined function;
(a0d) capturing a plurality of voice inputs of said at least one word in said voice pickup component of said device;
(a0e) processing said plurality of voice inputs into a plurality of digital signals in said device;
(a0f) sending said plurality of digital signals from said device to said host computer system;
(a0g) analyzing said plurality of digital signals with said voice analysis software;
(a0h) developing said at least one recognition pattern from said analysis of said plurality of digital signals with said voice analysis software;
(a0i) storing said at least one recognition pattern in said function recognition table in said memory in said host computer system; and
(a0j) transferring said at least one recognition pattern in said function recognition table in said memory in said host computer system for storage in said function recognition table in said static memory in said device, wherein said command number is associated with said at least one recognition pattern.

### Claim 15 (depends on 14)

The voice command annotation method for a device according to claim 14 further comprising the steps of:
(a0k) repeating steps (a0c) through (a0j) for a plurality of predetermined functions, wherein a plurality of recognition patterns are developed from a plurality of said plurality of voice inputs for a plurality of said at least one words, and further wherein said plurality of recognition patterns are stored in said function recognition table in said static memory in said device, wherein each of said plurality of recognition patterns are associated with one of a plurality of command numbers stored in said function recognition table in said static memory.

### Claim 16 (depends on 14)

The voice command annotation method for a device according to claim 14 wherein step (a0a) is replaced by the new step (a0a), steps (a0b), (a0f), and (a0i) are eliminated, and step (a0j) is replaced by the new step (a0j):
(a0a) accessing voice analysis software in said static memory in said device; and
(a0j) storing said at least one recognition pattern in said function recognition table in said static memory in said device, wherein said at least one recognition pattern is associated with said command number stored in said function recognition table in said static memory.

### Claim 17 (depends on 16)

The voice command annotation method for a device according to claim 16 further comprising the steps of:
(a0k) repeating steps (a0c) through (a0j) for a plurality of predetermined functions, wherein a plurality of recognition patterns are developed from a plurality of said plurality of voice inputs for a plurality of said at least one words, and further wherein said plurality of recognition patterns are stored in said function recognition table in said static memory in said device, wherein each of said plurality of recognition patterns are associated with one of a plurality of command numbers stored in said function recognition table in said static memory.

### Claim 18

A device with voice command annotation comprising:
a voice pickup component for capturing a voice disposition command, and for outputting said voice disposition command as an analog signal;
an analog-to-digital converter, connected to said voice pickup component, for converting said analog signal into a digital signal;
a controller, connected to said analog-to-digital converter, for receiving said digital signal from said analog-to-digital converter and storing said digital signal in a voice command annotation file; and
a host connection, connected to said controller, for establishing a connection with a host computer system, wherein said host computer system comprises
a memory,
voice analysis software loaded into said memory, and
file transfer software loaded into said memory, wherein said file transfer software processes said voice command annotation file by calling said voice analysis software to perform a plurality of frequency domain transforms on said digital signal in said voice command annotation file, generating a first recognition pattern, wherein said first recognition pattern is a spectral transform of said digital signal, and further wherein said file transfer software compares said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory, wherein when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, said file transfer software retrieves a command number associated with said at least one recognition pattern and accesses through said command number said at least one instruction, wherein said at least one instruction is linked to said command number, and further wherein said file transfer software executes said at least one instruction.

### Claim 19 (depends on 18)

The device with voice command annotation according to claim 18 further comprising:
an annotation control button connected to said controller, wherein said device is enabled to capture said voice disposition command with said voice pickup component when button down input from said annotation control button is received by said controller, and until button up input from said annotation control button is received by said controller, said voice disposition command is captured by said voice pickup component between said button down input and said button up input, and is converted to said analog signal, and then converted to said digital signal by said analog-to-digital converter, and stored as said voice annotation file in said static memory in said device, and associated with said data file stored in said static memory.

### Claim 20 (depends on 18)

The device with voice command annotation according to claim 18 wherein said controller associates said voice command annotation file to said data file by giving said data file and said voice command annotation file a same file name, and giving said data file a first file extension name, and giving said voice command annotation file a second file extension name.

### Claim 21 (depends on 18)

The device with voice command annotation according to claim 18 wherein said device is a portable scanner device.

### Claim 22 (depends on 18)

The device with voice command annotation according to claim 18 wherein said device is a digital camera.

### Claim 23

A device with voice command annotation comprising:
a voice pickup component for capturing a voice disposition command;
a controller, connected to said voice pickup component, for deriving a first recognition pattern from said voice disposition command; and
a static memory, connected to said controller, for storing at least one recognition pattern in a function recognition table, and for storing a command number in said function recognition table associated with said at least one recognition pattern;
wherein, said controller compares said first recognition pattern with said at least one recognition pattern in said function recognition table stored in said static memory, and further wherein, when said first recognition pattern and said at least one recognition pattern match, said controller retrieves said command number from said function recognition table and stores said command number in a voice command annotation file in said static memory.

### Claim 24 (depends on 23)

The device with voice command annotation according to claim 23 further comprising:
an analog-to-digital converter, connected to said voice pickup component;
wherein said voice disposition command is output to said analog-to-digital converter as an analog signal, and said analog-to-digital converter converts said analog signal into a digital signal, and further wherein said analog-to-digital converter outputs said digital signal to said controller, and said controller converts said digital signal into said first recognition pattern.

### Claim 25 (depends on 24)

The device with voice command annotation according to claim 24 wherein said controller converts said digital signal into said first recognition pattern by performing a plurality of frequency domain transforms on said digital signal, generating said first recognition pattern, wherein said first recognition pattern is a spectral transform of said digital signal.

### Claim 26 (depends on 23)

The device with voice command annotation according to claim 23 further comprising.
a host connection connected to said controller; and
a host computer system having a communications interface, wherein said host connection of said device is connected to said communications interface of said host computer system, allowing communication to occur between said device and said host computer system;
wherein a plurality of voice inputs of at least one word to be used for invoking a predetermined function are captured by said voice pickup component, processed in said device into a plurality of digital signals, and said plurality of digital signals are transferred from said device to said host computer system, and further wherein said plurality of digital signals are analyzed, with voice analysis software loaded into a memory in said host computer system, generating a recognition pattern for said at least one word from said plurality of digital signals, wherein said recognition pattern is associated with said command number for said predetermined function, and further wherein said recognition pattern and said command number are transferred from said host computer system to said device and stored in said function recognition table in said static memory in said device.

### Claim 27 (depends on 23)

The device with voice command annotation according to claim 23 wherein said device is a portable scanner device.

### Claim 28 (depends on 23)

The device with voice command annotation according to claim 23 wherein said device is a digital camera.
