# US 2019/0171352 A1 · Information processing apparatus, information processing system, and non-transitory computer readable medium storing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171352 A1 - Document ID: US20190171352A1 - Status: Published application - Kind: A1 - Application number: 16/197,374 - Filed: 2018-11-21 - Published: 2019-06-06 - Earliest claimed: 2017-12-01 - Assignees: Fuji Xerox Co., Ltd. - Applicants: Fuji Xerox Co., Ltd. - Inventors: Hirohito SHIBATA; Xingya XU - Web page: https://patentgrep.com/patent/US20190171352A1 ## CPC - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/0354: with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/0486: Drag-and-drop - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06V 30/32: Digital ink - G06V 30/347: {Sampling; Contour coding; Stroke extraction} - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An information processing apparatus includes a reception unit, a recognition unit, an extraction unit, and a processing unit. The reception unit receives drawing information which is information on a drawing operation by a user. The recognition unit recognizes whether or not a last part of the drawing operation is a gesture which the user designates a command, based on the drawing information. The extraction unit extracts a type of a figure drawn by the user before the user performs the gesture. The processing unit performs a process based on the extracted type of the figure. ## Claims ### Claim 1 An information processing apparatus comprising: a reception unit that receives drawing information which is information on a drawing operation by a user; a recognition unit that recognizes whether or not a last part of the drawing operation is a gesture which the user designates a command, based on the drawing information, an extraction unit that extracts a type of a figure drawn by the user before the user performs the gesture; and a processing unit that performs a process based on the extracted type of the figure. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the reception unit further receives sound information which is information on sound during the drawing operation by the user, and wherein the processing unit performs the process based on the sound information in addition to the type of the figure. ### Claim 3 (depends on 2) The information processing apparatus according to claim 2, wherein the processing unit performs the process based on the sound information before and after the gesture. ### Claim 4 (depends on 3) The information processing apparatus according to claim 3, wherein the extraction unit specifies target sound information to be processed by utilizing interruption of sound. ### Claim 5 (depends on 2) The information processing apparatus according to claim 2, wherein the processing unit performs a process of performing conversion into an object corresponding to the type of the figure. ### Claim 6 (depends on 5) The information processing apparatus according to claim 5, wherein the processing unit determines the object from a relationship between the type of the figure and the sound information. ### Claim 7 (depends on 6) The information processing apparatus according to claim 6, wherein the processing unit performs conversion into an icon meant by the sound information, as the object. ### Claim 8 (depends on 6) The information processing apparatus according to claim 6, wherein the processing unit performs conversion into a link connecting two icons, as the object. ### Claim 9 (depends on 6) The information processing apparatus according to claim 6, wherein the processing unit performs conversion into a text that is meant by the sound information, as the object, and disposes the text. ### Claim 10 (depends on 9) The information processing apparatus according to claim 9, wherein the processing unit performs conversion such that a text meant by the sound information is present within a frame corresponding to the figure, as the object. ### Claim 11 (depends on 9) The information processing apparatus according to claim 9, wherein, when sound information is not present, the processing unit performs conversion into a frame corresponding to the figure, as an object, and no text is present in the frame. ### Claim 12 (depends on 9) The information processing apparatus according to claim 9, wherein the processing unit disposes the text as the object along a drawing direction of a drawing operation when drawing the figure. ### Claim 13 (depends on 5) The information processing apparatus according to claim 5, wherein the processing unit determines a position to dispose the object and a size thereof, according to a position at which the figure is drawn and a size of the figure. ### Claim 14 (depends on 1) The information processing apparatus according to claim 1, wherein when the user draws a predetermined shape, the recognition unit recognizes that the drawing operation is the gesture. ### Claim 15 (depends on 14) The information processing apparatus according to claim 14, wherein the predetermined shape is a shape representing a closed space. ### Claim 16 (depends on 15) The information processing apparatus according to claim 15, wherein the recognition unit recognizes a shape representing a closed space having a size equal to or smaller than a predetermined size as the gesture. ### Claim 17 (depends on 1) The information processing apparatus according to claim 1, wherein recognition of the gesture performed by the recognition unit and extraction of a type of the figure performed by the extraction unit are performed from a drawing operation. ### Claim 18 (depends on 17) The information processing apparatus according to claim 17, wherein recognition of the gesture performed by the recognition unit and extraction of a type of the figure performed by the extraction unit are performed from a drawing operation of a one stroke drawing. ### Claim 19 (depends on 18) The information processing apparatus according to claim 18, wherein the extraction unit regards a part other than the drawing operation of the gesture as the figure. ### Claim 20 An information processing system comprising: a display device that displays an image based on a drawing operation by a user; and an information processing apparatus that generates display information of an image to be displayed on the display device based on the drawing operation, wherein the information processing apparatus includes a reception unit that receives drawing information which is information on a drawing operation by a user; a recognition unit that recognizes whether or not a last part of the drawing operation is a gesture which the user designates a command, based on the drawing information, an extraction unit that extracts a type of a figure drawn by the user before the user performs the gesture; and a processing unit that performs a process based on the extracted type of the figure. ### Claim 21 A non-transitory computer readable medium storing program causing a computer to execute a process, the process comprising: receiving drawing information which is information on a drawing operation by a user; recognizing whether or not a last part of the drawing operation is a gesture which the user designates a command, based on the drawing information; extracting a type of a figure drawn by the user before the user performs the gesture; and performing a process based on the extracted type of the figure.