# US 10,765,588 B2 · Information processing apparatus and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,765,588 B2 - Document ID: US10765588B2 - Status: Granted - Kind: B2 - Application number: 16/315,019 - Filed: 2017-06-23 - Published: 2019-10-10 - Granted: 2020-09-08 - Earliest claimed: 2016-08-05 - Assignees: Sony Corporation; Sony Mobile Communications Inc. - Applicants: Sony Corporation; Sony Mobile Communications Inc. - Inventors: Kumi Yashiro; Tetsuya Naruse; Junichi Kosaka; Yasumasa Suzuki; Hiroko Nishioka; Hitoshi Rikukawa; Kohei Takada; Masahiko Suzuki - Web page: https://patentgrep.com/patent/US10765588B2 ## CPC - A61H 3/06: Walking aids for blind persons (replacing direct visual perception by another kind of perception A61F9/08) - A61F 9/08: Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception - A61H 3/061: {with electronic detecting or guiding means} - G06T 7/00: Image analysis - G06V 10/443: {by matching or filtering} - G06V 10/757: {Matching configurations of points or features} - G08G 1/005: including pedestrian guidance indicator - G09B 21/00: Teaching, or communicating with, the blind, deaf or mute (audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B) - G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08) - A61H 2201/5007: computer controlled - A61H 2201/5023: Interfaces to the user - A61H 2201/5048: Audio interfaces, e.g. voice or music controlled - A61H 2201/5058: Sensors or detectors - A61H 2201/5097: wireless - A61H: PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY (electrotherapy, magnetotherapy, radiation therapy or ultrasound therapy A61N) - A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract There is provided a processing unit that includes a direction decision unit and a guide information generation unit. The direction decision unit determines a direction in which a person who behaves without a sense of sight walks. The guide information generation unit generates guide information for the person who behaves without the sense of sight to walk in the determined direction. The present technology is applicable, for example, to a smartphone or the like used by the person who behaves without the sense of sight. ## Claims ### Claim 1 An information processing apparatus, comprising: an image sensor configured to capture a current image at a specific time duration after a start of walk of a person; and a central processing unit (CPU) configured to: compare the current image and a reference image, wherein a time of capture of the reference image corresponds to the specific time duration; determine a first direction of the walk of the person based on the comparison of the current image and the reference image; and generate guide information for the person based on the determined first direction, wherein the guide information indicates the person to walk in the first direction. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the reference image is stored in advance, and the image sensor is further configured to capture the current image in real time. ### Claim 3 (depends on 2) The information processing apparatus according to claim 2, wherein the CPU is further configured to: extract a first feature point from the current image, wherein the first feature point corresponds to a second feature point of the reference image; compare the first feature point and the second feature point; and determine the first direction based on the comparison of the first feature point and the second feature point. ### Claim 4 (depends on 3) The information processing apparatus according to claim 3, wherein the CPU is further configured to change an area of the current image to match the first feature point with the second feature point, the area corresponds to the first feature point, and the first feature point corresponds to a color of a traffic light in the current image. ### Claim 5 (depends on 2) The information processing apparatus according to claim 2, wherein the CPU is further configured to: binarize the current image; and compare the reference image with the binarized current image. ### Claim 6 (depends on 2) The information processing apparatus according to claim 2, wherein the CPU is further configured to: calculate an azimuth in the current image; and calculate an azimuth of the image sensor based on the capture of the current image; determine a second direction in which the person faces, wherein the second direction is determined based on the azimuth of the current image and the azimuth of the image sensor; and determine the first direction based on the second direction. ### Claim 7 (depends on 2) The information processing apparatus according to claim 2, wherein the image sensor is further configured to capture a plurality of first images, the plurality of first images includes the current image, and the CPU is further configured to: estimate pieces of direction information in the plurality of first images; integrate the pieces of direction information; determine a second direction in which the person faces, wherein the second direction is determined based on the integration; and determine the first direction based on the second direction. ### Claim 8 (depends on 2) The information processing apparatus according to claim 2, wherein the CPU is further configured to register the reference image. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, wherein the CPU is further configured to: calculate an amount of deviation from the first direction, wherein the calculation is based on pedestrian dead reckoning, and the amount of deviation is in a horizontal direction; calculate a body orientation deviation, wherein the body orientation deviation is an angular deviation of the person with respect to the first direction; and generate the guide information for the person based on the amount of deviation and the body orientation deviation. ### Claim 10 (depends on 1) The information processing apparatus according to claim 1, wherein the CPU is further configured to: determine the first direction based on the person who is in front of a crosswalk; and generate the guide information based on the walk of the person on the crosswalk. ### Claim 11 (depends on 1) The information processing apparatus according to claim 1, further comprising: an output interface configured to output the generated guide information, wherein the guide information is a voice message or electronic sound. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the guide information is associated with a checkpoint, and the checkpoint is on a route of the walk of the person. ### Claim 13 (depends on 12) The information processing apparatus according to claim 12, wherein the guide information is a voice message. ### Claim 14 (depends on 1) The information processing apparatus according to claim 1, wherein the CPU is further configured to: determine a positional relationship between the person and an unmanned aircraft that tracks the person; and generate the guide information based on the positional relationship. ### Claim 15 (depends on 1) The information processing apparatus according to claim 1, wherein the CPU is further configured to: compare an aerial image captured by an unmanned aircraft with the current image; and generate the guide information based on the comparison of the aerial image and the current image. ### Claim 16 (depends on 1) The information processing apparatus according to claim 1, further comprising a display screen, wherein the information processing apparatus is a smartphone, and the CPU is further configured to: detect that a smartphone cover is in a closed state with respect to the smartphone; and control the display screen based on the closed state of the smartphone cover. ### Claim 17 (depends on 16) The information processing apparatus according to claim 16, further comprising: a near field communication unit configured to: establish a near field communication between the smartphone cover and the smartphone; and acquire information from the smartphone cover via the near field communication, wherein, in the closed state of the smartphone cover, the CPU is further configured to control the display screen based on the acquired information. ### Claim 18 An information processing method, comprising, in an information processing apparatus: capturing a current image at a specific time duration after a start of walk of a person; comparing the current image and a reference image, wherein a time of capture of the reference image corresponds to the specific time duration; determining a direction of the walk of the person based on the comparison of the current image and the reference image; and generating guide information for the person based on the determined direction, wherein the guide information indicates the person to walk in the determined direction. ### Claim 19 A non-transitory computer-readable medium having stored thereon, computer executable-instructions that, when executed by a processor, cause the processor to execute operations, the operations comprising: capturing a current image at a specific time duration after a start of walk of a person; comparing the current image and a reference image, wherein a time of capture of the reference image corresponds to the specific time duration; determining a direction of the walk of the person based on the comparison of the current image and the reference image; and generating guide information for the person based on the determined direction, wherein the guide information indicates the person to walk in the determined direction.