# US 2021/0240988 A1 · Visual aid device and visual aid method by which user uses visual aid device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240988 A1 - Document ID: US20210240988A1 - Status: Published application - Kind: A1 - Application number: 17/059,368 - Filed: 2019-05-28 - Published: 2021-08-05 - Earliest claimed: 2018-05-28 - Assignees: Samsung Life Public Welfare Foundation - Applicants: Samsung Life Public Welfare Foundation - Inventors: Dong Hui LIM - Web page: https://patentgrep.com/patent/US20210240988A1 ## CPC - G09B 21/008: {using visual presentation of the information for the partially sighted} - A61F 9/08: Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception - G02B 27/01: Head-up displays - G06F 18/24: Classification techniques - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 20/00: Machine learning - G06V 10/82: using neural networks - G06V 20/20: in augmented reality scenes - G08B 1/08: using electric transmission {; transformation of alarm signals to electrical signals from a different medium, e.g. transmission of an electric alarm signal upon detection of an audible alarm signal} - G08B 21/02: Alarms for ensuring the safety of persons - G08B 21/18: Status alarms (G08B21/02 takes precedence) - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 5/45: Picture in picture {, e.g. displaying simultaneously another television channel in a region of the screen} - H04N 5/57: Control of contrast or brightness - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - G08B 13/19613: {Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion} - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vision assistance apparatus may include an image acquisition unit configured to acquire an image by capturing the scene of the front which a user watches, a sensor unit configured to acquire sensing information on objects located in front of the user, a control unit configured to analyze the image acquired by the image acquisition unit and generate a notification signal for the front scene through an analysis result of the image and the sensing information acquired by the sensor unit, and an output unit configured to provide the user with the notification signal generated by the control unit in the form of sound. ## Claims ### Claim 1 A vision assistance apparatus comprising: an image acquisition unit configured to acquire an image by capturing a scene of a front which a user watches; a sensor unit configured to acquire sensing information on objects located in front of the user; a control unit configured to analyze the image acquired by the image acquisition unit and generate a notification signal for the front scene through an analysis result of the image and the sensing information acquired by the sensor unit; and an output unit configured to provide the user with the notification signal generated by the control unit in the form of sound, wherein the control unit executes a predetermined learning model for extracting features from the acquired image, and analyzes the image by classifying the objects in the front scene on the basis of the features extracted using the learning model. ### Claim 2 (depends on 1) The vision assistance apparatus according to claim 1, further comprising: a display unit configured to output an auxiliary image for vision assistance of the user generated by correcting the image acquired by the image acquisition unit. ### Claim 3 (depends on 2) The vision assistance apparatus according to claim 2, further comprising: a mode selection unit configured to select an operation mode according to the user. ### Claim 4 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes a total blindness mode, and in a case where the total blindness mode is selected as the operation mode, the sensor unit measures a distance value from the object to the user, and when the control unit determines that the measured distance value is within a predetermined reference value, the notification signal is outputted through the output unit. ### Claim 5 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes a low vision mode, and when the low vision mode is selected as the operation mode, the display unit outputs the auxiliary image generated by adjusting at least one of contrast data, color data, focus data, brightness data, and size data of the acquired image. ### Claim 6 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes a glaucoma mode, and when the glaucoma mode is selected as the operation mode, the image acquisition unit acquires a fish-eye image captured by a fish-eye lens camera, and the display unit outputs the auxiliary image formed by remapping the fish-eye image into tunnel vision of the user. ### Claim 7 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes a macular hole mode, and when the macular hole mode is selected as the operation mode, the display unit outputs the auxiliary image formed by allowing a part deficient due to a dark spot generated in a center of the visual field of the user to be separately displayed on a peripheral part adjacent to the center of the visual field of the user. ### Claim 8 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes a strabismus mode, and when the strabismus mode is selected as the operation mode, the auxiliary image includes a first auxiliary image for a left eye of the user and a second auxiliary image for a right eye of the user, and the display unit outputs the first auxiliary image and the second auxiliary image formed by shifting the image acquired by the image acquisition unit, on the basis of a viewing angle of the left eye of the user and a viewing angle of the right eye of the user. ### Claim 9 (depends on 3) The vision assistance apparatus according to claim 3, wherein the operation mode includes an auditory assistance mode, and when the auditory assistance mode is selected as the operation mode, the control unit generates an image signal for the object by analyzing sound information from the object acquired by the sensor unit, and the auxiliary image formed by allowing the image signal generated by the control unit to be displayed on the image acquired by the image acquisition unit is outputted through the display unit. ### Claim 10 A vision assistance method of a user using a vision assistance apparatus, the vision assistance method comprising: a step of acquiring an image by capturing a scene of a front which a user watches; a step of acquiring sensing information on objects located in front of the user; a step of analyzing the acquired image; a step of generating a notification signal for the front scene through an analysis result of the acquired image and the acquired sensing information; and a step of providing the user with the generated notification signal in the form of sound, wherein the step of analyzing the acquired image comprises: a step of executing a predetermined learning model to extract features of the acquired image; and a step of classifying objects in the front scene on the basis of the extracted features. ### Claim 11 (depends on 10) The vision assistance method according to claim 10, wherein a step of selecting an operation mode according to the user is performed before execution of the step of generating a notification signal for the front scene through an analysis result of the acquired image and the acquired sensing information, and a step of displaying an auxiliary image for vision assistance for the user, which is generated by correcting the acquired image, is performed together with the step of providing the user with the generated notification signal in the form of sound. ### Claim 12 (depends on 10) A non-transitory computer-readable recording medium storing a program for performing the method according to claim 10.