# US 11,908,340 B2 · Magnification enhancement of video for visually impaired viewers > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,340 B2 - Document ID: US11908340B2 - Status: Granted - Kind: B2 - Application number: 16/938,411 - Filed: 2020-07-24 - Published: 2021-01-28 - Granted: 2024-02-20 - Earliest claimed: 2019-07-24 - Assignees: ARRIS Enterprises LLC - Applicants: ARRIS Enterprises LLC - Inventors: Thomas L. Du Breuil - Web page: https://patentgrep.com/patent/US11908340B2 ## CPC - G09B 21/008: {using visual presentation of the information for the partially sighted} - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - H04N 5/2621: {Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability} - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method for modifying a video stream for display includes receiving a request to display a video stream in a display mode that facilitates viewing by visually impaired viewers. An unmodified region of interest (ROI) is identified in a current video frame of the video stream. A location of the unmodified ROI is tracked in each of the subsequent video frames. The unmodified ROI is magnified by a prescribed amount in at least some of the video frames in which the unmodified ROI is being tracked to produce a magnified ROI. For each of the video frames in which the unmodified ROI is being tracked and magnified, the magnified ROI is combined with at least a portion of the video frame in which the ROI is located to thereby produce a modified video stream. ## Claims ### Claim 1 A method for modifying a video stream for display on a display device, comprising: receiving a request at a vision enhancement system to display a video stream in a display mode that facilitates viewing by visually impaired viewers; identifying an unmodified region of interest (ROI) in a current video frame of the video stream; tracking, with the vision enhancement system, a location of the unmodified ROI in each of the video frames of the video stream subsequent to the current frame; magnifying, with the vision enhancement system, the unmodified ROI by a prescribed amount in at least some of the video frames in which the unmodified ROI is being tracked to produce a magnified ROI corresponding to an unmodified ROI; for each of the video frames in which the unmodified ROI is being tracked and magnified, combining, with the vision enhancement system, the magnified ROI with at least a portion of the video frame in which the ROI is located to thereby produce a modified video stream; and causing, with the vision enhancement system, the modified video stream to be presented on a display device; where the vision enhancement system includes a trained, automated neural network that automatically identifies the unmodified ROI using an audio component of the video stream. ### Claim 2 (depends on 1) The method of claim 1, wherein combining, with the vision enhancement system, the magnified ROI with at least a portion of the video frame in which the ROI is located includes overlaying the magnified ROI over at least a portion of the corresponding unmodified ROI. ### Claim 3 (depends on 1) The method of claim 1, wherein identifying the unmodified ROI includes receiving viewer input that identifies the unmodified ROI in the video stream. ### Claim 4 (depends on 3) The method of claim 3, wherein the viewer input is received through a voice-activated smart assistant or a viewer-interactive on-screen pointer overlaid on a presentation of the video stream on the display device. ### Claim 5 (depends on 1) The method of claim 1, wherein magnifying the unmodified ROI further comprises magnifying the unmodified ROI by a default amount. ### Claim 6 (depends on 1) The method of claim 1, further comprising receiving user input specifying a magnification level and magnifying the unmodified ROI by the specified magnification level. ### Claim 7 (depends on 1) The method of claim 1, wherein identifying the unmodified ROI further includes classifying the unmodified ROI as belonging to a class based on a type of content identified in the unmodified ROI. ### Claim 8 (depends on 7) The method of claim 7, further comprising comparing the class in which the unmodified ROI has been classified to a predetermined list of ROI classes and only performing the tracking and magnifying of the unmodified ROI if the class is included on the predetermined list of ROI classes. ### Claim 9 (depends on 1) The method of claim 1, wherein the combining includes centering the magnified ROI over a center of the unmodified ROI in the modified video stream. ### Claim 10 (depends on 1) The method of claim 1, wherein the combining includes offsetting a center of the magnified ROI from a center of the unmodified ROI in the modified video stream. ### Claim 11 (depends on 1) The method of claim 1, further comprising automatically and dynamically adjusting, without user intervention, the prescribed amount of magnification by which the unmodified ROI is magnified. ### Claim 12 (depends on 1) The method of claim 1, further comprising visually enhancing the magnified ROI for enhanced viewability by visually impaired viewers. ### Claim 13 (depends on 1) The method of claim 1, wherein visually enhancing the magnified ROI includes changing a value of at least one video parameter selected from the group consisting of luminosity, contrast and color gamut. ### Claim 14 (depends on 13) The method of claim 13, further comprising receiving user input specifying a type of visual impairment for which viewability of the video stream is to be enhanced, wherein changing a value of at least one parameter includes changing the at least one value based on the type of visual impairment. ### Claim 15 A vision enhancement system for modifying a video stream for display on a display device, comprising: a vision analytics system configured to receive a request to display a video stream in a display mode that facilitates viewing by visually impaired viewers, the vision analytics system including: a region of interest (ROI) detection system configured to identify an unmodified ROI in a current video frame of the video stream; and an ROI tracking system configured to track a location of the unmodified ROI in each of the video frames of the video stream subsequent to the current frame; a region magnification module configured to magnify the unmodified ROI by a prescribed amount in at least some of the video frames in which the unmodified ROI is being tracked to produce a magnified ROI corresponding to an unmodified ROI; and an image combining module configured to combine, for each of the video frames in which the unmodified ROI is being tracked and magnified, the magnified ROI with at least a portion of the video frame in which the magnified ROI is located to thereby produce a modified video stream; where the vision enhancement system includes a trained, automated neural network that automatically identifies the unmodified ROI using an audio component of the video stream. ### Claim 16 (depends on 15) The vision enhancement system of claim 15, wherein the image combining module is further configured to combine the magnified ROI with at least a portion of the video frame in which the ROI is located includes overlaying the magnified ROI over at least a portion of the corresponding unmodified ROI. ### Claim 17 (depends on 15) The vision enhancement system of claim 15, wherein the vision analytics system is further configured to classify the unmodified ROI as belonging to a class based on a type of content identified in the unmodified ROI. ### Claim 18 (depends on 15) The vision enhancement system of claim 15, wherein the region magnification module is further configured to automatically and dynamically adjust, without user intervention, the prescribed amount of magnification by which the unmodified ROI is magnified.