# US 2014/0152530 A1 · Multimedia near to eye display system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0152530 A1 - Document ID: US20140152530A1 - Status: Published application - Kind: A1 - Application number: 13/692,509 - Filed: 2012-12-03 - Published: 2014-06-05 - Earliest claimed: 2012-12-03 - Assignees: Honeywell International Inc. - Applicants: Honeywell International Inc. - Inventors: Sharath Venkatesha; Kwong Wing Au - Web page: https://patentgrep.com/patent/US20140152530A1 ## CPC - G09G 3/001: {using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background (slide projectors per se G03B23/00 = 42 HP)} - G01B 11/00: Measuring arrangements characterised by the use of optical techniques - G02B 27/017: {Head mounted} - G01P 15/00: Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0141: {characterised by the informative content of the display} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G}) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method include receiving video images based on field of view of a wearer of a near to eye display system, analyzing the video images to identify an object in the wearer field of view, generating information as a function of the identified objects, and displaying the information on a display device of the near to eye display system proximate the identified object. ## Claims ### Claim 1 A method comprising: receiving video images based on fields of view of a near to eye display system; applying video analytics to enhance the video images and to identify regions of interest (ROI) on the video images; generating user assistance information as a function of at least one characteristic of the regions of interest; and augmenting the enhanced video with the derived information proximate to corresponding regions of interest via visual displays and audio of the near to eye display system. ### Claim 2 (depends on 1) The method of claim 1, wherein the user assistance information displayed on the near to eye display system is derived from: interactive video analysis and user inputs from voice and signals from hand held devices; information stored in memory; and information retrieved from cloud storage and the World Wide Web. ### Claim 3 (depends on 2) The method of claim 2, wherein the user assistance information comprises images, video clips, text, graphics, symbols including use of color, transparency, shading, and animation. ### Claim 4 (depends on 2) The method of claim 2, wherein the user assistance information is communicated to the user as audio, including: descriptions of the video images, identified regions of interest and their characteristics; and pre-recorded audio instructions, based on outputs of the video analysis. ### Claim 5 (depends on 1) The method of claim 1 wherein the at least one characteristic of regions of interest are selected from the group consisting of textural, spatial, structural, temporal and biometric features including appearance, shape, object identity, identity of person, motion, tracks, and events. ### Claim 6 (depends on 5) The method of claim 5 wherein the events further comprise application specific activities, industrial operations including identifying tools, determining a stage of an activity, operation, and the status of a stage. ### Claim 7 (depends on 1) The method of claim 1 wherein the video analytics to enhance the video images includes modifying the appearance, brightness and contrast by color and local intensity corrections on pixels in the images. ### Claim 8 (depends on 1) The method of claim 1 wherein characteristics of regions of interest further comprise estimated distance to the region of interest, a surface descriptor, and 3D measurements including at least one of volume, surface areas, length, width and height. ### Claim 9 (depends on 8) The method of claim 8 wherein the user assistance information is displayed adjacent the corresponding region of interest in the video. ### Claim 10 (depends on 9) The method of claim 9 wherein augmenting user assistance information further includes: a distance scale indicating the projected distances of the pixels from the near to eye display system; and a geometric object of same size as the corresponding region of interest, proximate the ROI. ### Claim 11 A multi-media visual system comprising: near-to-eye displays supported by a frame adapted to be worn by a user such that each display is positioned proximate an eye of the user; speakers coupled to deliver audio of user assistance information; a set of cameras supported by the frame, capturing video images of a scene in a field of view; a microphone receiving inputs from the wearer; and a processor coupled to receive images from the cameras and adapted to apply video analytics to enhance the video images, to identify regions of interest (ROI) on the video images and to generate user assistance information as a function of the characteristics of the regions of interest. ### Claim 12 (depends on 11) The multi-media visual system of claim 11 wherein the near to eye display consists of a transparent LCD for text display overlaid on LCD/LCoS/Light-Guide-Optics (LOE) for video display. ### Claim 13 (depends on 11) The multi-media visual system of claim 11 wherein the cameras are receptive to different spectra including visible, near infrared (NIR), ultraviolet (UV), short wave infrared bands, mid wave infrared or long wave infrared. ### Claim 14 (depends on 11) The multi-media visual system of claim 11 and further comprising: a MEMS accelerometer to provide orientation of the frame; cameras capturing images of the eyes of the user including pupil position; and remote input devices to receive requests from the wearer. ### Claim 15 (depends on 14) The multi-media visual system of claim 14 wherein the processor is further adapted to generate user assistance information based on inputs representing the frame orientation, pupil locations and user requests. ### Claim 16 (depends on 15) The multi-media visual system of claim 15 wherein user assistance information comprises: at least one of textural, spatial, structural, temporal and biometric features including appearance, shape, object identity, identity of person, motion, tracks, and events; and at least one of application specific activities, industrial operations including identifying tools, determining the stage of the activity, operation, and the status of the stage ### Claim 17 (depends on 16) The multi-media visual system of claim 16 wherein user assistance information further includes at least one of estimated distance to the region of interest, its surface descriptor, and 3D measurements including volume, surface areas, length, width, and height. ### Claim 18 (depends on 17) The multi-media visual system of claim 17 wherein the user assistance information is displayed proximate the corresponding region of interest in the video. ### Claim 19 (depends on 18) The multi-media visual system of claim 18 wherein the user assistance information further comprises: a distance scale indicating the projected distances of the pixels from the near to eye display system; and a geometric object of same size as the corresponding region of interest, proximate the ROI. ### Claim 20 (depends on 19) The multi-media visual system of claim 19 wherein the video analytics to enhance the video images includes at least one of modifying the appearance, brightness and contrast by color, and local intensity corrections on the pixels in the images.