Back to results
US 10,168,798 B2Granted
Head mounted display
AssigneeTower Spring Global Limited
InventorEric Kwong Hang Tsang
Life of this patent
Estimating the term…
Abstract
Head-mounted display (HMD) provides a virtual reality to a user that controls the HMD with a keyboard. The HMD has a housing with a left lens and a right lens, a smartphone that includes a display, a camera that captures video signals of the keyboard, and a processer. The display simultaneously displays the virtual reality and the keyboard through which instructions are sent from the user to the HMD.
Claims17
A head-mounted display (HMD) that provides a virtual reality to a user that controls the HMD with a keyboard, comprising:
- a housing having a left lens and a right lens;
- a smartphone inserted in the housing and including:
- a display having a left portion and a right portion, the left portion viewable by a left eye of the user through the left lens, the right portion viewable by a right eye of the user through the right lens;
- a receiver that receives a first video signal from a VR source;
- a camera that captures a second video signal of the keyboard;
- and
- a processor, in communication with the display, the receiver, the camera, and the keyboard, that converts the first video signal from the receiver to a first left image and a first right image, converts the second video signal from the camera to a second left image of the keyboard and a second right image of the keyboard, receives input information from the keyboard, and sends the converted images and the input information to the display,
- wherein the display of the smartphone shows the first left image on the left portion and the first right image on the right portion such that the user is able to view the virtual reality on a first window of the display through the left and right lenses,
- wherein the display of the smartphone shows the second left image of the keyboard on the left portion and the second right image of the keyboard on the right portion such that the user is able to view the keyboard on a second window of the display simultaneously with the first window,
- wherein the processor monitors boundaries of the first window and the second window to detect an overlap condition;
- where the second window overlaps the first window such that content in the first window is obscured from the user's view;
- and
- further wherein, when the processor detects the overlap condition, the processor automatically adjusts a transparency and blending effect of an overlap area between the first window and the second window such that the transparency in the overlap area differs from the transparency in other regions of the first and second windows that occupy separate areas;
- and such that the virtual reality of the first window and the keyboard of the second window are both viewable to the user;
- wherein the keyboard of the second window is superimposed on the virtual reality of the first window;
- and wherein the following formulas are used to obtain a blended image of the keyboard superimposed on the virtual reality image in the first window;
- g
- =
- ∑
- i
- :
- keyboardimage
-
- Y
- ky
-
- [
- i
- ]
- ∑
- i
- :
- backgroundimage
-
- Y
- bg
-
- [
- i
- ]
- formula
-
-
- I
- img
- out
- =
- img
- ky
- ×
- (
- a
- g
- )
- +
- img
- bg
- ×
- (
- 1
- -
- a
- )
- formula
-
-
- II
- where in formula I, Yky and Ybg, are luminance signals of keyboard image and background image that is virtual reality image, g is a relative luminance gain of the keyboard image to the background image;
- and in formula II, where alpha (a) is a transparency power of display region, showing how the luminance gain g calculated by the formula I is applied to alpha blending process to calculate pixel values in the blended image.
66 more elements
A method executed by a head-mounted display (HMD) that provides a virtual reality (VR) to a user and allows the user to view physical objects in an environment of the user simultaneously with VR objects, the method comprising:
- converting, by a processor of the HMD, a first video signal to a first left image and a first right image;
- receiving, by the processor of the HMD, a second video signal of the physical objects in the environment of the user from a camera of the HMD and converting the second video signal to a second left image and a second right image;
- displaying, on a display of the HMD, the first left image on a left portion of the display viewable by a left eye of the user through a left lens of the HMD, and the first right image on a right portion of the display viewable by a right eye of the user through a right lens of the HMD such that the user is able to view the virtual reality on a first window of the display through the left and right lenses;
- displaying, on the display of the HMD, the second left image on the left portion of the display and the second right image on the right portion of the display such that the user is able to view the physical objects in the environment of the user on a second window of the display;
- monitoring, by the processor of the HMD, boundaries of the first window and the second window to detect an overlap condition where the second window overlaps the first window such that a content in the first window is obscured from a view of the user;
- and
- adjusting automatically, by the processor of the HMD, a transparency and blending effect of an overlap area between the first window and the second window when an overlap condition is detected by the processor, such that the transparency in the overlap area differs from the transparency in other regions of the first and second windows that occupy separate areas and thereby the virtual reality of the first window and the physical objects of the second window are both viewable to the user;
- displaying a keyboard simultaneously with the virtual reality, wherein the keyboard is one of the physical objects in the environment of the user,
- wherein the keyboard of the second window is superimposed on the virtual reality of the first window, and the following formulas are used to obtain a blended image of the keyboard superimposed on the virtual reality image in the first window:
- g
- =
- ∑
- i
- :
- keyboardimage
-
- Y
- ky
-
- [
- i
- ]
- ∑
- i
- :
- backgroundimage
-
- Y
- bg
-
- [
- i
- ]
- formula
-
-
- I
- img
- out
- =
- img
- ky
- ×
- (
- a
- g
- )
- +
- img
- bg
- ×
- (
- 1
- -
- a
- )
- formula
-
-
- II
- where in formula I, Yky and Ybg, are luminance signals of keyboard image and background image that is virtual reality image, g is a relative luminance gain of the keyboard image to the background image;
- and in formula II, where alpha (a) is a transparency power of display region, showing how the luminance gain g calculated by the formula I is applied to alpha blending process to calculate pixel values in the blended image.
59 more elements
A method executed by a head-mounted display (HMD) that provides a semi-immersive virtual reality to a user, the method comprising:
- converting, by a processor of the HMD, a first video signal from a VR source to a first left image and a first right image;
- receiving, by the processor of the HMD, a second video signal of a keyboard from a camera in the HMD and converting the second video signal to a second left image and a second right image;
- displaying, on a display of the HMD, the first left image on a left portion of the display viewable by a left eye of the user through a left lens of the HMD, and the first right image on a right portion of the display viewable by a right eye of the user through a right lens of the HMD such that the user is able to view a virtual reality on a first window of the display through the left and right lenses;
- displaying, on the display of the HMD, the second left image on the left portion of the display, and the second right image on the right portion of the display such that the user is able to simultaneously view the keyboard on a second window of the display along with the virtual reality on the first window;
- monitoring, by the processor of the HMD, boundaries of the first window and the second window to detect an overlap condition where the second window overlaps the first window;
- adjusting automatically, by the processor of the HMD, a transparency and blending effect of an overlap area between the first window and the second window when an overlap condition is detected by the processor, such that the transparency in the overlap area differs from the transparency in other regions of the first and second windows that occupy separate areas and thereby the virtual reality of the first window and the physical objects of the second window are both viewable to the user;
- and
- receiving instructions from the user via keys of the keyboard to the virtual reality displayed in the first window while the keys are shown in the second window,
- wherein the keyboard of the second window is superimposed on the virtual reality of the first window, and the transparency and blending effect of the both windows are detected and adjusted by the processor, and the following formulas are used to obtain a blending image of the keyboard superimposed on the virtual reality image in the first window:
- g
- =
- ∑
- i
- :
- keyboardimage
-
- Y
- ky
-
- [
- i
- ]
- ∑
- i
- :
- backgroundimage
-
- Y
- bg
-
- [
- i
- ]
- formula
-
-
- I
- img
- out
- =
- img
- ky
- ×
- (
- a
- g
- )
- +
- img
- bg
- ×
- (
- 1
- -
- a
- )
- formula
-
-
- II
- where in formula I, Yky and Ybg, are luminance signals of keyboard image and background image that is virtual reality image, g is a relative luminance gain of the keyboard image to the background image;
- and in formula II, where alpha (a) is a transparency power of display region, showing how the luminance gain g calculated by the formula I is applied to alpha blending process to calculate pixel values in the blended image.
59 more elements
Dependencies are inferred from the claim text.