Safety for wearable virtual reality devices via object detection and tracking

A non-transitory computer readable medium impressed with instructions for detecting a physical object during a virtual reality experience, which instructions when executed by one or more processors cause the processors to perform the following:

• using at least one camera in connection with a wearable virtual reality device to capture a sequence of images of a real world environment near a user wearing the device;
• using features in some images of the sequence of images to track proximity of at least one physical object near the user, including:
• at a first time t0, using a sensor, sensing a first position for at least one physical object in a first reference frame of the real world environment;
• causing display of a first virtual representation of the physical object at the first position, wherein the first virtual representation is rendered in a virtual environment of the device;
• and
• at a second time t1, responsive to repositioning of the real world environment and the sensor due to body movement of the user, sensing, in the real world environment, a second position of the physical object and at least some portions of the physical object that appear different from the first position;
• and
• causing display of a second virtual representation of the physical object at an actual second position, including:
• sensing motion of the wearable virtual reality device and calculating a second reference frame that accounts for repositioning of the device;
• calculating a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame;
• and
• transforming the first position and the second position of the physical object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby a sensed second position is transformed to the actual second position;
• and
• responsive to detecting that either one of the wearable virtual reality device and the physical object is approaching the other, automatically generating an alert to be injected into a virtual reality experience of the user.

A system, including:

• a wearable virtual reality device;
• at least one camera mounted to the wearable virtual reality device that captures a sequence of images of a real world environment ahead of a user wearing the device;
• and
• a computer readable medium including instructions for detecting at least one physical object during a virtual reality experience, which instructions when executed by one or more processors cause the one or more processors to perform the following:
• using the at least one camera in connection with the wearable virtual reality device to capture the sequence of images of the real world environment ahead of the user wearing the device;
• using features in some images of the sequence of images to track proximity of the at least one physical object ahead of the user, including:
• at a first time t0, using a sensor, sensing a first position for the at least one physical object in a first reference frame of the real world environment;
• causing display of a first virtual representation of the physical object at the first position, wherein the first virtual representation is rendered in a virtual environment of the wearable virtual reality device;
• and
• at a second time t1, responsive to repositioning of the real world environment and the sensor due to body movement of the user, sensing, in the real world environment, a second position of the physical object and at least some portions of the physical object that appear different from the first position;
• and
• causing display of a second virtual representation of the physical object at an actual second position, including:
• sensing motion of the wearable virtual reality device and calculating a second reference frame that accounts for repositioning of the device;
• calculating a transformation that renders the first position in the first reference frame and the second position in the second reference frame into a common reference frame;
• and
• transforming the first position and the second position of the physical object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby a sensed second position is transformed to the actual second position;
• and
• responsive to detecting that either one of the wearable virtual reality device and the physical object is approaching the other, automatically generating an alert to be injected into a virtual reality experience of the user.