Porting physical object into virtual reality

A system comprising:

• a display device;
• a processor;
• and
• a non-transitory computer readable medium storing instructions when executed by the processor cause the processor to:
• detect a first contour or a first layout of an input device in a physical space relative to a user of the input device,
• identify, from a plurality of candidate models having various levels of similarity to the first contour or the first layout of the input device and corresponding to a same device type as the input device, a candidate model having a second contour or a second layout in a virtual space, that is closest to the first contour or the first layout of the input device in the physical space, as a virtual model of the input device,
• present, by the display device to the user, the virtual model of the input device in the virtual space at a location and an orientation, the location and the orientation of the virtual model in the virtual space corresponding to a location and an orientation of the input device in the physical space relative to the user, and
• visually provide, relative to the virtual model in the virtual space, through the display device, spatial feedback to the user on the user's interaction with the input device in the physical space.

A head mounted display comprising:

• an imaging sensor configured to capture an image of an input device in a physical space;
• a processor configured to:
• detect from the captured image, a first contour or a first layout of the input device in the physical space relative to a user of the input device, and
• identify, from a plurality of candidate models having various levels of similarity to the first contour or the first layout of the input device and corresponding to a same device type as the input device, a candidate model having a second contour or a second layout in a virtual space, that is closest to the first contour or the first layout of the input device in the physical space, as a virtual model of the input device;
• and
• a display device configured to:
• present to the user the virtual model of the input device in the virtual space at a location and an orientation, the location and the orientation of the virtual model in the virtual space corresponding to a location and an orientation of the input device in the physical space relative to the user, and
• visually provide, relative to the virtual model in the virtual space, spatial feedback to the user on the user's interaction with the input device in the physical space.