# US 2019/0320125 A1 · Systems, methods, and computer-readable media for using a video capture device to alleviate motion sickness via an augmented display for a passenger > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0320125 A1 - Document ID: US20190320125A1 - Status: Published application - Kind: A1 - Application number: 16/451,962 - Filed: 2019-06-25 - Published: 2019-10-17 - Earliest claimed: 2016-12-28 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Daniel Gray GOLDSTEIN - Web page: https://patentgrep.com/patent/US20190320125A1 ## CPC - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 7/183: {for receiving images from a single remote source} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Systems, methods, and computer-readable media are disclosed for using a video capture device to alleviate motion sickness via an augmented display for a passenger. One method includes receiving, from a video capture device of a vehicle, video data of a view outside of the vehicle while the vehicle is in motion; displaying, on a display device for a passenger of the vehicle, a request for a mode of display of the video data, the mode of displaying including a passive video mode and an active video mode; receiving a selection of the mode of display; displaying a request for a video display type when the passive video mode is selected; and displaying the video data including a menu, the menu including a plurality of selectable icons to interact with the video data when the active video mode is selected. ## Claims ### Claim at position 1 1-20. (canceled) ### Claim 21 A computer-implemented method for using a video capture device to alleviate motion sickness via an augmented display for a passenger, the method comprising: receiving, from a video capture device, video data of a view outside of a vehicle while the vehicle is in motion; analyzing video data of the video capture device of the view outside of the vehicle while the vehicle is in motion to identify at least one target object in the video data representing a real object in the view outside of the vehicle; displaying, on the display device for the passenger, the video data including a menu, the menu including a plurality of selectable icons to interact with the identified at least one target object in the video data, wherein the plurality of selectable icons are based on at least one of the identified real object and the analyzed video data; and displaying, on the display device for the passenger, augment video data that includes at least one virtual object on or around the identified real object based on a selection of a first selectable icon of the plurality of selectable icons. ### Claim 22 (depends on 21) The method according to claim 21, wherein the at least one virtual object includes one or more of i) walls, ii) pot holes, iii) oil spills, iv) slime, and v) simulated objects in which at least one of the vehicle, a virtual vehicle, and a neighboring vehicle interacts. ### Claim 23 (depends on 21) The method according to claim 21, wherein the video capture device of the vehicle is one of mounted in the vehicle or on the vehicle. ### Claim 24 (depends on 21) The method according to claim 21, wherein the video data of the video capture device is of a forward-facing view outside of the vehicle while the vehicle is in motion. ### Claim 25 (depends on 21) The method according to claim 21, further comprising: receiving the selection of the first selectable icon of the plurality of selectable icons, wherein the selection of the first selectable icon identifies the at least one virtual object to include in the augmented video data. ### Claim 26 (depends on 21) The method according to claim 21, wherein displaying augment video data includes: creating, by the display device, an augmented video data by one of merging, augmenting, and fusing the video data of the video capture device with the at least one virtual object, wherein the at least one virtual object are based on the selection of the first selectable icon by the passenger. ### Claim 27 (depends on 21) The method according to claim 21, further comprising: sending a request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data. ### Claim 28 (depends on 26) The method according to claim 26, further comprising: receiving a response to the request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data; and assigning a point value based on the received response. ### Claim 29 A system for using a video capture device to alleviate motion sickness via an augmented display for a passenger, the system including: a data storage device that stores instructions for using a video capture device to alleviate motion sickness via an augmented display for a passenger; and a processor configured to execute the instructions to perform a method including: receiving, from a video capture device, video data of a view outside of a vehicle while the vehicle is in motion; analyzing video data of the video capture device of the view outside of the vehicle while the vehicle is in motion to identify at least one target object in the video data representing a real object in the view outside of the vehicle; displaying, on the display device for the passenger, the video data including a menu, the menu including a plurality of selectable icons to interact with the identified at least one target object in the video data, wherein the plurality of selectable icons are based on at least one of the identified real object and the analyzed video data; and displaying, on the display device for the passenger, augment video data that includes at least one virtual object on or around the identified real object based on a selection of a first selectable icon of the plurality of selectable icons. ### Claim 30 (depends on 29) The system according to claim 29, wherein the at least one virtual object includes one or more of i) walls, ii) pot holes, iii) oil spills, iv) slime, and v) simulated objects in which at least one of the vehicle, a virtual vehicle, and a neighboring vehicle interacts. ### Claim 31 (depends on 29) The system according to claim 29, wherein the video capture device of the vehicle is one of mounted in the vehicle or on the vehicle. ### Claim 32 (depends on 29) The system according to claim 29, wherein the video data of the video capture device is of a forward-facing view outside of the vehicle while the vehicle is in motion. ### Claim 33 (depends on 29) The system according to claim 29, further comprising: receiving the selection of the first selectable icon of the plurality of selectable icons, wherein the selection of the first selectable icon identifies the at least one virtual object to include in the augmented video data. ### Claim 34 (depends on 29) The system according to claim 29, wherein displaying augment video data includes: creating, by the display device, an augmented video data by one of merging, augmenting, and fusing the video data of the video capture device with the at least one virtual object, wherein the at least one virtual object are based on the selection of the first selectable icon by the passenger. ### Claim 35 (depends on 29) The system according to claim 29, further comprising: sending a request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data. ### Claim 36 (depends on 35) The system according to claim 35, further comprising: receiving a response to the request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data; and assigning a point value based on the received response. ### Claim 37 A non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for using a video capture device to alleviate motion sickness via an augmented display for a passenger, the method including: receiving, from a video capture device, video data of a view outside of a vehicle while the vehicle is in motion; analyzing video data of the video capture device of the view outside of the vehicle while the vehicle is in motion to identify at least one target object in the video data representing a real object in the view outside of the vehicle; displaying, on the display device for the passenger, the video data including a menu, the menu including a plurality of selectable icons to interact with the identified at least one target object in the video data, wherein the plurality of selectable icons are based on at least one of the identified real object and the analyzed video data; and displaying, on the display device for the passenger, augment video data that includes at least one virtual object on or around the identified real object based on a selection of a first selectable icon of the plurality of selectable icons. ### Claim 38 (depends on 37) The computer-readable medium of claim 37, wherein the at least one virtual object includes one or more of i) walls, ii) pot holes, iii) oil spills, iv) slime, and v) simulated objects in which at least one of the vehicle, a virtual vehicle, and a neighboring vehicle interacts. ### Claim 39 (depends on 37) The computer-readable medium of claim 37, further comprising: sending a request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data. ### Claim 40 (depends on 39) The computer-readable medium of claim 39, further comprising: receiving a response to the request to identify at least one of a type of color, a type of vehicle, a type of road feature, a word and a character in the augmented video data; and assigning a point value based on the received response.