# US 2026/0137480 A1 · Techniques For Remotely Modifying Video Data Quality To Optimize Streaming In Surgical Extended Reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0137480 A1 - Document ID: US20260137480A1 - Status: Published application - Kind: A1 - Application number: 19/373,833 - Filed: 2025-10-30 - Published: 2026-05-21 - Earliest claimed: 2024-11-15 - Assignees: Stryker Corporation - Applicants: Stryker Corporation - Inventors: Steven D. Scherf - Web page: https://patentgrep.com/patent/US20260137480A1 ## CPC - A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - G02B 27/017: {Head mounted} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06N 20/00: Machine learning - G06T 7/0002: {Inspection of images, e.g. flaw detection} - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/234363: {by altering the spatial resolution, e.g. for clients with a lower screen resolution} - H04N 21/234381: {by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping} - H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)} - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - A61B 2017/00216: {with eye tracking or head position tracking control} - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/2059: {Mechanical position encoders} - A61B 2034/2063: {Acoustic tracking systems, e.g. using ultrasound} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/3966: {Radiopaque markers visible in an X-ray image} - A61B 2090/3979: {infrared} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Extended reality surgical systems and methods involve a surgical device that includes a video source that is configured to generate video data including surgical content. A head-mounted device (HMD) includes an HMD display positionable in front of a user's eyes. The HMD is configured to receive the video data from a remote source and to stream the video data on the HMD display. The HMD acquires information indicative of the user's visual experience in consuming the video data. The information is transmitted to the remote source, and the remote source dynamically modifies at least one quality parameter of the video data based on the information and wirelessly transmits the modified video data to the HMD for streaming. ## Claims ### Claim 1 A surgical system comprising: a surgical device that includes a video source that is configured to generate video data including surgical content; a head-mounted device (HMD) comprising an HMD controller, and an HMD display positionable in front of a user's eyes; and a connectivity system in communication with the video source and being remote from the HMD, the connectivity system being configured to wirelessly communicate with the HMD controller and wirelessly transmit the video data to the HMD controller; wherein the HMD controller is configured to: present, on the HMD display, a virtual window containing the video data; acquire spatial information related to one or both of: the virtual window relative to a field-of-view of the HMD display; and a gaze of the user relative to the virtual window; and wirelessly transmit the spatial information to the connectivity system; and wherein the connectivity system is configured to: receive the spatial information acquired by the HMD controller; dynamically modify at least one quality parameter of the video data based on the spatial information to generate modified video data; and wirelessly transmit the modified video data to the HMD controller for presentation on the HMD display. ### Claim 2 (depends on 1) The surgical system of claim 1, wherein the at least one quality parameter of the video data comprises one or more of: a resolution, a bitrate, and a frame rate. ### Claim 3 (depends on 2) The surgical system of claim 2, wherein the at least one quality parameter further comprises a compression parameter of the video data. ### Claim 4 (depends on 1) The surgical system of claim 1, wherein the connectivity system further dynamically modifies the at least one quality parameter based on an optimization algorithm that seeks to minimize data transmission related to the video data while maintaining the user's experience in viewing the video data. ### Claim 5 (depends on 1) The surgical system of claim 1, wherein: the spatial information is related to a size of the virtual window relative to the field-of-view of the HMD display; and the connectivity system is configured to dynamically modify the at least one quality parameter of the video data based on the size of the virtual window relative to the field-of-view. ### Claim 6 (depends on 5) The surgical system of claim 5, wherein the connectivity system is configured to: dynamically modify the at least one quality parameter to increase quality of the video data in response to detecting an increase in the size of the virtual window; and dynamically modify the at least one quality parameter to decrease quality of the video data in response to detecting a decrease in the size of the virtual window. ### Claim 7 (depends on 5) The surgical system of claim 5, wherein: the at least one quality parameter comprises a resolution of the video data; and the connectivity system is configured to dynamically modify the resolution of the video data proportional to the size of the virtual window. ### Claim 8 (depends on 1) The surgical system of claim 1, wherein the spatial information is related to the gaze of the user relative to the virtual window. ### Claim 9 (depends on 8) The surgical system of claim 8, wherein: the spatial information indicates a sub-region of interest of the virtual window that the user is focused on and indicates a remaining region of the virtual window that the user is not focused on; and based on the spatial information, the connectivity system is configured to: dynamically modify the at least one quality parameter to increase quality of the video data in the sub-region of interest of the virtual window; and dynamically modify the at least one quality parameter to decrease quality of the video data in the remaining region of the virtual window. ### Claim 10 (depends on 8) The surgical system of claim 8, wherein: the spatial information indicates an object of interest of the virtual window that the user is focused on and indicates a remaining region of the virtual window that the user is not focused on; and based on the spatial information, the connectivity system is configured to dynamically modify the at least one quality parameter of the video data in a region defining the object of interest of the virtual window. ### Claim 11 (depends on 8) The surgical system of claim 8, wherein: the HMD controller is configured to present, on the HMD display, other content outside of the virtual window containing the video data; the spatial information indicates that the gaze of the user is focused on the virtual window containing the video data and indicates that the user is not focused on the other content; and based on the spatial information, the connectivity system is configured to dynamically modify the at least one quality parameter to increase quality of the video data in the virtual window. ### Claim 12 (depends on 8) The surgical system of claim 8, wherein: the HMD controller is configured to present, on the HMD display, other content outside of the virtual window containing the video data; the spatial information indicates that the gaze of the user is focused on the other content and indicates that the user is not focused on the virtual window containing the video data; and based on the spatial information, the connectivity system is configured to dynamically modify the at least one quality parameter to decrease quality of the video data in the virtual window. ### Claim 13 (depends on 1) The surgical system of claim 1, wherein: the HMD controller is configured to present, on the HMD display, the virtual window containing the video data according to a frame rate of at least 60 Hz; and the connectivity system is configured to receive the spatial information and dynamically modify the at least one quality parameter of the video data based on the spatial information, for each frame. ### Claim 14 (depends on 1) The surgical system of claim 1, wherein the surgical device comprises one of: a surgical scope comprising a camera as the video source; a surgical robot or surgical tool comprising a camera as the video source; a navigation system comprising a camera as the video source; a second HMD comprising a camera as the video source; an ultrasound scanner coupled to the video source; or a navigation system that executes a clinical application that provides the video data. ### Claim 15 (depends on 1) The surgical system of claim 1, wherein the connectivity system is configured to: detect information about the video data; and dynamically modify the at least one quality parameter of the video data based further on the detected information. ### Claim 16 (depends on 15) The surgical system of claim 15, wherein: the detected information comprises contextual information related to the surgical content of the video data; and the connectivity system is configured to dynamically modify the at least one quality parameter of the video data based further on the detected contextual information. ### Claim 17 (depends on 16) The surgical system of claim 16, wherein the contextual information comprises one or more of: a surgical step; an aspect of a surgical step; a portion of a graphical user interface; a surgical task requiring attention by the user of the HMD; a critical anatomical structure; presence of a surgical tool; and an alert or warning. ### Claim 18 (depends on 1) The surgical system of claim 1, wherein: the detected information comprises quantitative and/or qualitative information related to the video data; and the connectivity system is configured to dynamically modify the at least one quality parameter of the video data based further on the detected quantitative and/or qualitative information. ### Claim 19 A connectivity system for use with a surgical system, the surgical system comprises a surgical device that includes a video source to generate video data including surgical content, a head-mounted device (HMD) with an HMD controller and an HMD display positionable in front of a user's eyes, the connectivity system comprising: a housing located remote from the HMD; a controller disposed within the housing and being configured to: receive the video data from the video source; wirelessly transmit the video data to the HMD controller for presentation, on the HMD display, of a virtual window containing the video data; wirelessly receive spatial information from the HMD controller, the spatial information related to one or both of: the virtual window relative to a field-of-view of the HMD display; and a gaze of the user relative to the virtual window; dynamically modify at least one quality parameter of the video data based on the spatial information to generate modified video data; and wirelessly transmit the modified video data to the HMD controller for presentation on the HMD display. ### Claim 20 A head-mounted device (HMD) for use with a surgical system, the surgical system comprises a surgical device that includes a video source to generate video data including surgical content, and a connectivity system in communication with the video source, the HMD comprising: a structure to be worn on the head of a user; an HMD display supported by the structure and positionable in front of a user's eyes; and an HMD controller being configured to: wirelessly communicate with the connectivity system to receive the video data; present, on the HMD display, a virtual window containing the video data; acquire spatial information related to one or both of: the virtual window relative to a field-of-view of the HMD display; and a gaze of the user relative to the virtual window; wirelessly transmit the spatial information to the connectivity system; wirelessly receive modified video data from the connectivity system, the modified video data having been altered by connectivity system through dynamic modification of at least one quality parameter of the video data based on the spatial information; and present the modified video data on the HMD display.