# US 11,310,347 B2 · Transferring a user interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,347 B2 - Document ID: US11310347B2 - Status: Granted - Kind: B2 - Application number: 16/809,000 - Filed: 2020-03-04 - Published: 2020-09-10 - Granted: 2022-04-19 - Earliest claimed: 2019-03-04 - Assignees: Siemens Healthcare GmbH - Applicants: Siemens Healthcare GmbH - Inventors: Hans Schweizer - Web page: https://patentgrep.com/patent/US11310347B2 ## CPC - H04L 67/04: specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability - H04L 41/045: comprising client-server management architectures - H04L 43/0852: Delays - H04L 43/0864: Round trip delays - H04L 65/80: Responding to QoS - H04L 67/025: for remote control or remote monitoring of applications - H04L 67/08: specially adapted for terminal emulation, e.g. Telnet - H04L 67/125: involving control of end-device applications over a network - H04L 67/131: Protocols for games, networked simulations or virtual reality - H04L 67/565: Conversion or adaptation of application format or content (adding application control or application functional data H04L67/561) - H04L 67/5651: Reducing the amount or size of exchanged application data - H04L 67/59: Providing operational support to end devices by off-loading in the network or by emulation, e.g. when they are unavailable - H04L 67/61: taking into account QoS or priority requirements - H04L 67/63: Routing a service request depending on the request content or context - G16H 40/67: for remote operation - H04L 43/16: Threshold monitoring - H04L 67/75: Indicating network or usage conditions on the user display - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A method is defined for transferring a user interface from at least one server via a proxy to at least one client. A user interface of the server is transferred to the client such that the user interface is displayed on the client, and the server may be controlled from the client. In order to display the user interface on the client, a data stream of a plurality of successive frames is transmitted from the server to the client by a packet-based transfer. A latency of the transfer is determined recurrently for individual frames in order to monitor the transfer. In addition, a medical device and a server are defined. ## Claims ### Claim 1 A method for transferring a user interface from at least one server via a proxy to at least one client, the method comprising: transferring a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayed on the client, and the server is controllable from the client; transmitting a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determining a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, wherein the latency is determined as an absolute frame latency, wherein when transferring a particular frame of the plurality of successive frames, the server sends a time stamp that indicates a start time of a server-side processing of the particular frame, wherein a client-side processing is divided into a receive process and a display process for the frame, and wherein the absolute frame latency is determined as a sum of two time segments. ### Claim 2 (depends on 1) The method of claim 1, wherein the latency is determined as a difference between a start of a server-side processing of a particular frame of the plurality of successive frames and an end of a client-side processing of the particular frame. ### Claim 3 (depends on 1) The method of claim 1, wherein the time stamp includes a start of reading the particular frame from a frame buffer of the server. ### Claim 4 (depends on 1) The method of claim 1, wherein the two time segments include a first time segment that extends from the start time until an end of the receive process, and a second time segment that defines a time length of the display process. ### Claim 5 (depends on 4) The method of claim 4, wherein the time length of the display process is stored as a predefined offset in a memory, or wherein the client measures the time length of the display process for a particular frame of the plurality of successive frames. ### Claim 6 (depends on 1) The method of claim 1, wherein the user interface of the server is transferred via the proxy to a plurality of clients, resulting in a dedicated data stream for each client of the plurality of clients, wherein each client of the plurality of clients determines the absolute frame latency for the associated data stream, and wherein the proxy determines a largest absolute frame latency and transmits the largest absolute frame latency to the server. ### Claim 7 (depends on 1) The method of claim 1, wherein the latency is determined as a relative latency between two successive frames of the plurality of successive frames. ### Claim 8 (depends on 1) The method of claim 1, wherein the client determines the latency and compares the latency with at least one threshold value, and wherein the client outputs an indicator, disables the user interface of the server on the client, or outputs the indicator and disables the user interface of the server on the client when the latency exceeds the threshold value. ### Claim 9 (depends on 1) The method of claim 1, wherein the client determines the latency and transmits the latency to the server in a reply, such that the server is allowed to react to the latency. ### Claim 10 (depends on 1) The method of claim 1, wherein the server sends the plurality of successive frames at a frame rate that is limited by the proxy to a maximum bandwidth by the proxy forwarding frames to the client within a given time segment only until the maximum bandwidth is reached, and not forwarding to the client any of subsequent frames within a time segment. ### Claim 11 (depends on 1) The method of claim 1, wherein the plurality of successive frames from the server are forwarded by the proxy at a frame rate that is adapted dynamically by ascertaining a round-trip time between the server and the client, and depending on the round-trip time, forwarding only a subset of frames of the plurality of successive frames to the client, wherein the subset of frames that is forwarded is defined by an adjustable factor that specifies a number of successive frames, of which one is forwarded and all others are discarded, and wherein the adjustable factor is increased by one when the round-trip time exceeds an upper threshold value, and is decreased by one when the round-trip time goes below a lower threshold value. ### Claim 12 (depends on 1) The method of claim 1, wherein at a given point in time, only some of a plurality of data streams from a plurality of servers are displayed on the client, and the remaining data streams of the plurality of data streams are paused, so that frames are not transferred to the client, the plurality of servers including the server. ### Claim 13 (depends on 1) The method of claim 1, wherein the proxy is connected via an external network to the server or to a plurality of servers, the plurality of servers including the server, and wherein the network is a standard Ethernet network or a WLAN network. ### Claim 14 (depends on 1) The method of claim 1, wherein the server is a first medical device, wherein the proxy and the client are both part of a second medical device, and wherein the client is a monitor, so that a user interface of the first medical device is looped through to a monitor of the second medical device such that the first medical device is controllable from the second medical device. ### Claim 15 A medical device comprising: a proxy; and a client, wherein the medical device is configured to: transfer a user interface of a server to the client such that the user interface is displayed on the client, and the server is controllable from the client; transmit a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determine a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, and wherein the client is configured to determine the latency and transmit the latency to the server in a reply, such that the server is allowed to react to the latency. ### Claim 16 A server comprising: a processor configured to transfer a user interface from at least one server via a proxy to at least one client, the transfer comprising: transfer of a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayable on the client, and the server is controllable from the client; and transmission of a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayable on the client, wherein the at least one client is configured to determine a latency of the transfer recurrently for individual frames of the plurality of successive frames and transmit the latency to the server in a reply, such that the transfer is monitored and the server is allowed to react to the latency. ### Claim 17 A method for transferring a user interface from at least one server via a proxy to at least one client, the method comprising: transferring a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayed on the client, and the server is controllable from the client; transmitting a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determining a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, wherein the client determines the latency and compares the latency with at least one threshold value, and wherein the client outputs an indicator, disables the user interface of the server on the client, or outputs the indicator and disables the user interface of the server on the client when the latency exceeds the threshold value. ### Claim 18 A method for transferring a user interface from at least one server via a proxy to at least one client, the method comprising: transferring a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayed on the client, and the server is controllable from the client; transmitting a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determining a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, wherein the client determines the latency and transmits the latency to the server in a reply, such that the server is allowed to react to the latency. ### Claim 19 A method for transferring a user interface from at least one server via a proxy to at least one client, the method comprising: transferring a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayed on the client, and the server is controllable from the client; transmitting a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determining a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, wherein the server sends the plurality of successive frames at a frame rate that is limited by the proxy to a maximum bandwidth by the proxy forwarding frames to the client within a given time segment only until the maximum bandwidth is reached, and not forwarding to the client any of subsequent frames within a time segment. ### Claim 20 A method for transferring a user interface from at least one server via a proxy to at least one client, the method comprising: transferring a user interface of a server of the at least one server to a client of the at least one client such that the user interface is displayed on the client, and the server is controllable from the client; transmitting a data stream of a plurality of successive frames from the server to the client by a packet-based transfer, such that the user interface is displayed on the client; and determining a latency of the transfer recurrently for individual frames of the plurality of successive frames, such that the transfer is monitored, wherein the plurality of successive frames from the server are forwarded by the proxy at a frame rate that is adapted dynamically by ascertaining a round-trip time between the server and the client, and depending on the round-trip time, forwarding only a subset of frames of the plurality of successive frames to the client, wherein the subset of frames that is forwarded is defined by an adjustable factor that specifies a number of successive frames, of which one is forwarded and all others are discarded, and wherein the adjustable factor is increased by one when the round-trip time exceeds an upper threshold value, and is decreased by one when the round-trip time goes below a lower threshold value.