# US 10,122,963 B2 · Bidirectional audio/video: system and method for opportunistic scheduling and transmission > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,963 B2 - Document ID: US10122963B2 - Status: Granted - Kind: B2 - Application number: 13/960,113 - Filed: 2013-08-06 - Published: 2014-12-11 - Granted: 2018-11-06 - Earliest claimed: 2013-06-11 - Assignees: Milestone AV Technologies LLC - Applicants: Milestone AV Technologies LLC - Inventors: Tim Wall; Roger Holschbach - Web page: https://patentgrep.com/patent/US10122963B2 ## CPC - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 7/15: Conference systems - H10W 72/5445: {being orthogonal to a side surface of the chip, e.g. parallel arrangements} - H10W 90/753: {between laterally-adjacent chips} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H10W: GENERIC PACKAGES, INTERCONNECTIONS, CONNECTORS OR OTHER CONSTRUCTIONAL DETAILS OF DEVICES COVERED BY CLASS H10 ## Abstract A bidirectional transceiver includes a media transmitter, a media receiver, a network transmitter, and a network receiver. Each of these components includes an interface to a respective differential wire pair of a cable. By coupling a pair of such transceivers, a media stream and network connectivity can be sent over the cable in either or both directions. Using an opportunistic scheduling method, each media stream can contain audio, video, and control information in uncompressed form. Power can be transferred over the cable so that a media device containing one transceiver can be powered by a media device containing the other transceiver. ## Claims ### Claim 1 A system, comprising: a) a first bidirectional transceiver, which includes (i) a first digital media transmitter, coupled to an interface to a first differential wire pair of a cable; (ii) a first media receiver, coupled to an interface to a second differential wire pair of the cable; (iii) a first network transmitter, coupled to an interface to a third differential wire pair of the cable; (iv) a first network receiver, coupled to an interface to a fourth differential wire pair of the cable; and (v) a DC voltage comparator, which is used prior to applying power by the first bidirectional transceiver with a current-limited probe voltage switched onto a first half of the second differential wire pair connected on a source side, wherein the DC voltage comparator is connected on a second half of the second differential wire pair on the source side to determine whether an electrical load is connected to the third and fourth differential wire pairs by detecting a first known voltage resulting from the current-limited probe voltage switching if the electrical load is connected and detecting a second known voltage resulting from the current-limited probe voltage switching if the electrical load is not connected. ### Claim 2 (depends on 1) The system of claim 1, further comprising: b) a second bidirectional transceiver, which includes (i) a second digital media transmitter; (ii) a second digital media receiver; (iii) a second network transmitter; and (iv) a second network receiver, c) the cable, wherein (i)the first differential wire pair is coupled to the second digital media receiver; (ii) the second differential wire pair is coupled to the second digital media transmitter; (iii) the third differential wire pair is coupled to the second network receiver; and (iv) the fourth differential wire pair is coupled to the second network transmitter. ### Claim 3 (depends on 2) The system of claim 2, comprising: d) a power supply, electrically coupled in the first bidirectional transceiver to the third and fourth differential wire pairs; and e) a load, electrically coupled in the second bidirectional transceiver to the third and fourth differential wire pairs, and powered by the power supply. ### Claim 4 (depends on 3) The system of claim 3, wherein the cable is CAT 5, CAT 6, or CAT 7 cable. ### Claim 5 (depends on 2) The system of claim 2, wherein the first differential wire pair is used to detect a disconnect of a device previously connected to the second bidirectional transceiver. ### Claim 6 (depends on 1) The system of claim 1, further comprising: b) a first media device that (i) receives a first digital audio stream through a first interface and a first digital video stream through a second interface, (ii) combines the first digital audio stream and the first digital video stream into a first media stream, wherein the first digital audio stream and the first digital video stream are unencoded; (iii) transmits the first media stream using the first digital media transmitter in uncompressed form in real-time. ### Claim 7 (depends on 6) The system of claim 6, wherein, after combining the first digital audio stream and the first digital video stream, the first media device encodes the first media stream using SerDes encoding. ### Claim 8 (depends on 6) The system of claim 6, further comprising: c) an assembler/scheduler, in the first media device, that (i) establishes a priority ordering among input data types, wherein the input data types include audio data from the first digital audio stream, and video data from the first digital video stream, and (ii) inserts the input data types into the first media stream, sequenced according to the priority ordering on a bandwidth-available basis. ### Claim 9 (depends on 8) The system of claim 8, wherein a data of a given type is entered into the first media stream only when data of types having higher priority are not available for insertion. ### Claim 10 (depends on 8) The system of claim 8, wherein the input data types further include control information, which pertains to control and status of the system. ### Claim 11 (depends on 6) The system of claim 6, further comprising; c) a second media device that (i) includes a second bidirectional transceiver, and (ii) receives the first media stream using a second media receiver, within the second bidirectional transceiver; and d) assembler/scheduler logic, within the second media device, that (i) extracts data from the first media stream, (ii) separates the data according to data type, the data types including an audio data and video data, (iii) synchronizes the audio data with the video data, and (iv) after synchronization, creates a second audio stream from the audio data, and a second video stream from the video data. ### Claim 12 (depends on 11) The system of claim 11, wherein synchronizing audio data with video data uses interpolation in time of adjacent frames. ### Claim 13 (depends on 1) The system of claim 1, wherein the DC voltage comparator is further used by the first bidirectional transceiver to determine whether a device connected to the third and fourth differential wire pairs is supported, thereby protecting equipment from load damage. ### Claim 14 (depends on 13) The system of claim 13, wherein load detection and determination of whether a device is supported are performed using the first and second differential wire pairs. ### Claim 15 A media transmitter apparatus, comprising: a) a video source interface and an audio source interface; b) input formatter logic, which includes (i) logic that reformats a real-time digital video stream received through the video source interface into a first video stream in a predetermined video format, and (ii) logic that reformats a real-time digital audio stream received through the audio source interface into a first audio stream in a predetermined audio format; c) packetizer logic, which creates packets from the first audio stream and the first video stream, a plurality of the audio packets and a plurality of the video packets each having a header that includes metadata. regarding the respective packet, wherein the metadata specifies a source of the respective stream specifying a resource other than the media transmitter apparatus, a stream destination specifying a far-end media appliance, and a description of media type; d) input adaptive rate multiplexer logic, which inserts the packets into a first media stream, sequencing the packets according to a packet prioritization scheme, wherein the first media stream contains at least one control side-band channel comprising, control data packets; e) a cable Interface; f) transport logic, which encodes the first media stream into a second media stream, and transmits the second media stream through the cable interface; and g) an adaptive rate controller communicatively coupled to a receiving adaptive rate controller and configured to regulate transmission speed of the second media stream to correct at least one error condition determined based on the control side-band channel control data packets. ### Claim 16 (depends on 15) The media transmitter apparatus of claim 15, wherein audio information from the audio source interface and video information from the video source interface are in uncompressed form in the second media stream. ### Claim 17 (depends on 15) The media transmitter apparatus of claim 15, further comprising: a control source interface; wherein the second media stream includes a representation of control information that is received through the control source interface. ### Claim 18 (depends on 15) The media transmitter apparatus of claim 15, further comprising: a protocol control interface, through which the media transmitter apparatus receives requests to modify a rate at which the media transmitter apparatus transmits packets through the cable interface. ### Claim 19 (depends on 15) The media transmitter apparatus of claim 15, wherein the input adaptive rate multiplexer logic, upon detecting a period of lower bandwidth of high priority packets, injects a lower priority packet into the sequence of the packets. ### Claim 20 (depends on 19) The media transmitter apparatus of claim 19, wherein the high priority packets include audio packets. ### Claim 21 (depends on 19) The media transmitter apparatus of claim 19, wherein low priority packets include control packets. ### Claim 22 (depends on 15) The media transmitter apparatus of claim 15, wherein the metadata specifies a source and a destination for the respective stream. ### Claim 23 (depends on 22) The media transmitter apparatus of claim 22, wherein the metadata further specifies a media type for the respective stream. ### Claim 24 (depends on 22) The media transmitter apparatus of claim 22, wherein the metadata further specifies a format for the respective stream. ### Claim 25 (depends on 22) The media transmitter apparatus of claim 22, wherein the metadata further specifies a size for the respective stream. ### Claim 26 (depends on 22) The media transmitter apparatus of claim 22, wherein the metadata further specifies handshaking and data validity information. ### Claim 27 (depends on 22) The media transmitter apparatus of claim 22, wherein the input adaptive rate multiplexer logic, upon detecting a period of lower bandwidth of high priority packets, injects a lower priority packet into the sequence of the packets. ### Claim 28 (depends on 15) The media transmitter apparatus of claim 15, wherein the adaptive rate controller prevents overflow conditions, monitors error conditions, and triggers retries of transmissions. ### Claim 29 (depends on 15) The media transmitter apparatus of claim 15, wherein the input adaptive rate multiplexer logic maintains real-time transmission by dropping at least one packet. ### Claim 30 (depends on 15) The media transmitter apparatus of claim 15, wherein the metadata further includes a destination for the respective stream at a target appliance external to the media transmitter apparatus. ### Claim 31 (depends on 15) The media transmitter apparatus of claim 15, wherein the metadata further includes a media format and a size. ### Claim 32 A system, comprising: a) a media receiver apparatus, including (i) a cable interface; (ii) transport logic, which receives a first media stream, in uncompressed form in real-time, through the cable interface, and decodes the first media stream into a second media stream, wherein the first media stream contains at least one control side-band channel comprising control data packets; (iii) output adaptive rate multiplexer logic, which removes packets from the second media stream; (iv) depacketizer logic, which removes headers that contain packet metadata from packets, and creates audio frames, video frames, and system control information, wherein the metadata for plurality of audio packets, video packets, and system control packets each contain a stream source specifying a resource other than a media transmitter from which the respective packet was received, a stream destination specifying a far-end media appliance, a size, and a description of media type; (v) reclocker logic, which synchronizes the audio and video frames using header metadata, and creates a first audio stream in a predetermined audio format, and a first video stream in a predetermined video format; (vi) output formatter logic, which reformats the first audio stream through an audio output interface in real-time, and reformats the first video stream through a video output interface in real-time; and (vii) adaptive rate controller logic in electrical communication with a transmitting rate controller and configured to regulate transmission speed of the first media stream to correct at least one error condition determined based on the control sideband channel control data packets. ### Claim 33 (depends on 32) The system of claim 32, the media receiver apparatus further including a control destination interface, through which the output formatter logic sends a representation of system control information contained in the first media stream. ### Claim 34 (depends on 32) The system of claim 32, the media receiver apparatus further including a protocol control interface, through which the media receiver apparatus transmits a request to modify a current rate at which the media transmitter is receiving packets through the cable interface. ### Claim 35 (depends on 34) The system of claim 34, wherein the request is triggered by a determination that data in a buffer has exceeded a certain amount. ### Claim 36 (depends on 34) The system of claim 34, further comprising: b) a cable, coupled to the cable interface of the media receiver apparatus; c) a media transmitter apparatus, to which the cable is coupled, that modifies a current rate at which media transmitter transmits packets through the cable in response to the request.