# US 2021/0083992 A1 · Augmented reality for internet connectivity installation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0083992 A1 - Document ID: US20210083992A1 - Status: Published application - Kind: A1 - Application number: 17/019,184 - Filed: 2020-09-11 - Published: 2021-03-18 - Earliest claimed: 2019-09-13 - Assignees: Ubiquiti Inc. - Applicants: Ubiquiti Inc. - Inventors: Samuel DIDEAR; Koji KANAO; Robert J. PERA; Heqing HUANG; Michael ANDERSON - Web page: https://patentgrep.com/patent/US20210083992A1 ## CPC - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - G06K 19/06028: {using bar codes} - G06T 15/005: {General purpose rendering architectures} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 20/20: in augmented reality scenes - H04L 12/4641: {Virtual LANs, VLANs, e.g. virtual private networks [VPN] (LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272)} - H04L 43/0811: by checking connectivity - H04L 45/02: Topology update or discovery - H04L 45/64: using an overlay routing layer - H04L 49/70: {Virtual switches} - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04L 67/131: Protocols for games, networked simulations or virtual reality - H04L 67/75: Indicating network or usage conditions on the user display - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Apparatuses (e.g., systems and devices) and methods to view otherwise hidden connectivity of networking devices. An electronic device such as a network device connected to a variety of different devices, may be observed through a real-time image onto which information about the connectivity and/or connections may be display to show identity, connectivity state and other information associated with ports of the network device(s). These methods may be implemented on a mobile device used to capture images of the network device and present an overlay of virtual objects on the captured images to a user in real time. The virtual objects may dynamically move or change shape in real time depending on the movement of the mobile device. One or more filtering techniques may be used to stabilize the virtual objects with respect to the captured images. ## Claims ### Claim 1 A method of displaying a network device with augmented reality using a mobile device, the method comprising: receiving an identifying code from the network device by the mobile device; displaying captured camera images of the network device on a display of the mobile device, the images including a plurality of ports of the network device; retrieving information associated with a connectivity state of each of the plurality of ports using the identifying code; determining a spatial relationship of the mobile device with respect to the one or more ports using the identifying code; and overlaying one or more virtual ports over the network device on the images, wherein the virtual ports include information about the connectivity of the network device in the one or more virtual ports. ### Claim 2 (depends on 1) The method of claim 1, wherein an alignment of the one or more virtual ports with respect to the camera images is dynamically adjusted based on the determined spatial relationship. ### Claim 3 (depends on 1) The method of claim 1, further wherein each of the virtual ports indicates the connectivity state of the port based on the retrieved information. ### Claim 4 (depends on 1) The method of claim 1, further comprising overlaying a virtual label on the image proximate to each of the virtual ports, the virtual label including text indicating one or more characteristics of a device connected to each of the ports corresponding to each of the virtual ports. ### Claim 5 (depends on 1) The method of claim 1, wherein each of the virtual ports has a size and shape that substantially matches that of the image of the one or more ports. ### Claim 6 (depends on 1) The method of claim 1, wherein a position of the virtual port dynamically changes in accordance with perspective changes of a camera of the mobile device with respect to the network device. ### Claim 7 (depends on 1) The method of claim 1, wherein each of the virtual ports has a different appearance when a corresponding port in the network device is connected, as compared to when the corresponding port is not connected to a device. ### Claim 8 (depends on 1) The method of claim 1, further comprising accessing a library of virtual objects based on the information associated with connectivity state of the port, the virtual objects including one or more icons corresponding to a type of device connected to one or more ports of the network device, wherein the virtual ports include the one or more icons. ### Claim 9 (depends on 8) The method of claim 8, wherein the type of the device is a mobile phone, a tablet computer, a laptop computer, a desktop computer, a wireless access point device, a virtual assistant device, a television, a home appliance, or a security system device. ### Claim 10 (depends on 8) The method of claim 8, wherein the one or more icons has a different appearance when the port is connected to the device compared to when the port was previously connected and is no longer connected to the device. ### Claim 11 (depends on 1) The method of claim 1, further comprising stabilizing the one or more virtual ports with respect to the network device by filtering out one or more degrees of freedom used to determine the spatial relationship of the mobile device with respect to the network device. ### Claim 12 (depends on 11) The method of claim 11, wherein filtering out one or more degrees of freedom comprises eliminating data associated with rotation about an axis in coordinate space. ### Claim 13 (depends on 1) The method of claim 1, wherein the identifying code is an optical code that is on a touch display of the network device. ### Claim 14 (depends on 1) The method of claim 1, wherein the identifying code is an RF code. ### Claim 15 (depends on 1) The method of claim 1, wherein the connectivity state of the port includes whether the port is connected to the device. ### Claim 16 (depends on 1) The method of claim 1, wherein the connectivity state of the port includes information associated with an operating status of a computer connected to the port or that was connected to the port. ### Claim 17 (depends on 1) The method of claim 1, wherein the connectivity state of the port includes information as to whether a device that is connected or was connected to the port is connected to a second device. ### Claim 18 (depends on 17) The method of claim 17, wherein the connectivity state of the port includes an operating status of the device and an operating status of the second device. ### Claim 19 (depends on 1) The method of claim 1, wherein the mobile device is a mobile phone, tablet computer, laptop computer or headset. ### Claim 20 (depends on 1) The method of claim 1, wherein the network device is a network switch, router or access point. ### Claim 21 (depends on 1) The method of claim 1, further comprising dynamically adjusting the overlay of the one or more virtual ports. ### Claim 22 A method of displaying a network device with augmented reality using a mobile device, the method comprising: displaying captured camera images of the network device on a display of the mobile device, the images including: one or more ports of the network device, and an optical code on the network device that includes network connectivity information associated with the network device; retrieving information associated with a connectivity state of the one or more ports using the optical code; accessing a library of virtual objects based on the information associated with connectivity state of the one or more ports, the virtual objects including an icon corresponding to a type of device connected to each of the one or more ports of the network device; determining a spatial relationship of the mobile device with respect to the one or more ports using the optical code; and overlaying one or more virtual ports, wherein the virtual ports each include the icon corresponding to the type of device connected to the one or more ports, over the image of the network device, wherein a quality of the virtual port indicates the connectivity state of the port based on the retrieving information. ### Claim 23 A system for displaying a network device with augmented reality using a mobile device, the system comprising: one or more processors; memory coupled to the one or more processors, the memory configured to store computer-program instructions, that, when executed by the one or more processors, perform a computer-implemented method comprising: receiving an identifying code from the network device by the mobile device; displaying captured camera images of the network device on a display of the mobile device, the images including a plurality of ports of the network device; retrieving information associated with a connectivity state of each of the plurality of ports using the identifying code; determining a spatial relationship of the mobile device with respect to the one or more ports using the identifying code; and overlaying one or more virtual ports over the network device on the images, wherein the virtual ports include information about the connectivity of the network device in the one or more virtual ports. ### Claim at position 24 -35. (canceled)