# US 2021/0280291 A1 · Augmented reality for drug delivery devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0280291 A1 - Document ID: US20210280291A1 - Status: Published application - Kind: A1 - Application number: 17/328,380 - Filed: 2021-05-24 - Published: 2021-09-09 - Earliest claimed: 2018-11-28 - Assignees: Ypsomed AG - Applicants: Ypsomed AG - Inventors: Florian Kühni; Dominik Reubi - Web page: https://patentgrep.com/patent/US20210280291A1 ## CPC - G16H 20/17: delivered via infusion or injection - G16H 40/67: for remote operation - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract Computer implemented methods for supporting an operation of a drug delivery device include the steps of: capturing, by a camera of a mobile device, a marker representative of a position of the drug delivery device; transmitting, by the mobile device, camera data of the camera relating to the captured marker to a provider; providing in response to the transmitted camera data, by the provider, drug delivery device data to the mobile device; displaying, by the mobile device, in a camera preview the received drug delivery device data, wherein the displaying may be controlled by a projection of the marker. The drug delivery device data includes instructions to a user relating to an operation of the drug delivery device. ## Claims ### Claim 1 A computer implemented method for supporting an operation of a drug delivery device for injection or infusion, the method comprising the steps of: using a camera of a mobile device to capture a marker of the drug delivery device representative of a position of the drug delivery device; using the mobile device to transmit camera data of the camera relating to the captured marker to a provider; using the mobile device to receive drug delivery device data transmitted from the provider in response to the transmitted camera data; and using the mobile device to display in a camera preview on a display screen of the mobile device the provided drug delivery device data, wherein the display is controlled by a projection of the marker, and wherein the drug delivery device data comprises instructions to a user relating to a drug delivery operation of the drug delivery device. ### Claim 2 (depends on 1) The method according to claim 1, wherein the instructions comprise operation information about setting a dose using the drug delivery device. ### Claim 3 (depends on 1) The method according to claim 1, wherein the instructions comprise one or more of: operation information about adjusting settings of the drug delivery device, preparing the drug delivery device for an injection or an infusion, exchanging a cartridge in the drug delivery device, mounting a needle, correcting a set dose, administering a set dose or disposal of the drug delivery device. ### Claim 4 (depends on 1) The method according to claim 1, wherein the display of the drug delivery device data in the camera preview is implemented through a website displayed on the display screen by a web browser of the mobile device. ### Claim 5 (depends on 1) The method according to claim 1, wherein the drug delivery device is configured as a pen shaped injector or a patch injector. ### Claim 6 (depends on 1) The method according to claim 1, further comprising, prior to the step of using the camera of the mobile device to capture the marker, performing an authentication process, the authentication process comprising the steps of: using a sensor of the mobile device to recognize a machine-readable identification code and reading out information from the code, wherein the code is assigned to the drug delivery device; using the mobile device to automatically contact the provider based on the read out information; receiving a request to access camera data of the mobile device from the provider in response to contacting the provider; and receiving a manual confirmation by the user on the mobile device authorizing access to the camera data by the provider. ### Claim 7 (depends on 1) The method according to claim 1, further comprising the steps of: using a sensor of the mobile device to recognize a machine-readable identification code and reading out information from the code, wherein the code is assigned to the drug delivery device; using the mobile device to transmit the read out information from the identification code to the provider; receiving, from the provider in response to the transmitted information, at least one of data relating to the drug delivery device and data relating to a medication contained in the drug delivery device; and using the mobile device to display, in the camera preview, the data received from the provider. ### Claim 8 (depends on 7) The method according to claim 7, wherein the data relates to the medication contained in the drug delivery device and comprises at least one of: medication type, expiration of medication, or validity of medication. ### Claim 9 (depends on 7) The method according to claim 7, wherein the data relates to the medication contained in the drug delivery device, and wherein the provider compares the information about the medication with previously stored user-specific medication data, and the method further comprises the steps of: using the mobile device to receive from the provider a result of the comparison; and using the mobile device to display in the camera preview the result of the comparison. ### Claim 10 (depends on 1) The method according to claim 1, further comprising an electronic module integrated in or attachable to the drug delivery device, the electronic module adapted to monitor a drug delivery process executed by the drug delivery device, wherein the method further comprises the steps of: using the electronic module to record delivery data related to the drug delivery process; using the electronic module to transmit the delivery data to the mobile device; and using the mobile device to display the delivery data in the camera preview. ### Claim 11 (depends on 10) The method according to claim 10, wherein the delivery data related to the drug delivery process comprise at least one of: an amount of a last administered dose, an indication of time of a last dose, an amount of the next dose to be administered, an indication of time of a next dose to be administered, or a total amount of an administered dose. ### Claim 12 (depends on 1) The method according to claim 1, further comprising the steps of: pairing the mobile device with a remote computing device; and using the mobile device to transmit camera data captured by the camera of the mobile device to the remote computing device. ### Claim 13 (depends on 1) The method according to claim 1, further comprising the step of using the mobile device to output an acoustic signal based on the drug delivery device data received from the provider. ### Claim 14 A web-based computer program product comprising instructions which, when the program is executed by a computing device, cause the computing device to carry out the steps of: receiving, from a mobile device, camera data of a camera of the mobile device, the camera data relating to a captured marker, wherein the marker is representative of a position of the drug delivery device; and providing, in response to the received camera data, drug delivery device data to the mobile device, wherein the drug delivery device data comprises instructions to a user relating to a drug delivery operation of the drug delivery device. ### Claim 15 A method of using a mobile device, the mobile device comprising a processor configured to perform the steps of: using a camera of the mobile device to capture a marker representative of a position of a drug delivery device; using the mobile device to transmit camera data of the camera relating to the captured marker to a provider; using the mobile device to receive, in response to the transmitted camera data, drug delivery device data from the provider; and using the mobile device to display, in a camera preview on a display screen of the mobile device, the provided drug delivery device data, wherein the display is controlled by a projection of the marker, and wherein the drug delivery device data comprises instructions to a user relating to a drug delivery operation of the drug delivery device.