# US 2023/0274142 A1 · Method for training a conditional neural process for determining a position of an object from image data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0274142 A1 - Document ID: US20230274142A1 - Status: Published application - Kind: A1 - Application number: 18/167,733 - Filed: 2023-02-10 - Published: 2023-08-31 - Earliest claimed: 2022-02-28 - Applicants: Robert Bosch GmbH - Inventors: Ning Gao; Anh Vien Ngo; Gerhard Neumann; Hanna Ziesche; Michael Volpp - Web page: https://patentgrep.com/patent/US20230274142A1 ## CPC - G06N 3/08: Learning methods - G06N 3/045: Combinations of networks - G06N 3/0499: Feedforward networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06V 10/225: {based on a marking or identifier characterising the area} - G06V 10/82: using neural networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for training a conditional neural process for determining a position of an object from image data. The method includes: providing training data for training the conditional neural process, wherein the training data comprise labeled image data showing a particular object and labeled comparison image data regarding the particular object; and training the conditional neural process based on the provided training data, wherein the training of the conditional neural process comprises applying functional contrastive learning, and wherein the training of the conditional neural process comprises applying an end-to-end learning approach. ## Claims ### Claim 1 A method for training a conditional neural process for determining a position of an object from image data, the method comprising the following steps: providing training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and training the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach. ### Claim 2 (depends on 1) The method according to claim 1, wherein the step of training the conditional neural process based on the provided training data furthermore includes the following steps: generating first latent representations based on the labeled image data and information about the labeled image data; generating second latent representations based on the labeled comparison image data and the information about the labeled comparison image data; determining, using the functional contrastive learning, a first cost function based on the first latent representations and the second latent representations; and training the conditional neural process based on the first cost function. ### Claim 3 (depends on 1) The method according to claim 1, wherein the step of training the conditional neural process based on the provided training data furthermore includes the following steps: determining, using the conditional neural process, a position of the particular object in the image data based on the labeled image data, the labeled comparison image data, and information about the labeled comparison image data; determining a comparison position of the particular object in the labeled image data based on the information about the labeled image data; determining a second cost function based on the determined position of the particular object in the image data and the comparison position of the particular object; and training the conditional neural process based on the second cost function. ### Claim 4 (depends on 1) The method according to claim 1, wherein the image data and the comparison image data respectively are image data showing complete images. ### Claim 5 A method for determining a position of an object, the method comprising the following steps: providing image data, wherein the image data include target image data showing the object and labeled comparison image data regarding the object; providing a trained conditional neural process, the conditional neural process being trained for determining a position of an object from image data by: providing training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and training the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach; and determining, using the trained conditional neural process for determining a position of an object from image data, the position of the object based on the provided image data. ### Claim 6 A method for controlling a controllable system, the method comprising the following steps: determining a position of an object by: providing image data, wherein the image data include target image data showing the object and labeled comparison image data regarding the object; providing a trained conditional neural process, the conditional neural process being trained for determining a position of an object from image data by: providing training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and training the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach; and determining, using the trained conditional neural process for determining a position of an object from image data, the position of the object based on the provided image data; and controlling the controllable system based on the determined position of the object. ### Claim 7 A control device for training a conditional neural process for determining a position of an object from image data, the control device comprising: a provisioning unit configured to provide training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and a training unit configured to train the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach. ### Claim 8 (depends on 7) The control device according to claim 7, wherein the training unit includes: a first generation unit configured to generate first latent representations based on the labeled image data and information about the labeled image data; a second generation unit configured to generate second latent representations based on the labeled comparison image data and information about the labeled comparison image data; and a first determination unit configured to determine, using the functional contrastive learning, a first cost function based on the first latent representations and the second latent representations, and wherein the training unit is configured to train the conditional neural process based on the first cost function. ### Claim 9 (depends on 8) The control device according to claim 8, wherein the training unit includes: a second determination unit configured to determine, using the conditional neural process, a position of the particular object in the image data based on the labeled image data, the labeled comparison image data, and the information about the labeled comparison image data; a third determination unit configured to determine a comparison position of the particular object in the labeled image data based on the information about the labeled image data; and a fourth determination unit configured to determine a second cost function based on the determined position of the object in the image data and the comparison position of the object; wherein the training unit is configured to train the conditional neural process based on the second cost function. ### Claim 10 (depends on 7) The control device according to claim 7, wherein the image data and the comparison image data respectively are image data showing complete images. ### Claim 11 A control device for determining a position of an object, the control device comprising: a provisioning unit configured to provide image data, wherein the image data comprise target image data showing the object and labeled comparison image data regarding the object; a reception unit configured to receive a trained conditional neural process, the conditional neural process being trained by a control device for training a conditional neural network for determining a position of an object from image data for determining a position of an object from image data, the control device for training including: a provisioning unit configured to provide training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and a training unit configured to train the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach; and a determination unit configured to determine, using the provided trained conditional neural process for determining an object from image data, the position of the object based on the provided image data. ### Claim 12 A control device for controlling a controllable system, the control device comprising: a reception unit configured to receive a position of an object determined by a control device for determining a position of an object including: a provisioning unit configured to provide image data, wherein the image data comprise target image data showing the object and labeled comparison image data regarding the object; a reception unit configured to receive a trained conditional neural process, the conditional neural process being trained by a control device for training a conditional neural network for determining a position of an object from image data for determining a position of an object from image data, the control device for training including: a provisioning unit configured to provide training data for training the conditional neural process, wherein the training data include labeled image data showing a particular object and labeled comparison image data regarding the particular object; and a training unit configured to train the conditional neural process based on the provided training data, wherein the training of the conditional neural process includes applying functional contrastive learning, and the training of the conditional neural process includes applying an end-to-end learning approach; and a determination unit configured to determine, using the provided trained conditional neural process for determining an object from image data, the position of the object based on the provided image data; and a control unit configured to control the controllable system based on the determined position of the object.