US 2026/0087683 A1Published application

Dynamic Generation of Data Objects Using Smart Safety Glasses

AssigneeProcore Technologies, Inc.

InventorsAaron Kettl, Bharath Hariharan, David Borden, Sharmishtha Gupta

Life of this patent

Estimating the term…

Loading patent…

Abstract

A smart wearable device for hands-free data capture facilitates documentation of real-time observations and creation of new data objects within a software application based on the observations. Image data depicting one or more physical objects observed by a user of the smart wearable device, voice data indicating a user request related to the physical objects depicted in the image data, and contextual information related to the user and the physical objects depicted in the image data, are used to generate a prompt for input to a generative AI model that is to respond to the user request. Based on the prompt, the generative AI model outputs an application-specific command that is to be carried out by a given software tool of the software application, such as a command to create a new data object of a given type within the software application.

Claims20

Dependent claims

A computing system comprising:

an end-user device;
a wearable computing device;
at least one processor;
at least one non-transitory computer-readable medium;
and
program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is configured to:
obtain, via a sensor system of the wearable computing device, (i) image data depicting one or more objects associated with a construction project and (ii) voice data indicating a request related to the one or more objects depicted in the image data;
determine, via the end-user device, contextual information related to the image data;
based on the voice data, identify the request, wherein the request comprises a request to generate a data object of a given type within a construction management software application;
based on the image data, the contextual information, and the request, generate a prompt that is to be provided as input to a generative AI model;
provide the prompt as input to the generative AI model;
based on the prompt, generate, via the generative AI model, a command for the construction management software application to generate the data object of the given type, wherein the data object of the given type comprises data related to the one or more objects depicted in the image data;
execute the command via the construction management software application and thereby generate the data object of the given type;
and
cause a representation of the data object of the given type to be displayed via the end-user device.

12 more elements

A non-transitory computer-readable medium, wherein the non-transitory computer-readable medium is provisioned with program instructions that, when executed by at least one processor, cause a computing system to:

obtain, via a sensor system of a wearable computing device, (i) image data depicting one or more objects associated with a construction project and (ii) voice data indicating a request related to the one or more objects depicted in the image data;
determine, via an end-user device, contextual information related to the image data;
based on the voice data, identify the request, wherein the request comprises a request to generate a data object of a given type within a construction management software application;
based on the image data, the contextual information, and the request, generate a prompt that is to be provided as input to a generative AI model;
provide the prompt as input to the generative AI model;
based on the prompt, generate, via the generative AI model, a command for the construction management software application to generate the data object of the given type, wherein the data object of the given type comprises data related to the one or more objects depicted in the image data;
execute the command via the construction management software application and thereby generate the data object of the given type;
and
cause a representation of the data object of the given type to be displayed via the end-user device.

6 more elements

A method carried out by a computing system, the method comprising:

obtaining, via a sensor system of a wearable computing device, (i) image data depicting one or more objects associated with a construction project and (ii) voice data indicating a request related to the one or more objects depicted in the image data;
determining, via an end-user device, contextual information related to the image data;
based on the voice data, identifying the request, wherein the request comprises a request to generate a data object of a given type within a construction management software application;
based on the image data, the contextual information, and the request, generating a prompt that is to be provided as input to a generative AI model;
providing the prompt as input to the generative AI model;
based on the prompt, generating, via the generative AI model, a command for the construction management software application to generate the data object of the given type, wherein the data object of the given type comprises data related to the one or more objects depicted in the image data;
executing the command via the construction management software application and thereby generate the data object of the given type;
and
causing a representation of the data object of the given type to be displayed via the end-user device.

6 more elements

Dependencies are inferred from the claim text.