Imaging system for object recognition and assessment

#The present invention relates generally to a system which combines one or more sensors configured to capture two-dimensional and/or three dimensional image data of one or more objects. In particular, the present invention relates to a system configured to capture and export the spectral image data and dimensional data of one or more objects, such as for example food items, to a network based application. The network based application portion of the system draws upon a plurality of databases of algorithms and tabulated data to facilitate characteristics of the one or more objects, and implementation of comprehensive programs for users to automatically track and monitor desired variables, features, and characteristics of the one or more objects.

#Generally, people have been using a multitude of different methods and devices to track dietary food consumption and fitness. These methodologies include systems for counting calories, tracking points, excluding or limiting certain types of foods, etc. These systems can also include fitness tracking for estimating how many calories the user may have burned in relation in estimated caloric intake. Overall, the widespread use of smartphones, tablets, and other mobile devices has revolutionized the way in which many people monitor and guide their food consumption habits.

#However, these devices, systems, applications, and methodologies experience several shortcomings. In particular, the current diet monitoring technologies experience several shortcomings based in part on shortcomings in the object imaging and characterization technologies. The most popular food related tracking and recording applications are intended for use with a smartphone, allowing users to download the application, set up a user account in which the user enters their goals for weight loss, amount of exercise, macronutrient intake, blood pressure, sleep patterns, etc. Conventional applications require the user to manually log their food intake and exercise through self-reporting based on the selections from a network database of foods and associated values. Using the manually reported information, diet monitoring applications typically output nutritional information based on the user input. Many users find the manual entry of foods difficult, bothersome, and confusing, which leads to users failing to use the application consistently or correctly for long periods of time and in turn reducing the effectiveness of such existing technology.

#These and other characteristics of the present invention will be more fully understood by reference to the following detailed description in conjunction with the attached drawings, in which:

#FIG. 1 depicts a system for capturing and automatically determining characteristics of one or more objects, in accordance with the present invention;

#FIG. 2 depicts the system for capturing and automatically determining characteristics of one or more objects, implemented for food item objects, in accordance with aspects of the present invention;

#An illustrative embodiment of the present invention relates to a system for imaging and determining characteristics of (e.g., identifying, analyzing, etc.) one or more objects at the same time and obtaining additional information about the one or more objects based at least in part on identification and volume determinations of the one or more objects without requiring use of a fiduciary object as a reference for determining scale, or the like. In particular, the present invention relates to a system using a device utilizing both visible and near infrared three-dimensional imaging for identifying and analyzing one or more objects, such as food items, within a field of view, and determining the volume and other characteristics of those objects, such as caloric and nutritional values for food items. The technology of the present invention enables multiple objects (such as an entire plate of food) to be located within a single field of view, if desired, and the processes of the present invention to be implemented to image and determine characteristics of those multiple objects. The present invention utilizes a single two-dimensional and three dimensional sensor or combination of sensors with one or more light sources to capture image data (or captured light voltage) for the one or more objects using a vast portion of the electromagnetic spectrum. The system of the present invention provides a unique combination of different sensory methods that are brought together through the use of sensor fusion technology. Specifically, the two-dimensional and three-dimensional sensor(s) are capable of capturing image data over electromagnetic wavelengths that cover the visible spectral region of about 400 nm to 700 nm and the non-visible near infrared spectral region of about 700 nm to 1050 nm. The identification of the one or more objects can be obtained combining image analysis of the image data in the visible spectral range and spectral analysis of the image data in the near infrared spectral region. The spectral analysis can be performed on the data captured by the two-dimensional and three-dimensional sensor(s) to derive spectral signatures for the one or more objects within a captured image and utilize the spectral signatures to identify the one or more objects within a captured image.

#Additionally, the two-dimensional and three-dimensional sensor(s) can capture three-dimensional data of the object and use the captured three-dimensional data to determine an estimated volume of the one or more objects. The combined spectral signatures and three-dimensional volumetric data can be captured and analyzed to automatically determine various characteristics of the target one or more objects. In an example embodiment, the combined visual image data, spectral signature data, and three-dimensional volumetric data of food items can be used to automatically determine actual food weights, and separate and isolate various foods on a plate, and determine a standard nutritional facts label for each food item of the captured meal image. Additionally, the three-dimensional data can be used to determine a difference in volumetric data at different periods of time. The difference in volumetric data can be applied for various useful purposes, including for example determining an amount of food items consumer by a user and the associated nutritional facts for the amount of consumed food items.

#When utilized in combination with the dietary system of the present invention, the combined two-dimensional visual image data, spectral signature image data, and three-dimensional volumetric data can be used to automatically track a user's nutritional intake and provide additional information for the user to assist in meeting nutritional goals. One implementation of automatic nutritional tracking requires the user capturing image data and three-dimensional volumetric data before eating a meal and again after eating the meal. The system of the present invention can use the image data and three-dimensional volumetric data to automatically identify the food items within the meal, separate the individual types of food items, determine an initial food volume for each type and a final food volume for each type, and calculate the nutritional values of the consumed portion of the meal (e.g., the nutritional values of the difference of the initial food volume and the final food volume). This process provides the user with all the nutritional information for the food consumed while only requiring the user to capture before and after image data for a meal without requiring the use of a fiduciary item or manually input data from the user. The use of the two-dimensional and three-dimensional sensor(s) and unconventional steps involved in object assessment system/dietary tracking system provides a unique improvement to the object recognition technologies. In particular, the unconventional steps of using data captured by image sensors to identify individual items within a group of items, determining volumetric data for each of the individually identified items, and determining additional characteristic information about the identified items based on the identification and the volume is an improvement to traditional object recognition technologies. As would be appreciated by one skilled in the art, the examples of tracking nutritional information for identified food items is intended to be for exemplary purposes only and the present invention can be utilized to identify and calculate volumetric information and other characteristic information for various types of objects for a particular purpose.