A refrigerator includes a main body, a door, at least one camera mounted to the main body and/or the door and at least one computing device. The camera(s) is/are positioned to have a field of view that includes an entrance opening leading to a storage compartment and an interior side of the door when the door is open. The camera(s) is/are configured to capture images of an item being loaded into or being removed from the storage compartment. The computing device(s) can instruct the camera(s) to capture the images, analyze the captured images and determine whether the images contain an object of interest. The computing device(s) can assign a sensed door angle and an object location to images containing the object of interest. The computing device(s) can assign a direction of movement for the object of interest based on the sensed door angle and changes in the object location.
Drawings
Claims13
A refrigerator comprising:
a main body and a door pivotally mounted to and movable with respect to the main body between a plurality of open positions and a closed position, the main body and the door defining at least one storage compartment within a refrigerated enclosure when the door is in the closed position;
at least one camera mounted to at least one of the main body and the door, the at least one camera positioned to have a field of view that includes an entrance opening leading to the at least one storage compartment and an interior side of the door when the door is in at least one of the plurality of open positions, the at least one camera being configured to capture images of an item being loaded into or being removed from the at least one storage compartment;
and
at least one computing device in operable connection with the at least one camera, the at least one computing device configured to:
instruct the at least one camera to capture the images based on a trigger signal received by the at least one computing device;
analyze the captured images from the at least one camera and determine whether the images contain an object of interest;
assign a sensed door angle and an object location to each of n images among the analyzed images containing the object of interest, wherein the sensed door angle assigned with a respective image among the n images corresponds to an angle of the door with respect to the main body when the respective image was captured and the object location corresponds to a coordinate position for the object of interest in the respective image;
and
assign a direction of movement for the object of interest being at least one of into, out of or internally within the refrigerated enclosure, wherein the direction of movement is assigned based at least in part on changes in the sensed door angle among the n images and changes in the object location among the n images.
6 more elements
A non-transitory computer readable storage medium storing instructions that, when executed by a computer having a processor, cause the processor to perform a method comprising:
instructing at least one camera to capture images based on a trigger signal received by the processor;
analyzing the images captured by the at least one camera and to determine whether the images contain an object of interest;
assigning a sensed door angle and an object location to each of n images among the images determined to contain the object of interest, wherein the sensed door angle assigned with a respective image among the n images corresponds to an angle of a door with respect to a main body of a refrigerator when the respective image was captured and the object location corresponds to a point overlapped by the object of interest in the respective image;
and
assigning a direction of movement for the object of interest being at least one of into, out of or internally within a refrigerated enclosure of the refrigerator, wherein the direction of movement is assigned based at least in part on changes in the sensed door angle among the n images and changes in the object location among the n images.
18 Jun 2044Estimated · howhide20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment, including 61 days of adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.
Scenes; Scene-specific elements: context or environment of the image
Life of this patent
About 17 years 8 months of term left · ends 18 Jun 2044 (estimated)
Priority18 Apr 2024
Granted21 Apr 2026
Term ends18 Jun 2044
Priority18 Apr 2024
Granted21 Apr 2026
Term ends18 Jun 2044
Application filedUS 18/638,955
Assigned to Springhouse Technologies Inc.assignment of assignor's interest
PublishedUS 2025/0327615 A1
GrantedUS 12,607,397 B2
TodayWithin its term
Term endsEstimated, includes 61 days of adjustment
Description7,257 words
Summary664 words
#Known refrigerators can include a camera for imaging and recording inventory disposed in an interior of the refrigerator. Some known refrigerators include cameras for imaging and recording inventory where the cameras are located and designed to each have a field of view directed toward an interior of the refrigerator. When inventory items are placed in such refrigerators at locations that obscure view of other items from the camera, the camera has difficulty accurately recording the obscured inventory.
#US 2016/0088262 A1 discloses a refrigerator that includes an imaging unit, a data storage unit, a control unit and a display unit. The imaging unit generates a goods loading/unloading video through video recording of storage goods loaded into or unloaded from the refrigerator. The data storage unit stores the goods loading/unloading video, goods information that may be stored in the refrigerator, and storage items information. The control unit recognizes loading/unloading of goods based on an optical flow detected through vision recognition from the goods loading/unloading video, and updates the storage items information in the refrigerator based on recognition information generated through vision recognition on loading/unloading of the goods and the goods information stored in the data storage unit. The display unit displays a managed state of storage items for a user based on the updated storage items information.
#U.S. Pat. No. 11,692,769 B2 discloses a refrigerator appliance including a cabinet defining a chilled chamber, a door rotatably hinged to the cabinet to provide selective access to the chilled chamber, and a camera assembly mounted to the cabinet for monitoring the chilled chamber. A controller is operably coupled to the camera assembly and is configured to obtain a raw image using the camera assembly, analyze the raw image to identify an anchor object, crop the raw image to generate a reduced image surrounding the anchor object, and analyze the reduced image to identify a food item being added to or removed from the chilled chamber.
Brief description of the drawings6 figures
#FIG. 1 is a perspective view of a refrigerator with a door shown in an open position.
#FIG. 2 is a front view of the refrigerator with a door shown in an open position.
#FIG. 3 is a block diagram representing an operating environment of a refrigerator including an inventory tracking system.
Detailed description6,480 words
#Referring to FIG. 1, a refrigerator 102 includes a main body 104 and a door 106 pivotally mounted to and movable with respect to the main body 104. Similar to known refrigerators, the door 106 is movable between a plurality of open positions, one of which being shown in FIG. 1, and a closed position. The main body 104 and the door 106 define at least one storage compartment within a refrigerated enclosure 108 when the door 106 is in the closed position. As illustrated, the main body 104 and the door 106 define a plurality of storage compartments 112, 114, 116, 118, 120, 122, 124, 126 within the refrigerated enclosure 108 when the door 106 is in the closed position. The plurality of storage compartments includes a first storage compartment 112, a second storage compartment 114, a third storage compartment 116, a fourth storage compartment 118, a fifth storage compartment 120, which are each located within the main body 104 in the illustrated embodiment, and a sixth storage compartment 122, a seventh storage compartment 124, and an eighth storage compartment 126, which are located on an interior side 128 the door 106 in the illustrated embodiment. The number and location of the storage compartments 112, 114, 116, 118, 120, 122, 124, 126 may differ from that shown in FIGS. 1 and 2.
#As illustrated in FIG. 1, a first shelf 132 and a second shelf 134 are disposed in the first storage compartment 112. A third shelf 136 is disposed in the second storage compartment 114. A first drawer 142 is disposed in the third storage compartment 116. A second drawer 144 is disposed in the fourth storage compartment 118. A third drawer 146 disposed in the fifth storage compartment 120. A first bin 152 is disposed in the sixth storage compartment 122. A second bin 154 is disposed in the seventh storage compartment 124. A third bin 156 is disposed in the eighth storage compartment 126.
#The door 106 is connected to the main body 104 through an upper hinge 162 and a lower hinge 164 in the illustrated embodiment. The door 106 is movable with respect to the main body 104 among a plurality of open positions based on an angle of the door 106 with respect to the main body 104.