# US 11,897,634 B2 · Systems and methods for determining a state of a stowage bin assembly within an internal cabin of a vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,634 B2 - Document ID: US11897634B2 - Status: Granted - Kind: B2 - Application number: 17/347,888 - Filed: 2021-06-15 - Published: 2022-02-03 - Granted: 2024-02-13 - Earliest claimed: 2020-07-29 - Assignees: The Boeing Company - Applicants: The Boeing Company - Inventors: Venkatesh Babu Radhakrishnan; Mugaludi Ramesha Rakesh; Madhanmohan Savadamuthu; Shubham Tripathi; Ethan Carl Owyang - Web page: https://patentgrep.com/patent/US11897634B2 ## CPC - B64D 11/003: {Stowage devices for passengers' personal luggage} - B64F 5/60: Testing or inspecting aircraft components or systems - G06T 7/0008: {checking presence/absence} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions - G06T 2207/10004: Still image; Photographic image - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20132: Image cropping - G06T 2207/30112: Baggage; Luggage; Suitcase - G06T 2207/30172: Centreline of tubular or elongated structure - G06T 2207/30268: Vehicle interior - Y02T 50/40: Weight reduction - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - B64F: GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION ## Abstract A system includes one or more components within an internal cabin of a vehicle. An imaging device is configured to obtain an image of the one or more components. A state determination control unit includes a processor. The state determination control unit is in communication with the imaging device. The state determination control unit receives image data including the image from the imaging device. Further, the state determination control unit determines a state of the one or more components based on the image data. ## Claims ### Claim 1 A system comprising: components within an internal cabin of a vehicle, wherein the components are in a row; an imaging device configured to obtain an image of the components in the row; and a state determination control unit comprising a processor, wherein the state determination control unit is in communication with the imaging device, wherein the state determination control unit is configured to receive image data including the image of the components from the imaging device, and wherein the state determination control unit is configured to determine a state of the components based on the image data including the image of the components in the row, wherein the state determination control unit is further configured to: generate, within the image, a first end line at a first end of the row and a second end line at a second end of the row opposite from the first end line, and generate, within the image, an upper boundary line associated with an upper edge of the row, and a lower boundary line associated with a lower edge of the row, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that bounds the components. ### Claim 2 (depends on 1) The system of claim 1, wherein the state comprises one or both of an open state or a closed state. ### Claim 3 (depends on 1) The system of claim 1, wherein the components are overhead stowage bin assemblies. ### Claim 4 (depends on 1) The system of claim 1, wherein the state determination control unit is further configured to determine a region of interest within the image data. ### Claim 5 (depends on 4) The system of claim 4, wherein the state determination control unit is further configured to determine the region of interest by determining a vanishing point within the image. ### Claim 6 (depends on 5) The system of claim 5, wherein the state determination control unit is further configured to determine a bisecting line that passes through the vanishing point, wherein the region of interest is determined by the bisecting line. ### Claim 7 (depends on 1) The system of claim 1, wherein the state determination control unit is further configured to correct for perspective within the image data. ### Claim 8 (depends on 1) The system of claim 1, wherein the state determination control unit is further configured to geometrically transform the quadrilateral into a rectangle having respective corresponding corners to form a perspective corrected image. ### Claim 9 (depends on 1) The system of claim 1, wherein the state determination control unit is further configured extrapolate a shape of each of the components within the image data. ### Claim 10 (depends on 1) The system of claim 1, wherein the state determination control unit is configured to determine the state of the components through clustering. ### Claim 11 (depends on 1) The system of claim 1, wherein the state determination control unit is configured to determine the state of the components by determining a difference in image attributes. ### Claim 12 (depends on 1) The system of claim 1, further comprising a user device, wherein the state determination control unit is further configured to output a state signal indicative of the state of the components to the user device. ### Claim 13 (depends on 1) The system of claim 1, wherein the state determination control unit is configured to output a component control signal that automatically operates the components based on state of the components. ### Claim 14 A method comprising: obtaining, by an imaging device, an image of components in a row within a vehicle; receiving, by a state determination control unit comprising a processor that is in communication with the imaging device, image data including the image of the components in the row from the imaging device; and determining, by the state determination control unit, a state of the components based on the image data including the image of the components in the row, wherein said determining comprises generating, within the image, a first end line at a first end of the row and a second end line at a second end of the row opposite from the first end line, and generating, within the image, an upper boundary line associated with an upper edge of the row, and a lower boundary line associated with a lower edge of the row, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that bounds the plurality of components, and wherein the state comprises one or both of an open state or a closed state, and wherein the one or more components are one or more overhead stowage bin assemblies. ### Claim 15 (depends on 14) The method of claim 14, wherein said determining further comprises: determining a region of interest within the image data; determining the region of interest by determining a vanishing point within the image; and determining a bisecting line that passes through the vanishing point, and determining the region of interest by the bisecting line. ### Claim 16 (depends on 14) The method of claim 14, further comprising correcting for perspective, by the state determination control unit, within the image data. ### Claim 17 (depends on 14) The method of claim 14, wherein said determining further comprises geometrically transforming the quadrilateral into a rectangle having respective corresponding corners to form a perspective corrected image. ### Claim 18 (depends on 14) The method of claim 14, wherein said determining further comprises: extrapolating a shape of each of the components within the image data; determining the state of the components through clustering; and determining a difference in image attributes. ### Claim 19 (depends on 14) The method of claim 14, further comprising: outputting, by the state determination control unit, a state signal indicative of the state of the components to a user device; and outputting, by the state determination control unit, a component control signal that automatically operates the components based on the state of the components. ### Claim 20 A non-transitory computer-readable storage medium comprising executable instructions that, in response to execution, cause a system comprising a processor, to perform operations comprising: receiving image data comprising an image of components within a row within an internal cabin of a vehicle from an imaging device within the internal cabin; determining a state of the one or more components based on the image data including the image of the components in the row, wherein said determining comprises generating, within the image, a first end line at a first end of the row and a second end line at a second end of the row opposite from the first end line, and generating, within the image, an upper boundary line associated with an upper edge of the row, and a lower boundary line associated with a lower edge of the row, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that bounds the plurality of components; and outputting a component control signal that automatically operates the components based on the state of the components. ### Claim 21 (depends on 19) The non-transitory computer-readable storage medium of claim 19, further comprising geometrically transforming the quadrilateral into a rectangle having respective corresponding corners to form a perspective corrected image.