# US 12,223,460 B2 · Package detection and intelligent sorting > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,460 B2 - Document ID: US12223460B2 - Status: Granted - Kind: B2 - Application number: 17/599,375 - Filed: 2020-03-16 - Published: 2022-06-09 - Granted: 2025-02-11 - Earliest claimed: 2019-04-10 - Assignees: Laitram, L.L.C. - Applicants: Laitram, L.L.C. - Inventors: Alexander I. Dragoset; Jason Lagneaux - Web page: https://patentgrep.com/patent/US12223460B2 ## CPC - B07C 3/14: using light-responsive detecting means {(for information carriers G06K7/14)} - G06Q 10/08: Logistics, e.g. warehousing, loading or distribution; Inventory or stock management - B07C 1/04: Forming a stream from a bulk; Controlling the stream, e.g. spacing the articles - B07C 3/08: using arrangements of conveyors {(transport of sheets B65H; transport in general B65G)} - B65G 47/14: arranging or orientating the articles by mechanical or pneumatic means during feeding (during transit by conveyor B65G47/24, B65G47/26) - B65G 47/69: the articles being accumulated temporarily {(accumulating articles during transit B65G47/261)} - B65G 2203/041: Camera - G06F 2218/12: Classification; Matching - B07C: POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING (specially adapted for a specific purpose covered by another class, see the relevant place, e.g. A43D33/06, B23Q7/12) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A sorting conveyor and a method for sorting packages to selected destinations. A neural network executed by a multi-core classification processor uses captured images of package units conveyed on an infeed conveyor to classify each package unit. A control processor controls a sorting conveyor to direct each package unit to a destination depending on the package unit's classification. ## Claims ### Claim 1 A conveyor system comprising: an infeed conveyor segment conveying separated package units downstream in a conveying direction; a package-unit detector detecting the package units at a detection position along the infeed conveyor segment, wherein the package-unit detector detects the footprints of the package units; an imaging system capturing images of the package units within a target zone of the infeed conveyor segment; a computer processing system executing program instructions to track the position of each of the package units as they are conveyed on the infeed conveyor segment and to provide each of the images of the package units as inputs to a classifier trained to recognize a set of package units and to assign to each of the package units a classification corresponding to one of the members of the set of package units the classifier is trained to recognize; a downstream conveyor segment receiving the package units from the infeed conveyor segment; wherein the computer processing system executes program instructions to control the downstream conveyor segment to convey each package unit depending on the package unit's classification. ### Claim 2 (depends on 1) A conveyor system as claimed in claim 1 wherein the computer processing system executes program instructions to: calculate the positions of each of the package units on the infeed conveyor segment as they advance along the conveyor from their footprints detected passing the detection position; control the imaging system to capture an image of one or more of the package units when the calculated position of one or more of the package units lies within the target zone on the infeed conveyor segment to produce an image of the one or more of the package units within the target zone; crop the image into one or more cropped images, each cropped image corresponding to one of the one or more package units within the target zone; classify the cropped images into one of the classifications. ### Claim 3 (depends on 2) A conveyor system as claimed in claim 2 wherein the computer processing system computes coordinates of the centroids of the detected footprints wherein the centroids define the positions of the package units on the infeed conveyor segment. ### Claim 4 (depends on 2) A conveyor system as claimed in claim 2 wherein the computer processing system rotates the images to a non-oblique orientation relative to the conveying direction before cropping the images. ### Claim 5 (depends on 2) A conveyor system as claimed in claim 2 wherein the computer processing system includes a multi-core processor having multiple cores executing program instructions in parallel. ### Claim 6 (depends on 1) A conveyor system as claimed in claim 1 wherein the computer processing system includes a first processor and a second processor that controls the downstream conveyor segment to sort each of the package units to one of a plurality of destinations depending on the package unit's classification. ### Claim 7 (depends on 6) A conveyor system as claimed in claim 6 wherein the set of package units the classifier is trained to recognize are: (a) a polybag; (b) a recognized single package not a polybag; (c) a stack of packages with less than a predetermined percentage of overlap; and (d) a stack of packages with more than the predetermined percentage of overlap. ### Claim 8 (depends on 7) A conveyor system as claimed in claim 7 wherein the second processor controls the sorter to sort package units classified as a stack of packages with more than the predetermined percentage of overlap or as an unrecognized package unit to a destination manned by a human operator. ### Claim 9 (depends on 7) A conveyor system as claimed in claim 7 wherein the second processor controls the sorter to sort package units classified as a recognized single package or as a stack of packages with less than the predetermined percentage of overlap to a downstream destination for further processing. ### Claim 10 A processor-implemented method for sorting packages conveyed on an infeed conveyor segment, the method comprising: (a) detecting the footprints of multiple package units advancing at a conveying speed in a conveying direction on an infeed conveyor segment past a detection position; (b) calculating the positions of each of the multiple package units detected passing the detection position on the infeed conveyor segment as they advance along the conveyor; (c) imaging one or more of the multiple package units in an image capture area on the infeed conveyor segment to produce an image of the one or more of the multiple package units within the capture area; (d) cropping the image into one or more cropped images, each cropped image corresponding to one of the one or more multiple package units within the image capture area; (e) classifying the cropped images into a plurality of classifications using artificial intelligence by assigning one of the classifications to each of the one or more multiple package units; (f) sorting each of the one or more multiple package units to a destination depending on the package unit's classification. ### Claim 11 (depends on 10) The method of claim 10 comprising executing steps (b)-(e) in parallel in a multi-core processor. ### Claim 12 (depends on 11) The method of claim 11 comprising steps (a) and (f) in a separate control processor in communication with the multi-core processor. ### Claim 13 (depends on 12) The method of claim 12 wherein the multi-core processor sends the classification assigned to each of the one or more multiple package units to the control processor. ### Claim 14 (depends on 10) The method of claim 10 wherein the package units are classified as: (a) a polybag; (b) a recognized single package not a polybag; (c) a stack of packages with less than a predetermined percentage of overlap; (d) a stack of packages with more than the predetermined percentage of overlap; and (e) an unrecognized package unit. ### Claim 15 (depends on 14) The method of claim 14 comprising sorting package units classified as a stack of packages with more than the predetermined percentage of overlap or as an unrecognized package unit to a destination manned by a human operator. ### Claim 16 (depends on 14) The method of claim 14 comprising sorting package units having a footprint greater than a predetermined footprint size to a destination manned by a human operator. ### Claim 17 (depends on 14) The method of claim 14 comprising sorting package units classified as a recognized single package or as a stack of packages with less than the predetermined percentage of overlap onto a conveyor conveying the package unit to further processing. ### Claim 18 (depends on 10) The method of claim 10 wherein the classifying is performed by a neural network trained to recognize package units of various kinds. ### Claim 19 (depends on 10) The method of claim 10 comprising computing coordinates of the corners of the detected footprints. ### Claim 20 (depends on 10) The method of claim 10 comprising computing coordinates of the centroids of the detected footprints wherein the centroids define the positions of the package units on the infeed conveyor segment. ### Claim 21 (depends on 10) The method of claim 10 comprising rotating the images to a non-oblique orientation relative to the conveying direction before cropping the images. ### Claim 22 (depends on 10) The method of claim 10 comprising preprocessing the cropped images in parallel before classifying the cropped images to format the cropped images for classifying. ### Claim 23 A conveyor system comprising: an infeed conveyor segment conveying package units downstream in a conveying direction at a conveying speed; a footprint detector disposed at a detection position along the infeed conveyor segment; a camera disposed along the infeed conveyor segment downstream of the detection position to capture an image of a capture area on the infeed conveyor segment; a sorting conveyor segment receiving package units from the infeed conveyor segment and selectively sorting the package units to multiple destinations; a control processor executing program instructions to: operate the footprint detector to detect the footprints of the package units advancing past the detection position in the conveying direction; control the sorting conveyor segment and the infeed conveyor segment; a classification processor in communication with the control processor and executing program instructions to: calculate the positions of each of the package units on the infeed conveyor segment as they advance along the conveyor from their footprints detected passing the detection position; control the camera to capture an image of one or more of the package units when the calculated position of one or more of the package units lies within a target zone in the capture area on the infeed conveyor segment to produce an image of the one or more of the package units within the capture area; crop the image into one or more cropped images, each cropped image corresponding to one of the one or more package units within the capture area; use artificial intelligence to classify the cropped images into a plurality of classifications by assigning one of the classifications to each of the one or more package units; wherein the control processor controls the sorting conveyor segment to sort each of the one or more package units to a destination depending on the package unit's classification. ### Claim 24 (depends on 23) A conveyor system as claimed in claim 23 wherein the classification processor classifies the cropped images in a neural network trained to recognize various package units and sends the classifications to the control processor. ### Claim 25 (depends on 23) A conveyor system as claimed in claim 23 wherein the control processor includes a virtual or separate programmable logic controller to control the sorting conveyor segment and the infeed conveyor segment. ### Claim 26 (depends on 23) A conveyor system as claimed in claim 23 wherein the classification processor is a multi-core processor having multiple cores executing the program instructions in parallel.