# US 11,307,411 B2 · Support device and system for article picking or sorting work > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,411 B2 - Document ID: US11307411B2 - Status: Granted - Kind: B2 - Application number: 14/411,664 - Filed: 2013-05-31 - Published: 2015-07-09 - Granted: 2022-04-19 - Earliest claimed: 2012-06-29 - Assignees: Toyo Kanetsu Solutions K.K.; M. Soft Co., Ltd. - Applicants: Toyo Kanetsu Solutions K.K.; M. Soft Co., Ltd. - Inventors: Kazuhito Watanabe; Tadao Ito; Juji Yokoya; Norio Yagishita; Norihiro Wada; Yuka Tsushima; Michio Yokoyama; Hideaki Yonaga; Yasuhide Matsumoto; Rokuro Iwase - Web page: https://patentgrep.com/patent/US11307411B2 ## CPC - G02B 27/017: {Head mounted} - B65G 1/1375: {the orders being assembled on a commissioning stacker-crane or truck} - G06Q 10/08: Logistics, e.g. warehousing, loading or distribution; Inventory or stock management - G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders - G06Q 10/08743: {using optimisation of routes} - B65G 2209/04: Indication location means - B65G 2209/06: Use of order trucks - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Some embodiments are directed to a device that supports work of picking or sorting an article from a warehouse and the like, which allows the picking work to be carried out accurately and quickly. The device includes: a camera that photographs an automatically-readable identifier that is attached to a storage location of an article; an information processing unit that detects the identifier from an image photographed by the camera and processes the image of the identifier; and a display that displays information such as image information and text information. The information processing unit processes, based on the detected identifier and picking work order information, image on a specific identifier, and the processed image is displayed on the display at a location related to the identifier. ## Claims ### Claim 1 An article picking or sorting work support device, comprising: A) an imaging unit that obtains image information including automatically-readable, optical feature information associated with an article or a storage location of an article or a location where an article is to be sorted; B) an information processing unit that detects said optical feature information from the image information obtained by said imaging unit and processes image information about said optical feature information; and C) a display that displays information such as said image information and text information, wherein said information processing unit processes, based on said detected, optical feature information and predetermined, picking or sorting work order information, image information about specific, optical feature information and displays said processed image information from an aisle to a rack or further to housing frontage or image information of the aisle to the rack or further to the housing frontage themselves, on said display so that the displayed said processed image information or said image information of the aisle to the rack or further to the housing frontage themselves are emphasized superimposed on said image information obtained by said imaging unit or superimposed on actual visual scenery. ### Claim 2 (depends on 1) The article picking or sorting work support device according to claim 1, further comprising: D) a guidance unit that guides a worker to a storage location of an article, wherein after said guidance unit guides the worker to said storage location of an article, said information processing unit processes, based on said detected, optical feature information and the predetermined picking or sorting work order information, image information about said specific, optical feature information and said processed image information is displayed on said display at a location related to said specific, optical feature information. ### Claim 3 (depends on 2) The article picking or sorting work support device according to claim 2, wherein said guidance unit uses any one method of a method that uses automatically-readable, optical feature information similar to said automatically-readable, optical feature information associated with said article or said storage location of an article; a guidance method that uses radio waves; inertial navigation that uses a combination of an accelerometer and a gyro; and a method that recognizes, by using an external camera, an optical feature attached on a worker or an instrument moving with the worker; or any combination of two or more methods thereof. ### Claim 4 (depends on 1) The article picking or sorting work support device according to claim 1, further comprising: E) a memory unit that stores picking or sorting work order information associated with optical feature information, further wherein said information processing unit processes, based on said detected, optical feature information and the picking or sorting work order information in said memory unit, image information about said specific, optical feature information, and said processed image information is displayed on said display at a location related to said specific, optical feature information. ### Claim 5 (depends on 1) The article picking or sorting work support device according to claim 1, further comprising: F) an article quality inspection unit that inspects quality of an article to be picked or sorted, further wherein if said article is not suitable for being picked or sorted, a predetermined, processing method is issued. ### Claim 6 (depends on 1) The article picking or sorting work support device according to claim 1, wherein said information processing unit processes image information about said specific, optical feature information, said processed image information is displayed on said display at a location related to said specific, optical feature information, and also picking or sorting work support information prepared by said information processing unit is displayed on said display. ### Claim 7 (depends on 1) The article picking or sorting work support device according to claim 1, wherein said display is a head-mounted display that can display both of actual, visual scenery information and prepared information, and image information about said specific, optical feature information processed by said information processing unit is displayed on said display with the image information being associated with actual, visual scenery information at a location related to said specific, optical feature information. ### Claim 8 (depends on 1) The article picking or sorting work support device according to claim 1, wherein said display is a transportable display, and image information about said processed, optical feature information is displayed on said display with the image information being associated with image information photographed by said imaging unit at a location related to said specific, optical feature information. ### Claim 9 (depends on 1) The article picking or sorting work support device according to claim 1, wherein said automatically-readable, optical feature information associated with an article or a storage location of an article is an identifier. ### Claim 10 (depends on 9) The article picking or sorting work support device according to claim 9, wherein base material of said identifier is printable base material. ### Claim 11 (depends on 9) The article picking or sorting work support device according to claim 9, wherein base material of said identifier is base material in which information is rewritable. ### Claim 12 (depends on 9) The article picking or sorting work support device according to claim 9, wherein a present position of a worker is grasped from image information on said identifier. ### Claim 13 (depends on 9) The article picking or sorting work support device according to claim 9, wherein said work order information or work support information is displayed superimposed on said identifier. ### Claim 14 (depends on 1) The article picking or sorting work support device according to claim 1, wherein for a picked article, or an article to be sorted, information that is attached on said article to identify said article is photographed by said imaging unit, and whether said article agrees with the article specified by said picking or sorting work order or not is recognized. ### Claim 15 (depends on 1) The article picking or sorting work support device according to claim 1, wherein in a field of the view of said imaging unit, information about work and displaying is input by means of a gesture using hands and fingers of a worker. ### Claim 16 (depends on 1) The article picking or sorting work support device according to claim 1, further comprising: an audio response device including an audio input unit and/or an audio output unit. ### Claim 17 (depends on 1) The article picking or sorting work support device according to claim 1, wherein said information processing unit is provided remotely. ### Claim 18 (depends on 1) The article picking or sorting work support device according to claim 1, wherein a work history of a worker can be collected. ### Claim 19 (depends on 1) The article picking or sorting work support device according to claim 1, wherein a work indicator is displayed on said display. ### Claim 20 (depends on 1) The article picking or sorting work support device according to claim 1, wherein if said articles to be picked or sorted, are insufficient, said information processing unit issues an order for dealing with the trouble to a worker, according to a predetermined rule. ### Claim 21 (depends on 1) The article picking or sorting work support device according to claim 1, wherein based on the situation that said optical feature cannot be detected, a failure of said optical feature itself or a structural object where said optical feature is provide is detected. ### Claim 22 (depends on 1) The article picking or sorting work support device according to claim 1, further comprising: a management terminal having a display and a work ordering unit. ### Claim 23 An article picking or sorting work support system, comprising: A) automatically-readable, optical feature information associated with an article or a storage location of an article or a location where an article is to be sorted, and B) a picking or sorting work support device including: an imaging unit that obtains image information about an article or a storage location of an article; an information processing unit that detects said optical feature information from the image information obtained by said imaging unit and processes image information about said optical feature information; and a display that displays information such as image information and text information, wherein said information processing unit of said picking or sorting work support device processes, based on said detected, optical feature information and predetermined, picking or sorting work order information, image information about specific, optical feature information and displays said processed image information from an aisle to a rack or further to housing frontage or image information of the aisle to the rack or further to the housing frontage themselves, on said display so that the displayed said processed image information or said image information of the aisle to the rack or further to the housing frontage themselves are emphasized superimposed on said image information obtained by said imaging unit or superimposed on actual visual scenery. ### Claim 24 (depends on 23) The article picking or sorting work support system according to claim 23, further comprising: a guidance unit that guides a worker to a storage location of an article, wherein after the guidance unit of said picking work support device guides the worker to the storage location of the article, said information processing unit processes, based on said detected, optical feature information and the predetermined picking or sorting work order information, image information about said specific, optical feature information and said processed image information is displayed on said display at a location related to said specific, optical feature information. ### Claim 25 (depends on 24) The article picking or sorting work support system according to claim 24, wherein said guidance unit uses any one method of a method that uses automatically-readable, optical feature information similar to said automatically-readable, optical feature information associated with said article or said storage location of the article; a guidance method that uses radio waves; inertial navigation that uses a combination of an accelerometer and a gyro; and a method that recognizes, by using an external camera, an optical feature attached on a worker or an instrument moving with the worker; or any combination of two or more methods thereof. ### Claim 26 (depends on 23) The article picking or sorting work support system according to claim 23, further comprising: a memory unit that stores picking or sorting work order information associated with said optical feature information, wherein said information processing unit processes, based on said detected, optical feature information and said picking or sorting work order information in said memory unit, image information about said specific, optical feature information and said processed image information is displayed on said display at a location related said specific, optical feature information. ### Claim 27 (depends on 23) The article picking or sorting work support system according to claim 23, further comprising: an article quality inspection unit that inspects quality of an article to be picked or sorted, wherein if said article is not suitable for being picked or sorted, a predetermined processing method is issued. ### Claim 28 (depends on 23) The article picking or sorting work support system according to claim 23, wherein said information processing unit processes image information about said specific, optical feature information, said processed image information is displayed on said display at a location related to said specific, optical feature information, and also picking or sorting work support information prepared by said information processing unit is displayed on said display. ### Claim 29 (depends on 23) The article picking or sorting work support system according to claim 23, wherein said display is a head-mounted display that can display both of actual, visual scenery information and prepared information, and image information about said specific, optical feature information processed by said information processing unit is displayed on said display with the image information being associated with actual, visual scenery information at a location related to said specific, optical feature information. ### Claim 30 (depends on 23) The article picking or sorting work support system according to claim 23, wherein said display is a transportable display, and image information about said processed, optical feature information is displayed on said display with the image information being associated with image information photographed by said imaging unit at a location related to said specific, optical feature information. ### Claim 31 (depends on 23) The article picking or sorting work support system according to claim 23, wherein said automatically-readable, optical feature information associated with an article or a storage location of an article is an identifier. ### Claim 32 (depends on 31) The article picking or sorting work support system according to claim 31, wherein base material of said identifier is printable base material. ### Claim 33 (depends on 31) The article picking or sorting work support system according to claim 31, wherein base material of said identifier is base material in which information is rewritable. ### Claim 34 (depends on 31) The article picking or sorting work support system according to claim 31, wherein a present position of a worker is grasped from image information on said identifier. ### Claim 35 (depends on 31) The article picking or sorting work support system according to claim 31, wherein said work order information or work support information is displayed superimposed on said identifier. ### Claim 36 (depends on 23) The article picking or sorting work support system according to claim 23, wherein for a picked article, or an article to be sorted, information attached on said article to identify said article is photographed by said imaging unit, and whether said article agrees with the article specified by said picking or sorting work order or not is recognized. ### Claim 37 (depends on 23) The article picking or sorting work support system according to claim 23, wherein in a field of the view of said imaging unit, information about work and displaying is input by means of a gesture using hands and fingers of a worker. ### Claim 38 (depends on 23) The article picking or sorting work support system according to claim 23, further comprising: an audio response device including an audio input unit and/or an audio output unit. ### Claim 39 (depends on 23) The article picking or sorting work support system according to claim 23, wherein said information processing unit is provided remotely. ### Claim 40 (depends on 23) The article picking or sorting work support system according to claim 23, wherein a work history of a worker can be collected. ### Claim 41 (depends on 23) The article picking or sorting work support system according to claim 23, wherein a work indicator is displayed on said display. ### Claim 42 (depends on 23) The article picking or sorting work support system according to claim 23, wherein if said articles to be picked or sorted, are insufficient, said information processing unit issues an order for dealing with the trouble to a worker, according to a predetermined rule. ### Claim 43 (depends on 23) The article picking or sorting work support system according to claim 23, wherein based on the situation that said optical feature cannot be detected, a failure of said optical feature itself or a structural object where said optical feature is provide is detected. ### Claim 44 (depends on 23) The article picking or sorting work support system according to claim 23, further comprising: a management terminal having a display and a work ordering unit.