# US 11,443,337 B2 · Method, medium, and system for planning image sensor deployment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,337 B2 - Document ID: US11443337B2 - Status: Granted - Kind: B2 - Application number: 17/372,162 - Filed: 2021-07-09 - Published: 2021-12-02 - Granted: 2022-09-13 - Earliest claimed: 2020-06-01 - Assignees: Trax Technology Solutions Pte Ltd. - Applicants: Trax Technology Solutions Pte Ltd. - Inventors: Youval Bronicki; David Dedi Arev; Odi Paneth; Ido Merkado; May Masha Mordechay - Web page: https://patentgrep.com/patent/US11443337B2 ## CPC - G06Q 10/06395: {Quality analysis or management} - G01C 21/206: {specially adapted for indoor navigation} - G01G 19/4144: {for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems} - G01G 19/52: Weighing apparatus combined with other objects, e.g. furniture (with walking sticks A45B3/08) - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - G05D 1/0282: {generated in a local control room} - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - G05D 1/648: Performing a task within a working area or space, e.g. cleaning - G06F 18/2431: Multiple classes - G06F 18/253: {of extracted features} - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 5/04: Inference or reasoning models - G06N 20/00: Machine learning - G06Q 10/06311: {Scheduling, planning or task assignment for a person or group} - G06Q 10/06315: {Needs-based resource requirements planning or analysis} - G06Q 10/06375: {Prediction of business process outcome or impact based on a proposed change} - G06Q 10/0639: Performance analysis of employees; Performance analysis of enterprise or organisation operations - G06Q 10/06398: {Performance of employee with respect to a job function} - G06Q 30/018: Certifying business or products - G06Q 30/0201: Market modelling; Market analysis; Collecting market data - G06Q 30/0208: Trade or exchange of goods or services in exchange for incentives or rewards - G06Q 30/0281: {Customer communication at a business location, e.g. providing product or service information, consulting} - G06Q 30/0601: Electronic shopping [e-shopping] - G06Q 30/0609: {Qualifying participants for shopping transactions (payment transaction verification G06Q20/401)} - G06Q 30/0623: {by investigating goods or services} - G06Q 30/0639: {Locating goods or services, e.g. based on physical position of the goods or services within a shopping facility} - G06Q 30/0643: {graphically representing goods, e.g. 3D product representation} - G06Q 90/20: {Destination assistance within a business structure or complex} - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06T 7/50: Depth or shape recovery - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/73: using feature-based methods - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G06V 10/82: using neural networks - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/20: in augmented reality scenes - G06V 20/36: {Indoor scenes} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - H04L 67/303: Terminal profiles - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/181: {for receiving images from a plurality of remote sources} - H04W 4/024: Guidance services - H04W 4/029: Location-based management or tracking services - H04W 4/35: for the management of goods or merchandise - A47L 11/4011: {Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor} - A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection - G01G 19/14: for weighing suspended loads (G01G3/00 takes precedence; incorporation of weighing devices in cranes B66C1/40, B66C13/16) - G01G 23/18: Indicating devices, e.g. for remote indication; Recording devices; Scales, e.g. graduated - G01J 1/4204: {with determination of ambient light (solar light G01J2001/4266)} - G01L 1/00: Measuring force or stress, in general (measuring force due to impact G01L5/00) - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders - G06Q 10/08724: {by shelf level inventory management, e.g. planograms} - G06Q 30/0633: {Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087)} - G06T 2207/10016: Video; Image sequence - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/10024: Color image - G06T 2207/30242: Counting objects in image - G06V 2201/09: Recognition of logos - H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G01G: WEIGHING (sorting by weighing B07C5/16) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08) - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) - G01L: MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE (weighing G01G) ## Abstract A method for planning deployment of image sensors may include determining a location of a store shelf within a retail store and obtaining a first coverage parameter corresponding to a first product type and a second coverage parameter corresponding to a second product type. The method may also include accessing a database to determine a first height of products of the first product type and a second height of products of the second product type, and determining a position for placing a camera configured to capture images of at least a portion of the store shelf by analyzing the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height. The method may further include providing, to a user interface of a user device, information relating to the determined position of the camera. ## Claims ### Claim 1 A non-transitory computer-readable medium including instructions that when executed by a processor cause the processor to perform a method for planning deployment of image sensors, the method comprising: determining a location of a store shelf within a retail store; obtaining a first coverage parameter corresponding to a first product type and a second coverage parameter corresponding to a second product type; accessing a database to determine a first height of products of the first product type and a second height of products of the second product type; determining, by analyzing the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height, a position for placing a camera that enables image recognition of products represented in an image captured by the camera, the determined position being expressed relative to a coordinate system or an object in the retail store; and providing, to a user interface of a user device, information relating to the determined position for placing the camera, the provided information including an indication of a particular store shelf to mount the camera to. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein determining the location of the store shelf comprises: obtaining a store plan; and determining a location of store shelf based on the store plan. ### Claim 3 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein the store plan includes a three-dimensional (3D) model of the retail store. ### Claim 4 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein the store plan is determined based on an analysis of images captured from the retail store to identify store shelves and products placed on the store shelves. ### Claim 5 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the first coverage parameter corresponding to the first product type comprises at least one of a depth within a shelf on which a plurality of products of the first product type are planned to be placed, a percent of the plurality of products of the first product type that have been to be captured in an image to enable recognition, a capturing resolution, or a capturing frequency. ### Claim 6 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises determining the first product type and the second product type based on a planogram. ### Claim 7 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises determining an orientation of the camera. ### Claim 8 (depends on 7) The non-transitory computer-readable medium of claim 7, wherein the method further comprises: receiving updated information relating to a product type associated with products placed on the store shelf; determining an updated orientation of the camera based on the updated information; and providing, to the user interface of the user device, the updated orientation of the camera. ### Claim 9 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises determining a capturing frequency for a usage of the camera. ### Claim 10 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises generating a simulated image corresponding to a field of view of the camera. ### Claim 11 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the information relating to the determined position for placing the camera is provided through an augmented reality (AR) system. ### Claim 12 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises: receiving updated information relating to a product type associated with products placed on the store shelf; determining an updated position for placing the camera based on the updated information; and providing, to the user interface of the user device, the updated position for placing the camera. ### Claim 13 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises: receiving updated information relating to the store shelf; determining an update to at least one of a position for placing the camera and an orientation of the camera based on the updated information; and providing, to the user interface of the user device, information relating to the update to at least one of the position for placing the camera and the orientation of the camera. ### Claim 14 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein: the store shelf comprises a first store shelf; and the method further comprises: determining a location of a second store shelf within a retail store; and determining the position for placing the camera by analyzing the location of the store shelf, the location of the second store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height. ### Claim 15 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises: determining at least one dimension of the store shelf within a retail store; and determining the position for placing the camera analyzing the location of the store shelf, the at least one dimension of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height. ### Claim 16 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises: receiving at least one property of the camera; and determining the position for placing the camera by analyzing the at least one property of the camera, the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height. ### Claim 17 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the method further comprises: selecting a camera among a plurality of cameras as the camera based on properties of each one of the plurality of cameras, the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height; and providing an indication of the selected camera to the user. ### Claim 18 A system for planning deployment of image sensors, the system comprising: at least one processor programmed to: determine a location of a store shelf within a retail store; obtain a first coverage parameter corresponding to a first product type and a second coverage parameter corresponding to a second product type; access a database to determine a first height of products of the first product type and a second height of products of the second product type; determine, by analyzing the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height, a position for placing a camera that enables image recognition of products represented in an image captured by the camera, the determined position being expressed relative to a coordinate system or an object in the retail store; and provide, to a user interface of a user device, information relating to the determined position for placing the camera, the provided information including an indication of a particular store shelf to mount the camera to. ### Claim 19 A method for planning deployment of image sensors, the method comprising: determining a location of a store shelf within a retail store; obtaining a first coverage parameter corresponding to a first product type and a second coverage parameter corresponding to a second product type; accessing a database to determine a first height of products of the first product type and a second height of products of the second product type; determining, by analyzing the location of the store shelf, the first coverage parameter, the second coverage parameter, the first height, and the second height, a position for placing a camera that enables image recognition of products represented in an image captured by the camera, the determined position being expressed relative to a coordinate system or an object in the retail store; and providing, to a user interface of a user device, information relating to the determined position for placing the camera, the provided information including an indication of a particular store shelf to mount the camera to.