# US 11,900,722 B2 · Electronic apparatus and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,900,722 B2 - Document ID: US11900722B2 - Status: Granted - Kind: B2 - Application number: 17/087,005 - Filed: 2020-11-02 - Published: 2021-05-20 - Granted: 2024-02-13 - Earliest claimed: 2019-11-19 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Heungwoo Han; Seongmin Kang - Web page: https://patentgrep.com/patent/US11900722B2 ## CPC - G06V 40/161: {Detection; Localisation; Normalisation} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/82: using neural networks - G06V 40/168: {Feature extraction; Face representation} - G06V 40/176: {Dynamic expression} - G06V 40/23: {Recognition of whole body movements, e.g. for sport training} - H04N 23/80: Camera processing pipelines; Components thereof - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/047: Probabilistic or stochastic networks - G06N 3/08: Learning methods - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An electronic apparatus may include a processor configured to: obtain an image captured by a camera; obtain a downscaled image by downscaling the captured image, wherein the downscaled image has an image that is less than a critical resolution, identify a region of interest included in the downscaled image by inputting the downscaled image into a first artificial intelligence model, the first artificial intelligence model being trained to identify a region of interest in an image; extract, from the captured image, an object image in the captured image corresponding to the identified region of interest, and obtain information on an object region included in the captured image by inputting the extracted object image into a second artificial intelligence model, the second artificial intelligence model being configured to identify an object region in an input image. ## Claims ### Claim 1 An electronic apparatus comprising: a processor configured to: obtain an image captured by a camera; obtain a downscaled image by downscaling the captured image, wherein the downscaled image is an image that is less than a critical resolution; identify a region of interest included in the downscaled image by inputting the downscaled image comprising a preceding image and a subsequent image in a time order into a first artificial intelligence model, the first artificial intelligence model being trained to identify a region of interest in an input image by comparing a preceding image comprised in the input image with a subsequent image comprised in the input image; extract, from the captured image greater than or equal to the critical resolution, an object image in the captured image corresponding to the identified region of interest, wherein the object image is an image that is greater than or equal to a critical size; and obtain information on an object region included in the captured image by inputting the extracted object image into a second artificial intelligence model, the second artificial intelligence model being configured to identify an object region in an input image. ### Claim 2 (depends on 1) The electronic apparatus of claim 1, further comprising: a memory that stores the first artificial intelligence model and the second artificial intelligence model. ### Claim 3 (depends on 2) The electronic apparatus of claim 2, wherein the memory further stores a third artificial intelligence model trained to obtain feature information of an object based on an object region included in an input image, and the processor is further configured to: obtain the feature information of an object included in the object region by inputting the extracted object image to the third artificial intelligence model. ### Claim 4 (depends on 3) The electronic apparatus of claim 3, wherein the third artificial intelligence model comprises a plurality of artificial intelligence models trained to obtain different feature information of the object, the processor is further configured to obtain second feature information of the object by inputting first feature information obtained from a first model of the plurality of artificial intelligence models to a second model of the plurality of artificial intelligence models, the first model being different from the second model, and the plurality of artificial intelligence models are each trained to obtain other feature information of the object based on an object region included in an input image. ### Claim 5 (depends on 3) The electronic apparatus of claim 3, wherein the information on the object is information about a user area adjacent to the electronic apparatus in the captured image, the processor is further configured to obtain feature information of a user by inputting the extracted object image corresponding to the user area to the third artificial intelligence model, and the feature information of the user comprises at least one of facial recognition information, gender information, body shape information, or emotion recognition information of the user. ### Claim 6 (depends on 2) The electronic apparatus of claim 2, wherein the memory further stores a fourth artificial intelligence model trained to identify an object in an input image, and the processor is further configured to: identify whether an object is included in the object region by inputting the extracted object image into the fourth artificial intelligence model, and based on the object being included in the object region, input the extracted object image to the second artificial intelligence model. ### Claim 7 (depends on 1) The electronic apparatus of claim 1, wherein the first artificial intelligence model is a model trained using a sample image less than the critical resolution, and the second artificial intelligence model is a model trained using a sample image greater than or equal to the critical resolution. ### Claim 8 (depends on 1) The electronic apparatus of claim 1, wherein the processor is further configured to: resize the object image to be the critical size, and obtain the information on the object region by inputting the resized object image of the critical size to the second artificial intelligence model. ### Claim 9 (depends on 1) The electronic apparatus of claim 1, wherein the region of interest comprises at least one of: a region including an object, a region where a motion occurs, a color change region, or an illuminance change region. ### Claim 10 (depends on 1) The electronic apparatus of claim 1, wherein the electronic apparatus is a mobile robot, and the processor is further configured to control the mobile robot to move. ### Claim 11 (depends on 1) The electronic apparatus of claim 1, wherein the processor is further configured to: detect an intruder or a fire generation region based on the object image, and based on detecting the intruder or the fire generation region, perform a corrective action, wherein the corrective action includes at least one of: outputting an alarm audibly via a speaker or visually via a display, controlling a display to display the object image and/or the region of interest, or transmitting the information on the object to a user terminal. ### Claim 12 (depends on 1) The electronic apparatus of claim 1, further comprising the camera. ### Claim 13 A method comprising: obtaining an image captured by a camera; obtaining a downscaled image by downscaling the captured image, wherein the downscaled image is an image that is less than a critical resolution; identifying a region of interest included in the downscaled image by inputting the downscaled image comprising a preceding image and a subsequent image in a time order into a first artificial intelligence model, the first artificial intelligence model being trained to identify a region of interest in an input image by comparing a preceding image comprised in the input image with a subsequent image comprised in the input image; extracting, from the captured image greater than or equal to the critical resolution, an object image in the captured image corresponding to the identified region of interest, wherein the object image is an image that is greater than or equal to a critical size; and obtaining information on an object region included in the captured image by inputting the extracted object image into a second artificial intelligence model, the second artificial intelligence model being configured to identify an object region in an input image. ### Claim 14 (depends on 13) The method of claim 13, wherein the first artificial intelligence model is a model trained using a sample image less than the critical resolution, and the second artificial intelligence model is a model trained using a sample image greater than or equal to the critical resolution. ### Claim 15 (depends on 13) The method of claim 13, wherein the obtaining information on the object region comprises: resizing the object image to be the critical size, and obtaining the information on the object region by inputting the resized object image of the critical size to the second artificial intelligence model. ### Claim 16 (depends on 13) The method of claim 13, the method further comprises: obtaining feature information of an object included in the object region by inputting the extracted object image to a third artificial intelligence model, and wherein the third artificial intelligence model is a model trained to obtain feature information of an object based on an object region included in an input image. ### Claim 17 (depends on 16) The method of claim 16, wherein the third artificial intelligence model comprises a plurality of artificial intelligence models trained to obtain different feature information of the object, wherein the obtaining the feature information comprises obtaining second feature information of the object by inputting first feature information obtained from a first model of the plurality of artificial intelligence models to a second model of the plurality of artificial intelligence models, the first model being different from the second model, and wherein the plurality of artificial intelligence models are each trained to obtain other feature information of the object based on an object region included in an input image. ### Claim 18 (depends on 16) The method of claim 16, wherein the information on the object is information about a user area adjacent to an electronic apparatus in the captured image, wherein the obtaining the feature information comprises obtaining feature information of a user by inputting the extracted object image corresponding to the user area to the third artificial intelligence model, and wherein the feature information of the user comprises at least one of facial recognition information, gender information, body shape information, or emotion recognition information of the user. ### Claim 19 (depends on 13) The method of claim 13, the method further comprises: identifying whether an object is included in the object region by inputting the extracted object image into the fourth artificial intelligence model, and based on the object being included in the object region, inputting the extracted object image to the second artificial intelligence model, wherein the fourth artificial intelligence model is a model trained to identify an object in an input image.