# US 2023/0058043 A1 · Mobile terminal and operating method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0058043 A1 - Document ID: US20230058043A1 - Status: Published application - Kind: A1 - Application number: 17/879,201 - Filed: 2022-08-02 - Published: 2023-02-23 - Earliest claimed: 2021-08-19 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Sungmok HWANG; Jinok Kim; Yeonkyung Chae - Web page: https://patentgrep.com/patent/US20230058043A1 ## CPC - G06T 7/62: of area, perimeter, diameter or volume - G06T 7/73: using feature-based methods - G06T 2207/20224: Image subtraction - G06T 2207/30204: Marker - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A mobile terminal may include a camera, a display unit, and a processor configured to: determine whether two markers attached to two doors are present from an image captured through the camera; when the two markers are present, acquire center points of the two markers; acquire a y-axis pixel distance difference between the acquired center points; and estimate a height difference between the two doors based on the acquired y-axis pixel distance difference. ## Claims ### Claim 1 A mobile terminal comprising: a camera; a processor configured to: determine whether two markers attached, respectively, to two doors are present in an image captured through the camera; when the two markers are determined to be present in the image, acquire center points of the two markers; acquire a y-axis pixel distance difference between the acquired center points; and estimate a height difference between the two doors based on the acquired y-axis pixel distance difference. ### Claim 2 (depends on 1) The mobile terminal of claim 1, wherein one of the markers comprises a plurality of unit pixels, and wherein the processor is configured to calculate the height difference based on multiplying the y-axis pixel distance difference between the center points by a length of one of the unit pixels. ### Claim 3 (depends on 2) The mobile terminal of claim 2, further comprising a memory configured to store the length of the one of the unit pixels included in the marker. ### Claim 4 (depends on 1) The mobile terminal of claim 1, further comprising: a display unit, wherein the processor is configured to: when the height difference is equal to or greater than a threshold value, display a height difference occurrence notification on the display unit; and when the height difference is less than the threshold value, display a height difference non-occurrence notification on the display unit. ### Claim 5 (depends on 4) The mobile terminal of claim 4, wherein the height difference occurrence notification comprises a value of the estimated height difference and an adjustment guide for adjusting the height difference. ### Claim 6 (depends on 1) The mobile terminal of claim 1, wherein the processor, when the two markers are determined to be not present in the image, is configured to: extract upper edges of the two doors from the image; extract a plurality of straight lines from the extracted upper edges; extract a first matched straight line corresponding to a first door and a second matched straight line corresponding to a second door from the plurality of extracted straight lines; and estimate, as the height difference, a y-axis distance difference between the first matched straight line and the second matched straight line. ### Claim 7 (depends on 6) The mobile terminal of claim 6, wherein the processor is configured to: extract the upper edges by using a Canny edge detection technique; and extract the plurality of straight lines from the extracted upper edges by using a Hough transform technique. ### Claim 8 (depends on 7) The mobile terminal of claim 7, wherein the processor is configured to: extract, as the first matched straight line, a first straight line corresponding to an average value of y-axis values of both end points of straight lines corresponding to an upper end of the first door among the plurality of straight lines; and extract, as the second matched straight line, a second straight line corresponding to an average value of y-axis values of both end points of straight lines corresponding to an upper end of the second door among the plurality of straight lines. ### Claim 9 An operating method of a mobile terminal, the operating method comprising: acquiring an image of a home appliance through a camera; determining whether two markers attached to two doors are present in the acquired image; when the two markers are determined to be present, acquiring center points of the two markers; acquiring a y-axis pixel distance difference between the acquired center points; and estimating a height difference between the two doors based on the acquired y-axis pixel distance difference. ### Claim 10 (depends on 9) The method of claim 9, wherein one of the markers comprises a plurality of unit pixels, and wherein estimating the height difference between the two doors includes multiplying the y-axis pixel distance difference between the center points by a length of one of the unit pixels. ### Claim 11 (depends on 9) The method of claim 9, further comprising: when the height difference is equal to or greater than a threshold value, displaying a height difference occurrence notification; and when the height difference is less than the threshold value, displaying a height difference non-occurrence notification. ### Claim 12 (depends on 11) The method of claim 11, wherein the height difference occurrence notification includes a value of the estimated height difference and an adjustment guide for adjusting the height difference. ### Claim 13 (depends on 9) The method of claim 9, further comprising: extracting, when the two markers are determined not to be present in the image, upper edges of the two doors from the image; extracting a plurality of straight lines from the extracted upper edges; extracting a first matched straight line corresponding to a first door and a second matched straight line corresponding to a second door from the plurality of extracted straight lines; and estimating, as the height difference, a y-axis distance difference between the first matched straight line and the second matched straight line. ### Claim 14 (depends on 13) The method of claim 13, wherein the method further comprises: extracting the upper edges by using a Canny edge detection technique; and extracting the plurality of straight lines from the extracted upper edges by using a Hough transform technique. ### Claim 15 (depends on 13) The method of claim 13, further comprising: extracting, as the first matched straight line, a first straight line corresponding to an average value of y-axis values of both end points of straight lines corresponding to an upper end of the first door among the plurality of straight lines; and extracting, as the second matched straight line, a second straight line corresponding to an average value of y-axis values of both end points of straight lines corresponding to an upper end of the second door among the plurality of straight lines. ### Claim 16 A non-transitory computer-readable storage medium for storing instructions causing a processor to execute an operating method of a mobile terminal, the operating method comprising: acquiring an image of a home appliance; determining, based on the image, whether two markers attached to two doors, respectively; when the two markers are determined to be attached to the two doors, acquiring center points of the two markers; acquiring a pixel distance difference between the acquired center points in a height direction; and estimating a height difference between the two doors based on the acquired pixel distance difference. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein one of the markers comprises a plurality of unit pixels, and wherein estimating the height difference between the two doors includes multiplying the pixel distance difference between the center points by a length of one of the unit pixels. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the operating method further comprises: when the height difference is equal to or greater than a threshold value, displaying a height difference occurrence notification on the display unit; and when the height difference is less than the threshold value, displaying a height difference non-occurrence notification on the display unit. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the operating method further comprises: extracting, when the two markers are determined not to be attached to the doors, upper edges of the two doors from the image; extracting a plurality of straight lines from the extracted upper edges; extracting a first matched straight line corresponding to a first door and a second matched straight line corresponding to a second door from the plurality of extracted straight lines; and estimating, as the height difference, a distance difference between the first matched straight line and the second matched straight line. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium of claim 19, wherein the operating method further comprises: extracting, as the first matched straight line, a first straight line corresponding to an average height value of both end points of straight lines corresponding to an upper end of the first door among the plurality of straight lines; and extracting, as the second matched straight line, a second straight line corresponding to an average height value of both end points of straight lines corresponding to an upper end of the second door among the plurality of straight lines.