# US 10,310,710 B2 · Determination of indentation levels of a bulleted list > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,710 B2 - Document ID: US10310710B2 - Status: Granted - Kind: B2 - Application number: 15/280,493 - Filed: 2016-09-29 - Published: 2018-03-29 - Granted: 2019-06-04 - Earliest claimed: 2016-09-29 - Assignees: Konica Minolta Laboratory U.S.A., Inc. - Applicants: Konica Minolta Laboratory U.S.A., Inc. - Inventors: Jason Grams - Web page: https://patentgrep.com/patent/US10310710B2 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 40/137: Hierarchical processing, e.g. outlines - G06F 40/171: by use of digital ink - G06V 30/40: Document-oriented image-based pattern recognition - G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Determination of indentation levels of a bulleted list contained in an input image involves iteratively performing comparison cycles in which the horizontal positions of vertically adjacent pairs of bullets are compared for similarity. If similar in position, the bullets in the pair are aggregated into the same indentation level. Subsequent comparison cycles use bullet positions which were adjusted in a prior comparison cycle to address progressive shifting that may sometimes be present in an input image. Readjustment cycles can be performed to aggregate bullets into the same indentation level, if similar in position, even when the bullets are not vertically adjacent to each other. ## Claims ### Claim 1 A method for determining indentation levels of a bulleted list in an image, the bulleted list having a plurality of bullets that are vertically arranged, each bullet having a horizontal position, the method comprising: storing, in memory of a data processing system, the horizontal position of each of the bullets; and performing, by a processor of the data processing system, a plurality of comparison cycles, wherein each comparison cycle is performed on a pair of vertically adjacent bullets, each pair includes a current bullet and a next bullet that is vertically adjacent to the current bullet, the plurality of comparison cycles includes a first comparison cycle and a second comparison cycle following the first comparison cycle, the next bullet of the first comparison cycle becomes the current bullet of the second comparison cycle, wherein the performance of each of the comparison cycles includes: obtaining an index position if the comparison cycle is the first comparison cycle, obtaining, from the memory of the data processing system, the horizontal position of the current bullet and the horizontal position of the next bullet, determining whether or not a difference between the obtained horizontal position of the current bullet and the obtained horizontal position of the next bullet satisfies a comparison requirement to identify the current bullet and the next bullet as being alike in indentation level, when the comparison requirement is satisfied, adjusting the horizontal position of the next bullet according to the index position and then storing the adjusted horizontal position of the next bullet in the memory of the data processing system, and adjusting the index position when the comparison requirement is not satisfied, wherein the image containing the bulleted list does not encode aggregation of the bullets into indentation levels, and the method further comprises: before the performance of the plurality of comparison cycles, deriving the horizontal position of each of the bullets from the image; and after the performance of the plurality of comparison cycles, generating a list object that encodes aggregation of the bullets into indentation levels according to the plurality of comparison cycles. ### Claim 2 (depends on 1) The method of claim 1, wherein adjusting the horizontal position of the next bullet includes setting the horizontal position of the next bullet equal to the index position, and adjusting the index position includes using the obtained horizontal position of the next bullet to modify the index position. ### Claim 3 (depends on 2) The method of claim 2, wherein using the horizontal position of the next bullet to modify the index position includes setting the index position equal to the obtained horizontal position of the next bullet. ### Claim 4 (depends on 1) The method of claim 1, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the comparison requirement is based on one or more of the line heights. ### Claim 5 (depends on 1) The method of claim 1, wherein the plurality of comparison cycles includes a third comparison cycle following the second comparison cycle, and the next bullet of the second comparison cycle becomes the current bullet of the third comparison cycle. ### Claim 6 (depends on 1) The method of claim 1, further comprising, after the performance of the plurality of comparison cycles: sorting the horizontal positions of the bullets to identify pairs of horizontally adjacent bullets, wherein each pair of horizontally adjacent bullets includes a first bullet and a second bullet having a horizontal position that is equal to that of the first bullet or is closest to that of the first bullet relative to the other bullets in the plurality of bullets; and performing a plurality of readjustment cycles, wherein each readjustment cycle is performed on one of the pairs of horizontally adjacent bullets, the plurality of readjustment cycles includes a first readjustment cycle and a second readjustment cycle following the first readjustment cycle, the second bullet of the first readjustment cycle becomes the first bullet of the second comparison cycle, wherein the performance of each of the readjustment cycles includes: obtaining an offset position if the readjustment cycle is the first readjustment cycle, determining whether or not a difference between the horizontal position of the first bullet and the horizontal position of the second bullet satisfies a readjustment requirement to identify the first bullet and the second bullet as being alike in indentation level, adjusting the horizontal position of the second bullet according to the offset position when the readjustment requirement is satisfied, and adjusting the offset position when the readjustment requirement is not satisfied. ### Claim 7 (depends on 6) The method of claim 6, wherein adjusting the horizontal position of the second bullet includes setting the horizontal position of the second bullet equal to the offset position, and adjusting the offset position includes applying an increment to the offset position. ### Claim 8 (depends on 6) The method of claim 6, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the readjustment requirement is based on one or more of the line heights. ### Claim 9 (depends on 6) The method of claim 6, wherein the plurality of readjustment cycles includes a third readjustment cycle following the second readjustment cycle, and the second bullet of the second readjustment cycle becomes the first bullet of the third readjustment cycle. ### Claim 10 A non-transitory computer readable medium having stored thereon computer readable instructions that, when executed by a hardware processor, cause a data processing system to perform a method for determining indentation levels of a bulleted list in an image, the bulleted list having a plurality of bullets that are vertically arranged, each bullet having a horizontal position, the method performed by the data processing system comprising: performing a plurality of comparison cycles, wherein each comparison cycle is performed on a pair of vertically adjacent bullets, each pair includes a current bullet and a next bullet that is vertically adjacent to the current bullet, the plurality of comparison cycles includes a first comparison cycle and a second comparison cycle following the first comparison cycle, the next bullet of the first comparison cycle becomes the current bullet of the second comparison cycle, wherein the performance of each of the comparison cycles includes: obtaining an index position if the comparison cycle is the first comparison cycle, determining whether or not a difference between the horizontal position of the current bullet and the horizontal position of the next bullet satisfies a comparison requirement to identify the current bullet and the next bullet as being alike in indentation level, adjusting the horizontal position of the next bullet according to the index position when the comparison requirement is satisfied, and adjusting the index position when the comparison requirement is not satisfied, wherein the image containing the bulleted list does not encode aggregation of the bullets into indentation levels, and the method performed by the data processing system further comprises: before the performance of the plurality of comparison cycles, deriving the horizontal position of each of the bullets from the image; and after the performance of the plurality of comparison cycles, generating a list object that encodes aggregation of the bullets into indentation levels according to the plurality of comparison cycles. ### Claim 11 (depends on 10) The non-transitory computer readable medium of claim 10, wherein adjusting the horizontal position of the next bullet includes setting the horizontal position of the next bullet equal to the index position, and adjusting the index position includes using the horizontal position of the next bullet to modify the index position. ### Claim 12 (depends on 11) The non-transitory computer readable medium of claim 11, wherein using the horizontal position of the next bullet to modify the index position includes setting the index position equal to the horizontal position of the next bullet. ### Claim 13 (depends on 10) The non-transitory computer readable medium of claim 10, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the comparison requirement is based on one or more of the line heights. ### Claim 14 (depends on 10) The non-transitory computer readable medium of claim 10, wherein the plurality of comparison cycles includes a third comparison cycle following the second comparison cycle, and the next bullet of the second comparison cycle becomes the current bullet of the third comparison cycle. ### Claim 15 (depends on 10) The non-transitory computer readable medium of claim 10, wherein the method performed by the data processing system further comprises, after the performance of the plurality of comparison cycles: sorting the horizontal positions of the bullets to identify pairs of horizontally adjacent bullets, wherein each pair of horizontally adjacent bullets includes a first bullet and a second bullet having a horizontal position that is equal to that of the first bullet or is closest to that of the first bullet relative to the other bullets in the plurality of bullets; and performing a plurality of readjustment cycles, wherein each readjustment cycle is performed on one of the pairs of horizontally adjacent bullets, the plurality of readjustment cycles includes a first readjustment cycle and a second readjustment cycle following the first readjustment cycle, the second bullet of the first readjustment cycle becomes the first bullet of the second comparison cycle, wherein the performance of each of the readjustment cycles includes: obtaining an offset position if the readjustment cycle is the first readjustment cycle, determining whether or not a difference between the horizontal position of the first bullet and the horizontal position of the second bullet satisfies a readjustment requirement to identify the first bullet and the second bullet as being alike in indentation level, adjusting the horizontal position of the second bullet according to the offset position when the readjustment requirement is satisfied, and adjusting the offset position when the readjustment requirement is not satisfied. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, wherein adjusting the horizontal position of the second bullet includes setting the horizontal position of the second bullet equal to the offset position, and adjusting the offset position includes applying an increment to the offset position. ### Claim 17 (depends on 15) The non-transitory computer readable medium of claim 15, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the readjustment requirement is based on one or more of the line heights. ### Claim 18 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the plurality of readjustment cycles includes a third readjustment cycle following the second readjustment cycle, and the second bullet of the second readjustment cycle becomes the first bullet of the third readjustment cycle. ### Claim 19 A data processing system for determining indentation levels of a bulleted list in an image, the bulleted list having a plurality of bullets that are vertically arranged, each bullet having a horizontal position, the system comprising: a hardware processor; and one or more memory devices associated with the hardware processor; wherein the hardware processor is configured to determine indentation levels by performing a method that comprises: performing a plurality of comparison cycles, wherein each comparison cycle is performed on a pair of vertically adjacent bullets, each pair includes a current bullet and a next bullet that is vertically adjacent to the current bullet, the plurality of comparison cycles includes a first comparison cycle and a second comparison cycle following the first comparison cycle, the next bullet of the first comparison cycle becomes the current bullet of the second comparison cycle, wherein the performance of each of the comparison cycles includes: obtaining an index position if the comparison cycle is the first comparison cycle, determining whether or not a difference between the horizontal position of the current bullet and the horizontal position of the next bullet satisfies a comparison requirement to identify the current bullet and the next bullet as being alike in indentation level, adjusting the horizontal position of the next bullet according to the index position when the comparison requirement is satisfied, and adjusting the index position when the comparison requirement is not satisfied, wherein the image containing the bulleted list does not encode aggregation of the bullets into indentation levels, and the method performed by the hardware processor further comprises: before the performance of the plurality of comparison cycles, deriving the horizontal position of each of the bullets from the image; and after the performance of the plurality of comparison cycles, generating a list object that encodes aggregation of the bullets into indentation levels according to the plurality of comparison cycles. ### Claim 20 (depends on 19) The system of claim 19, wherein adjusting the horizontal position of the next bullet includes setting the horizontal position of the next bullet equal to the index position, and adjusting the index position includes using the horizontal position of the next bullet to modify the index position. ### Claim 21 (depends on 20) The system of claim 20, wherein using the horizontal position of the next bullet to modify the index position includes setting the index position equal to the horizontal position of the next bullet. ### Claim 22 (depends on 19) The system of claim 19, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the comparison requirement is based on one or more of the line heights. ### Claim 23 (depends on 19) The system of claim 19, wherein the plurality of comparison cycles includes a third comparison cycle following the second comparison cycle, and the next bullet of the second comparison cycle becomes the current bullet of the third comparison cycle. ### Claim 24 (depends on 19) The system of claim 19, wherein the method performed by the hardware processor further comprises, after the performance of the plurality of comparison cycles: sorting the horizontal positions of the bullets to identify pairs of horizontally adjacent bullets, wherein each pair of horizontally adjacent bullets includes a first bullet and a second bullet having a horizontal position that is equal to that of the first bullet or is closest to that of the first bullet relative to the other bullets in the plurality of bullets; and performing a plurality of readjustment cycles, wherein each readjustment cycle is performed on one of the pairs of horizontally adjacent bullets, the plurality of readjustment cycles includes a first readjustment cycle and a second readjustment cycle following the first readjustment cycle, the second bullet of the first readjustment cycle becomes the first bullet of the second comparison cycle, wherein the performance of each of the readjustment cycles includes: obtaining an offset position if the readjustment cycle is the first readjustment cycle, determining whether or not a difference between the horizontal position of the first bullet and the horizontal position of the second bullet satisfies a readjustment requirement to identify the first bullet and the second bullet as being alike in indentation level, adjusting the horizontal position of the second bullet according to the offset position when the readjustment requirement is satisfied, and adjusting the offset position when the readjustment requirement is not satisfied. ### Claim 25 (depends on 24) The system of claim 24, wherein adjusting the horizontal position of the second bullet includes setting the horizontal position of the second bullet equal to the offset position, and adjusting the offset position includes applying an increment to the offset position. ### Claim 26 (depends on 24) The system of claim 24, wherein each bullet leads a plurality of objects enclosed in a rectangular perimeter having a line height, and the readjustment requirement is based on one or more of the line heights. ### Claim 27 (depends on 24) The system of claim 24, wherein the plurality of readjustment cycles includes a third readjustment cycle following the second readjustment cycle, and the second bullet of the second readjustment cycle becomes the first bullet of the third readjustment cycle.