# US 11,908,213 B2 · System and method to facilitate extraction and organization of information from paper, and other physical writing surfaces > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,213 B2 - Document ID: US11908213B2 - Status: Granted - Kind: B2 - Application number: 18/198,685 - Filed: 2023-05-17 - Published: 2023-09-14 - Granted: 2024-02-20 - Earliest claimed: 2021-02-12 - Assignees: ACCO Brands Corporation - Applicants: Acco Brands Corporation - Inventors: Bryan Povish; Juli Hallas; Ming Leung; Michael Rowe; Abbey Blundell - Web page: https://patentgrep.com/patent/US11908213B2 ## CPC - G06F 40/258: Heading extraction; Automatic titling; Numbering - G06V 30/1448: {based on markings or identifiers characterising the document or the area} - G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) - G06F 40/106: Display of layout of documents; Previewing - G06F 40/131: Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces - G06K 19/06037: {multi-dimensional coding} - G06V 30/142: using hand-held instruments; Constructional details of the instruments - G06V 30/15: {Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques} - G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables - G06F 40/151: Transformation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS ## Abstract Systems and methods for extracting information from a sheet of material to facilitate organization of information from paper, and other physical writing surfaces are provided. An example system includes a sheet of material and a device for scanning the sheet with an optical sensor. The sheet of material includes an indication region. The indication region allows for indictors to be marked corresponding with at least one a corresponding subregion to be extracted. The sheet of material further includes at least one fiducial mark for identifying a boundary of the sheet. The device includes a processor operably coupled to the optical sensor for causing the optical sensor to scan the sheet and detect a boundary thereof using the fiducial marks and further identify a designated subregion of the sheet. Upon identification of the designated subregion, the processor is configured to extract information contained in the designated subregion for organization of information. ## Claims ### Claim 1 A system for digitizing and extracting information from a sheet of material, the system comprising: a sheet of material having a plurality of subregions, the sheet including a plurality of fiducial marks adjacent a periphery of the sheet and further including a plurality of indicators each corresponding with at least one of the plurality of subregions; and a device including a processor, communication circuitry, a memory, and an optical sensor, the processor of the device operably coupled to the communication circuitry, the memory, and the optical sensor and configured to cause the optical sensor to scan the sheet and generate an image thereof, wherein the processor is further configured to detect a boundary of the sheet based at least in part on detection of the fiducial marks, wherein the processor is further configured to identify a designated subregion based at least in part on detection of at least one marked indicator corresponding with the designated subregion and extract information from the designated subregion, and wherein the processor is further configured to detect multiple, sequentially marked indicators of the plurality of indicators and extract information from each subregion corresponding with those marked indicators as a single image. ### Claim 2 (depends on 1) The system of claim 1, wherein the fiducial marks comprise a machine-readable data matrix including encoded data, and wherein the processor is further configured to identify information associated with the sheet of material based at least in part on the encoded data. ### Claim 3 (depends on 2) The system of claim 2, wherein the information associated with the sheet of material includes at least one of: a reference indicative of a stock keeping unit of the sheet; a reference indicative of an orientation of the sheet; dimensions of the sheet; margins of the sheet; a type of the plurality of indicators included on the sheet; and locations of the plurality of indicators included on the sheet. ### Claim 4 (depends on 2) The system of claim 2, wherein the machine-readable data matrix comprises at least one of an AprilTag and a QR code. ### Claim 5 A system for digitizing and extracting information from a sheet of material, the system comprising: a sheet of material having a plurality of subregions, the sheet including a plurality of fiducial marks adjacent a periphery of the sheet and further including a plurality of indicators each corresponding with at least one of the plurality of subregions, wherein the indicators are aligned along a longitudinal margin of the sheet and wherein the sheet comprises lined notebook paper and the plurality of subregions comprise one or more lines of the lined notebook paper; and a device including a processor, communication circuitry, a memory, and an optical sensor, the processor of the device operably coupled to the communication circuitry, the memory, and the optical sensor and configured to cause the optical sensor to scan the sheet and generate an image thereof, wherein the processor is further configured to detect a boundary of the sheet based at least in part on detection of the fiducial marks, and wherein the processor is further configured to identify a designated subregion based at least in part on detection of at least one marked indicator corresponding with the designated subregion and extract information from the designated subregion. ### Claim 6 (depends on 1) The system of claim 1, wherein the indicators comprise fillable shapes, and wherein the marked indicator is detected based at least in part on the fillable shape being at least partially filled by a user. ### Claim 7 (depends on 5) The system of claim 5, wherein the processor is further configured to detect multiple, sequentially marked indicators of the plurality of indicators and extract information from each subregion corresponding with those marked indicators as a single image. ### Claim 8 (depends on 5) The system of claim 5, wherein the information from the designated subregion is extracted by the processor as a cropped image. ### Claim 9 (depends on 1) The system of claim 1, wherein the processor is further configured to cause the communication circuitry to communicate the information extracted from the designated subregion to a remote server computer. ### Claim 10 (depends on 1) The system of claim 1, wherein the processor is further configured to generate a study aid based at least in part on the information extracted from the designated subregion, and wherein the processor is configured to store the study aid in the memory and wherein the study aid generated by the processor includes at least one of: a single-sided digital notecard; a double-sided digital notecard; and an outline. ### Claim 11 A notebook comprising: a plurality of scannable pages, each page including: a plurality of fiducial marks to facilitate identification of a peripheral boundary of the page, each of the plurality of fiducial marks comprising a machine-readable data matrix including encoded data indicative of information associated with the notebook; a plurality of subregions; and a plurality of indicators spaced along at least a portion of a length of each page, each of the plurality of indicators associated with and configured to identify at least one subregion of the plurality of subregions to facilitate extraction of information contained in the at least one subregion, wherein one of the plurality of fiducial marks is positioned at a different corner of each page and wherein each page comprises lined notebook paper and the plurality of subregions comprise one or more lines of the lined notebook paper. ### Claim 12 (depends on 11) The notebook of claim 11, wherein the machine-readable data matrix comprises at least one of an AprilTag and a QR code. ### Claim 13 (depends on 11) The notebook of claim 11, wherein the information associated with the notebook includes at least one of: a reference indicative of a stock keeping unit of the notebook; a reference indicative of an orientation of the pages; dimensions of the pages; a margin of the pages; a type of the indicators included on the pages; and locations of the indicators included on the pages. ### Claim 14 (depends on 11) The notebook of claim 11, wherein each of the indicators comprises a fillable shape configured to be at least partially filled in by a user. ### Claim 15 (depends on 11) The notebook of claim 11, wherein the indicators are aligned along a longitudinal margin of each page. ### Claim 16 (depends on 5) The system of claim 5, wherein the fiducial marks comprise a machine-readable data matrix including encoded data, and wherein the processor is further configured to identify information associated with the sheet of material based at least in part on the encoded data. ### Claim 17 (depends on 16) The system of claim 16, wherein the information associated with the sheet of material includes at least one of: a reference indicative of a stock keeping unit of the sheet; a reference indicative of an orientation of the sheet; dimensions of the sheet; margins of the sheet; a type of the plurality of indicators included on the sheet; and locations of the plurality of indicators included on the sheet. ### Claim 18 (depends on 16) The system of claim 16, wherein the machine-readable data matrix comprises at least one of an AprilTag and a QR code. ### Claim 19 (depends on 5) The system of claim 5, wherein the indicators comprise fillable shapes, and wherein the marked indicator is detected based at least in part on the fillable shape being at least partially filled by a user. ### Claim 20 (depends on 5) The system of claim 5, wherein the processor is further configured to cause the communication circuitry to communicate the information extracted from the designated subregion to a remote server computer.