# US 11,908,101 B2 · Writing surface boundary markers for computer vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,101 B2 - Document ID: US11908101B2 - Status: Granted - Kind: B2 - Application number: 17/681,200 - Filed: 2022-02-25 - Published: 2022-06-09 - Granted: 2024-02-20 - Earliest claimed: 2019-04-12 - Assignees: Rocket Innovations, Inc. - Applicants: Rocket Innovations, Inc. - Inventors: Joseph Lemay; Jacob Epstein - Web page: https://patentgrep.com/patent/US11908101B2 ## CPC - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 5/80: Geometric correction - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - B43L 1/00: Repeatedly-usable boards or tablets for writing or drawing (drawing-boards B43L5/00) - G06T 2207/20132: Image cropping - G06T 2207/30204: Marker - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B43L: ARTICLES FOR WRITING OR DRAWING UPON; {WRITING OR DRAWING AIDS;} ACCESSORIES FOR WRITING OR DRAWING (workshop equipment for marking-out work B25H7/00; teaching hand-writing or drawing G09B11/00) ## Abstract A system for capturing, organizing, and storing handwritten notes includes a plurality of boundary markers. The boundary markers are configured to be positioned on a writing surface. The system also includes a tangible non-transitory computer readable medium encoded with instructions which, when run on a camera-equipped computing device, causes the camera-equipped computing device to execute processes. The processes include capturing an image of the writing surface with the markers thereon. The processes also include detecting the boundary markers in the captured image. Additionally, the processes include identifying a virtual boundary in the captured image based on the positions of the boundary markers. The processes then unwarp a portion of the captured image within the virtual boundary to produce an unwarped image. ## Claims ### Claim 1 A system for capturing, organizing, and storing handwritten notes, the system comprising: a plurality of boundary markers configured to be positioned on a writing surface; a tangible non-transitory computer readable medium encoded with instructions which, when run on a camera-equipped computing device, causes the camera-equipped computing device to execute processes comprising: detecting the boundary markers from an image of the writing surface having the markers thereon; identifying a virtual boundary in the image based on the positions of the boundary markers; and unwarping a portion of the image within the virtual boundary to produce an unwarped image. ### Claim 2 (depends on 1) A system according to claim 1, wherein the executed processes further comprise broadcasting the unwarped image and updating the broadcast as new images are captured. ### Claim 3 (depends on 1) A system according to claim 1, further comprising saving the unwarped image in an image store. ### Claim 4 (depends on 1) A system according to claim 1, wherein the processes further comprise cropping the boundary markers out of the image. ### Claim 5 (depends on 1) A system according to claim 1, wherein the writing surface is a whiteboard or a wall. ### Claim 6 (depends on 1) A system according to claim 1, wherein the markers are formed from silicone. ### Claim 7 (depends on 1) A system according to claim 1, wherein the markers couple to the writing surface using an adhesive and/or microsuction. ### Claim 8 (depends on 1) A system according to claim 1, wherein the markers have a thickness of between about 0.5 millimeters and about 3 millimeters. ### Claim 9 (depends on 1) A system according to claim 1, wherein the executed processes further comprise broadcasting the unwarped image and updating the broadcast as new images are captured. ### Claim 10 A method for capturing and storing handwritten notes, the method comprising: placing a plurality of boundary markers on a writing surface, such that the boundary markers define a virtual boundary encompassing the handwritten notes; capturing an image of a writing surface with an electronic device; determining the boundary based on positions of the markers in the writing surface image; unwarping a portion of the captured image within the virtual boundary to produce an unwarped image; and cropping the unwarped image based on the position of the detected boundary. ### Claim 11 (depends on 10) The method as defined by claim 10, wherein the boundary markers include a fluorescent color. ### Claim 12 (depends on 11) The method as defined by claim 11, wherein identifying the position of the markers comprises identifying the fluorescent color in the image. ### Claim 13 (depends on 10) The method as defined by claim 10, further comprising storing the unwarped image. ### Claim 14 (depends on 10) The method as defined by claim 10, further comprising: removing the background from the writing surface image; and enhancing the writing surface image. ### Claim 15 A system for sharing handwritten notes, the system comprising: a computer device coupled with a camera, the camera configured to view a writing surface having content; a plurality of boundary markers configured to be positioned between the writing surface and the camera to define a virtual boundary around a portion of the writing surface that includes the content, the computer device configured to: (1) detect the boundary markers, (2) determine the virtual boundary, and (3) provide a deskewed image of the portion of the writing surface. ### Claim 16 (depends on 15) The system as defined by claim 15, further comprising a frame that includes a transparent annotation surface over the writing surface. ### Claim 17 (depends on 16) The system as defined by claim 16, wherein the annotation surface is configured to be written on with thermochromic ink. ### Claim 18 (depends on 15) The system as defined by claim 15, wherein the processes further comprise: removing the background from the captured image, cropping the captured image using the virtual boundary in the image, and enhancing the image. ### Claim 19 (depends on 15) The system as defined by claim 15, wherein the processes performed by the computing device are performed in response to taking a picture of the writing surface. ### Claim 20 (depends on 15) The system as defined by claim 15, wherein the computer device is further configured to broadcast the deskewed image.