Book scanning using machine-trained model

A method of preparing an input-output data pair for training a machine-trainable model, the method comprising:

• providing a markings page as part of a bound book, the markings page comprising texts and a plurality of predetermined markings thereon;
• providing, for each of the plurality of predetermined markings, 2-dimensional (2D) location when the markings page is at an unbound and flat state;
• opening the bound book such that the markings page is in an open state and is curled as opposed to being flat;
• capturing, using a camera, a photographic image of the markings page in the open state, wherein the markings page appearing on the photographic image is distorted from the unbound and flat state at least due to curling of the markings page in the open state, due to a 3-dimensional (3D) camera location relative to the markings page, and further due to a 3D camera orientation relative to the markings page at the time of capturing the photographic image;
• processing the photographic image to obtain a distorted 2D location of each of the plurality of predetermined markings on the photographic image;
• and
• computing a set of parameters representing distortion of the markings page on the photographic image, wherein the set of parameters comprises:
• a first subset of parameters relating to a 3D camera location relative to the markings page at the time of capturing the photographic image,
• a second subset of parameters relating to a 3D camera orientation of the camera relative to the markings page at the time of capturing the photographic image, and
• a third subset of parameters relating to curling of the markings page in the open state at the time of capturing the photographic image,
• wherein computing the set of parameters involves a process of iteration comprising:
• assigning values to the first, second and third subsets of parameters,
• computing a distorted 2D location for each of the plurality of predetermined markings on a distorted image that would be obtained, using the assigned values, by curling the markings page from the unbound and flat state and by taking a photographic image of the curled markings page using a camera located at a 3D camera location relative to the curled markings page and in a 3D camera orientation relative to the markings page;
• computing a loss representing a difference between the captured photographic image and the distorted image that would be obtained using the assigned values, wherein computing the loss uses distorted 2D locations for at least part of the plurality of predetermined markings on the captured photographic image and further uses distorted 2D locations for at least part of the plurality of predetermined markings on the distorted image;
• comparing the loss to a predetermined threshold value to determine if the loss is equal to or smaller than the predetermined threshold value;
• when it is determined that the loss is greater than the predetermined threshold value, assigning updated values to the set of parameters;
• and
• repeating a sequence of computing a distorted 2D location, computing a locational difference, computing a loss, comparing the loss, and assigning updated values until the loss is equal to or smaller than the predetermined threshold value,
• wherein upon determining that the loss is equal to or smaller than the predetermined value, associating the currently assigned values that have led to the loss equal to or smaller than the predetermined threshold value with the photographic image of the markings page or a modified version thereof such that the photographic image of the markings page or the modified version thereof is an input of the input-output data pair and the currently assigned values are output data of the input-output data pair.