Loading patent…
AssigneeChegg, Inc.
InventorsJoshua John Richardson, Vincent Le Chevalier, Kostiantyn Havriuk, Semenov Vasil Alexandrovich
Estimating the term…
An online review system determines scores for document authors and document reviewers. An author score is based on a number of comments added to a document by reviewers. One way to compute the author score is to use a ratio of a number of comments provided by reviewers of a document to a number of lines of document contents provided by the author. A reviewer score for a reviewer is based on an amount of comments subsequently made by other reviewers, and on the document content itself, such as a ratio of a number of comments provided in subsequent reviews of a document to a number of lines of original document contents. In some embodiments, the online review system monitors trends in scores of individuals or teams.
A computer-implemented method comprising:
A non-transitory computer-readable storage medium storing computer program instructions executable by one or more processors of a system to perform steps comprising:
Dependencies are inferred from the claim text.
Drawings are temporarily unavailable. Open the PDF at USPTO
From the US filing history. Foreign family members aren’t listed.
No references of this kind are recorded.
With adaptation to user needs: using ranking
Details of database functions independent of the retrieved data types: document management systems
Editing, e.g. inserting or deleting: annotation, e.g. comment data or footnotes
Performance analysis of employees; Performance analysis of enterprise or organisation operations: quality analysis or management
About 10 years 4 months of term left · ends 3 Mar 2037 (estimated)
This disclosure relates generally to content review systems, and in particular to scoring content and content reviews.
Written content, such as code for software programs and other written documents, often benefits from peer review. Software development, one example of content creation, may be overseen by managers such as project leads who spend many working hours reviewing code produced by their developers and appraising peer reviews provided by other developers, such as team members. Complex software projects can involve many developers in many locations, and reviewing code and code reviews in such a situation can be time-consuming and complicated for intermediate reviewers and project leads. A complicated reviewing process may result in a loss of productivity and poor content quality due to a possible lack of valuable peer and manager feedback and due to having no way of monitoring changes in quality of content and content reviews over time.
An online system collects content (e.g., as contained in documents) from authors and collects content reviews from participating reviewers. Authors upload their documents to the online system and receive feedback, reviews, ratings, or other recommendations from reviewers. Some examples of reviewers who may review a document include peer reviewers, managers, people managers, project managers, quality-assurance agents, quality assurance engineers, publication editors, technical leads, subject-matter experts, educators, an author providing a self-review, artificial intelligence agents, and members of the public. In some embodiments, reviewers provide feedback for a document by adding comments inline in the document. Comments may include suggestions for improvements, citations of logical errors, and ideas for stylistic changes, for example. In some embodiments, reviewers can also include positive feedback. In some embodiments, a set of predefined rules may be used to include metadata indicating why a reviewer included a comment in a document. Such rules may be associated with weighting values, such that a score based on a number of comments received may be influenced by the specific comment types.
FIG. 1 is a high level block diagram that illustrates an environment of an online review system, in accordance with an embodiment.
FIG. 2A illustrates an example user interface for an online review system, in accordance with an embodiment.
FIG. 2B illustrates a second example user interface for an online review system, in accordance with an embodiment.
The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
System Environment
Embodiments described herein provide for collection and analysis of documents and reviews of document contents in an online review system. In one embodiment, the online review system analyzes reviewed documents to determine scores for authors and reviewers who provided feedback about the content. The online review system may further use the scores to monitor trends related to the quality and efficiency of work done by authors and reviewers. The systems and methods described herein provide a tool for authors, reviewers, and managers to keep track of team performance.