# US 2003/0078804 A1 · Employee assessment tool

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0078804 A1
- Document ID: US20030078804A1
- Status: Published application
- Kind: A1
- Application number: 10/279,159
- Filed: 2002-10-24
- Published: 2003-04-24
- Earliest claimed: 2001-10-24
- Assignees: Employee Motivation & Performance Assessment, Inc.
- Inventors: Palmer Morrel-Samuels
- Web page: https://patentgrep.com/patent/US20030078804A1

## CPC

- G06Q 10/10: Office automation; Time management
- G06Q 10/0639: Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q 10/06398: {Performance of employee with respect to a job function}
- G06Q 30/0203: Market surveys; Market polls
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method of providing employee assessment services includes negotiating with an employer to administer surveys to its employees, and obtaining performance metrics relating to performance of a business of the employer. A survey is designed to obtain feature metrics relating to features of business culture germane to the business of the employer, and the survey is administered to the employees via a web-based interface. Obtained survey data are analyzed to identify statistically significant, noteworthy, consistent, and non-contradictory linkages between the feature metrics and the performance metrics, and these linkages are communicated to the employer in a readily understandable fashion.

## Claims

### Claim 1

A method of providing employee assessment services, comprising:
negotiating with an employer to administer surveys to its employees;
obtaining performance metrics relating to performance of a business of the employer;
designing a survey to obtain feature metrics relating to features of business culture relating to the business of the employer;
administering the survey to the employees, thereby obtaining survey data including the feature metrics;
compiling the survey data to identify statistically significant, noteworthy, consistent, and non-contradictory linkages between the feature metrics and the performance metrics; and
communicating the linkages between the feature metrics and the performance metrics to the employer.

### Claim 2 (depends on 1)

The method of claim 1, wherein said designing a survey includes designing an automated electronic survey to increase the utility of the survey for obtaining data comparable to those obtainable with a similar hardcopy survey, wherein said designing an automated electronic survey comprises:
providing content features including a visual display of a survey question with a graphic user interface component adapted to receive a response from a respondent;
providing information features including a plurality of visual displays communicating instructions for perceiving, interpreting, and completing the survey;
providing presentation features including visual display of the content features and the information features to enforce psychologically advantageous communication capabilities; and
providing access features including functional capabilities allowing the respondent to locate, initiate, navigate, complete, and submit the survey.

### Claim 3 (depends on 1)

The method of claim 1, wherein said administering the survey to the employees includes administering an automated electronic survey, including:
communicating survey questions to respondents, wherein the survey questions provide opportunities for respondents to record responses to the survey questions;
performing assessments of behaviors of the respondents during survey administration, thereby detecting a response of potentially decreased validity based on the assessments;
providing a respondent giving the response with an opportunity to review the response and its associated question, and to change the response based on the review; and
receiving the responses, thereby obtaining survey data.

### Claim 4 (depends on 1)

The method of claim 1, wherein said compiling survey data comprises:
gathering performance metrics relating to performance of the business;
obtaining survey data including feature metrics indicating features of business culture relating to the business;
constructing a model correlating the feature metrics with the performance metrics, thereby assigning a correlation coefficient relating to a particular feature metric and a particular performance metric; and
identifying a link between the particular feature metric and the particular performance metric based on statistical significance of the correlation coefficient and a magnitude of the correlation coefficient relative to a predetermined threshold.

### Claim 5

A method of designing an automated electronic survey to increase the utility of the survey for obtaining data comparable to those obtainable with a similar hardcopy survey, comprising:
providing content features including a visual display of a survey question with a graphic user interface component adapted to receive a response from a respondent;
providing information features including a plurality of visual displays communicating instructions for perceiving, interpreting, and completing the survey;
providing presentation features including visual display of the content features and the information features to enforce psychologically advantageous communication capabilities; and
providing access features including functional capabilities allowing the respondent to locate, initiate, navigate, complete, and submit the survey.

### Claim 6 (depends on 1)

The method of claim 1, wherein said providing content features includes providing a visual display of a survey question designed to elicit a response expressible in terms of frequency of occurrence, wherein said visual display includes a graphic user interface component having a plurality of exclusively selectable response option sub-components, wherein a plurality of selectable response option sub-components is arrayed to express a range corresponding to frequency of occurrence.

### Claim 7 (depends on 6)

The method of claim 6, wherein the range corresponding to frequency of occurrence corresponds to a rating scale of response options having exactly two extreme components, thereby generating ratio data wherein a metric distance between each response option is identical.

### Claim 8 (depends on 7)

The method of claim 7, wherein the two extreme components express frequencies of occurrence corresponding to never and always, thereby generating a normal distribution.

### Claim 9 (depends on 6)

The method of claim 6, wherein the rating scale has an odd number of alternatives, thereby providing a neutral answer option.

### Claim 10 (depends on 6)

The method of claim 6, wherein one of the response option sub-components permits the respondent to indicate no response.

### Claim 11 (depends on 5)

The method of claim 5, wherein said providing content features includes providing bivalent optional comment fields asking for exactly two optional comments, one comment being positive and the other being negative.

### Claim 12 (depends on 11)

The method of claim 11, wherein the bivalent optional comment fields have a requirement that the respondent provide a topic definition for the two optional comments.

### Claim 13 (depends on 11)

The method of claim 11, wherein the bivalent optional comment fields have equivalent probes inquiring of a single best feature and a single worst feature relating to a subject of the survey.

### Claim 14 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of graphics demonstrating cooperation between an employer of the respondent and a consulting company providing the survey.

### Claim 15 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of instructions communicating an explicit justification of demographic questions, wherein the instructions inform the respondent that a reason for requesting demographic information is to aggregate responses, not to identify individual respondents.

### Claim 16 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of instructions communicating an explicit assurance of anonymity, wherein the instructions inform the respondent that, with the exception of given responses, no identifying information is automatically collected during the survey.

### Claim 17 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of instructions communicating an explicit assurance of exclusive processing, wherein the instructions inform the respondent that data collected during the survey go directly and exclusively to an outside company for analysis.

### Claim 18 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of instructions communicating an explicit assurance of aggregation, wherein the instructions inform the respondent that an individual data record generated in the course of the survey will not be provided to an employer of the respondent.

### Claim 19 (depends on 5)

The method of claim 5, wherein said providing information features includes providing a visual display of graphics corresponding to an isometric progress indicator, wherein the indicator shows the respondent exactly how many questions they have completed and how many remain.

### Claim 20 (depends on 5)

The method of claim 5, wherein said providing information features includes providing time forecasting, wherein an instruction contains a precise count of questions in the survey and an approximate completion time.

### Claim 21 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing screen resizing functionality causing visual displays of graphics to automatically resize to fit a size of a window of an active display of the respondent, wherein an adaptive decorative border is implemented to handle active displays with different aspect ratios.

### Claim 22 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing skim proof instructions, wherein color highlighting of one key phrase in each paragraph of the instructions assures that respondents who skim the instructions acquire a first message that is substantially the same as a second message acquired by respondents who read the instructions with care.

### Claim 23 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing easy to read instructions suited to communication by an active display, wherein the instructions are broken into four short paragraphs, and each paragraph has one brief main point.

### Claim 24 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing a visual display of a section identifier, wherein respondents view in low contrast text a name of a section of the survey in which they are currently working.

### Claim 25 (depends on 5)

The method of claim 5, wherein said providing presentation features includes maintaining sparse screen content for each question, wherein each screen contains exactly one question.

### Claim 26 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing a graphic user interface component adapted to receive a response from the respondent using color-coded buttons showing one of three states, wherein un-selected response buttons appear gray, buttons that are about to be selected by a mouse-click because a cursor is nearby appear turquoise, and buttons that have been selected by a respondent's mouse-click appear as navy.

### Claim 27 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing task-dependent placement of demographic questions, wherein demographic questions are placed at a beginning of the survey if their completion is mandatory, and placed at an end of the survey if their completion is optional.

### Claim 28 (depends on 5)

The method of claim 5, wherein said providing presentation features includes providing visual symmetry between an active display of the respondent and a response scale provided by the graphic user interface component adapted to receive a response from the respondent, wherein the response scale appears in a center of a content window displayed on the active display, with no visual elements to disrupt the respondent's full scanning of the scale's entire length, regardless of the respondent's local settings.

### Claim 29 (depends on 5)

The method of claim 5, wherein said providing access features includes providing easy access for intended users, wherein a URL is chosen especially to be easy to remember and say even for users with few computer skills.

### Claim 30 (depends on 5)

The method of claim 5, wherein said providing access features includes providing exclusion of access for unwelcome users, wherein the survey is password protected on a company-wide basis so that non-employees of the employer will be unlikely to enter.

### Claim 31 (depends on 5)

The method of claim 5, wherein said providing access features includes providing user-generated passwords that require no memorization, wherein passwords are automatically generated as multi-letter strings created by a series of questions that each respondent has a high likelihood of answering with a unique set of responses.

### Claim 32 (depends on 5)

The method of claim 5, wherein said providing access features includes providing recovery capability for interrupted sessions, wherein a respondent may identify and reconstruct ratings after an unexpected interruption in the session using a password unique to the respondent.

### Claim 33 (depends on 32)

The method of claim 32, wherein said providing access features includes providing confidentiality safeguards for coincidental matches of passwords, wherein response data in fields that identify the respondent are automatically erased before recovery.

### Claim 34 (depends on 5)

The method of claim 5, wherein said providing access features includes providing a quick save and exit button, wherein a single button allows a respondent to save completed work on the survey and exit to a screen displaying no response information.

### Claim 35 (depends on 5)

The method of claim 5, wherein said providing access features includes providing lockout for omitted responses, wherein the respondent is required to enter a response to a question before advancing to a next question in a predetermined series of questions comprising content of the survey.

### Claim 36 (depends on 5)

The method of claim 5, wherein said providing access features includes providing capability for full navigation of the survey and revision of completed items, wherein a progress indicator has a go-to feature allowing the respondent to go to any previously completed item and at least one of review and revise a response.

### Claim 37 (depends on 5)

The method of claim 5, wherein said providing access features includes providing keyboard exclusion to discourage automatic responses, wherein a numeric keypad and keyboard are substantially disabled to prevent the respondent from continually hitting one key for every question.

### Claim 38 (depends on 5)

The method of claim 5, wherein said providing access features includes providing multi-test selection capability, wherein a test selection screen allows one company to have several different surveys running simultaneously.

### Claim 39 (depends on 5)

The method of claim 5, wherein said providing access features includes providing a spell-checker for textual input from the respondent, thereby allowing respondents to protect their identities in those cases where a respondent is famously associated with poor spelling or unique misspellings.

### Claim 40 (depends on 5)

The method of claim 5, wherein said providing access features includes providing a quit-box capability for respondent comments, wherein the comments are set up in a separate window on a respondent's active display, thereby allowing the respondent to cancel a comment easily by clicking on an “X” icon in a quit-box of the window.

### Claim 41 (depends on 5)

The method of claim 5, wherein said providing access features includes providing a finish-screen that allows at least one of revision and cancellation, wherein the respondent is presented with a final opportunity to review and revise responses before submission.

### Claim 42 (depends on 5)

The method of claim 5, wherein said providing access features includes providing a printable confirmation screen, wherein the respondent receives a recordable confirmation following submission of responses, wherein the recordable confirmation identifies the survey, thereby differentiating it from other, accessible surveys.

### Claim 43

A method of administering an automated electronic survey, comprising:
communicating survey questions to respondents, wherein the survey questions provide opportunities for respondents to record responses to the survey questions;
performing assessments of behaviors of the respondents during survey administration, thereby detecting a response of potentially decreased validity based on the assessments;
providing a respondent giving the response with an opportunity to review the response and its associated question, and to change the response based on the review; and
receiving the responses, thereby obtaining survey data.

### Claim 44 (depends on 43)

The method of claim 43, wherein said performing assessments of behaviors of the respondents during survey administration includes:
dividing responses to survey questions from a particular respondent on a particular survey into blocks, wherein the blocks have substantially equal numbers of responses and substantially reflect a sequence in which survey questions are communicated to the particular respondent;
computing standard deviations of responses in each block; and
examining the standard deviations to determine whether the responses are atypically uniform, thereby indicating that the respondent is likely experiencing fatigue.

### Claim 45 (depends on 44)

The method of claim 44, wherein said examining the standard deviations includes:
dividing a standard deviation for a particular block by an average standard deviation of all preceding blocks to obtain a result; and
equating a result not less than two with the respondent presumably experiencing fatigue.

### Claim 46 (depends on 43)

The method of claim 43, wherein said performing assessments of behaviors of the respondents during survey administration includes:
recording a reading-and-response time for a question and response; and
examining the reading-and-response time to determine whether it is too short based on predetermined criteria, thereby indicating that the respondent failed to adequately read the question.

### Claim 47 (depends on 43)

The method of claim 43, wherein said performing assessments of behaviors of the respondents during survey administration include:
recording a plurality of reading and response times for a plurality of questions completed by a particular respondent;
computing an average reading and response time for the particular respondent based on the plurality of reading and response times; and
determining whether a reading-and-response time for a particular question is too short based on the average reading and response time, thereby indicating that the respondent failed to adequately read the particular question.

### Claim 48 (depends on 43)

The method of claim 43, wherein said performing assessments of behaviors of the respondents during survey administration includes:
examining a first response to a first survey question from a particular respondent;
examining a second response to a second survey question from the particular respondent, wherein the first survey question and the second survey question perform related inquiries capable of eliciting contradictory responses; and
determining that the particular respondent misread at least one of the first survey question and the second survey question based on the first response being contradictory to the second response.

### Claim 49 (depends on 43)

The method of claim 43, wherein said performing assessments of behaviors of the respondents during survey administration includes:
computing a proportion of extreme responses from a particular respondent, wherein an extreme response corresponds to a response at an end of a rating scale suitable for recording responses to survey questions; and
comparing the proportion to a predetermined threshold to determine whether the particular respondent may have been at least one of overly harsh and overly generous in given responses.

### Claim 50

A method of compiling survey data to identify linkages between feature metrics and a plurality of performance metrics relating to performance of a business, comprising:
gathering performance metrics relating to performance of the business;
obtaining survey data including feature metrics indicating features of business culture relating to the business;
constructing a model correlating the feature metrics with the performance metrics, thereby assigning a correlation coefficient relating to a particular feature metric and a particular performance metric; and
identifying a link between the particular feature metric and the particular performance metric based on statistical significance of the correlation coefficient and a magnitude of the correlation coefficient relative to a predetermined threshold.

### Claim 51 (depends on 50)

The method of claim 50 comprising constructing multiple models controlling for multiple potentially significant confounding factors, thereby assigning multiple regression coefficients relating to the particular feature metric and the particular performance metric.

### Claim 52 (depends on 51)

The method of claim 51 comprising identifying a link between the particular feature metric and the particular performance metric based on aggregated magnitudes of the multiple regression coefficients that are statistically significant relative to at least one predetermined threshold.

### Claim 53 (depends on 52)

The method of claim 52 comprising deeming certain of the coefficients to be noteworthy coefficients based on their statistical significance and their respective magnitudes relative to a predetermined threshold, wherein said identifying a link between the particular feature metric and the particular performance metric is based on at least one associated correlation coefficient being deemed a noteworthy coefficient, and based on all associated noteworthy coefficients having a same valence, wherein a noteworthy coefficient having a value greater than zero is assigned a positive valence, and wherein a noteworthy coefficient having a value less than zero is assigned a negative valence.

### Claim 54

A web-based employee assessment tool for use with a computer network accessible to the public, comprising:
a server connected to the network and having a data store of at least one automated electronic survey designed to obtain feature metrics relating to features of business culture relating to at least one business of at least one employer, wherein said server is operable to distribute the automated electronic surveys to employees of the employer over the network and receive responses over the network from the employees that comprise data including the feature metrics;
an access control system controlling distribution of surveys by the server and adapted to allow employees of a particular employer to identify themselves as employees of the particular employer and to select to complete a particular survey designed to obtain feature metrics relating to features of business culture relating to a particular business of the particular employer; and
a plurality of filters adapted to perform assessments of behaviors of the employees during survey administration and thereby detect a response of potentially decreased validity based on the assessments, and adapted to provide an employee giving the response with an opportunity to review the response and its associated question, and to change the response based on the review.

### Claim 55 (depends on 54)

The assessment tool of claim 54, wherein said plurality of filters includes a fatigue detector adapted to divide responses to survey questions from a particular respondent on a particular survey into blocks, wherein the blocks have substantially equal numbers of responses and substantially reflect a sequence in which survey questions are communicated to the particular respondent, adapted to compute standard deviations of responses in each block, and adapted to examining the standard deviations to determine whether the responses are atypically uniform, thereby indicating the respondent experienced fatigue.

### Claim 56 (depends on 55)

The assessment tool of claim 55, wherein said fatigue detector is adapted to examine the standard deviations by dividing a standard deviation for a particular block by an average standard deviation of all preceding blocks to obtain a result, and by equating a result not less than two with the respondent experiencing fatigue.

### Claim 57 (depends on 54)

The assessment tool of claim 54, wherein said plurality of filters includes a skimming detector adapted to record a reading-and-response time for a question and response, and adapted to examine the reading-and-response time to determine whether it is too short based on predetermined criteria, thereby indicating that the respondent failed to adequately read the question.

### Claim 58 (depends on 54)

The assessment tool of claim 54, wherein said plurality of filters includes a skimming detector adapted to divide questions and responses to survey questions from a particular respondent on a particular survey into blocks, wherein the blocks have substantially equal numbers of questions and responses and substantially reflect a sequence in which survey questions are communicated to the particular respondent, adapted to record at least one reading and response time for questions and answers in a particular block, and adapted to examine at least one reading-and-response time to determine whether it is too short based on predetermined criteria, thereby indicating that the respondent failed to adequately read the questions of the particular block.

### Claim 59 (depends on 54)

The assessment tool of claim 54, wherein said plurality of filters includes a misreading detector adapted to examine a first response to a first survey question from a particular respondent, adapted to examine a second response to a second survey question from the particular respondent, wherein the first survey question and the second survey question perform related inquiries capable of eliciting contradictory responses, and adapted to determine that the particular respondent misread at least one of the first survey question and the second survey question based on the first response being contradictory to the second response.

### Claim 60 (depends on 54)

The assessment tool of claim 54, wherein said plurality of filters includes an extremity detector adapted to compute a proportion of extreme responses from a particular respondent, wherein an extreme response corresponds to a response at an end of a rating scale suitable for recording responses to survey questions, and adapted to compare the proportion to a predetermined threshold to determine whether the particular respondent may have been at least one of overly harsh and overly generous in given responses.

### Claim 61 (depends on 54)

The assessment tool of claim 54, wherein said access control system is adapted to allow employees of a particular employer to identify themselves as employees of the particular employer without revealing their identities as individuals.

### Claim 62 (depends on 54)

The assessment tool of claim 54, wherein a condition exists relating to a particular automated electronic survey for a particular employer residing in the data store of said server, and to said access control system allowing employees of the particular employer to identify themselves as employees of the particular employer and to select to complete the particular automated electronic survey, wherein said condition includes existence of a contract between the employer and an outside consulting company, wherein the contract contains a term guaranteeing confidentiality of individual employee responses, and that survey data will only be made available to the employer in a post-analytically aggregate form, wherein the particular automated electronic survey is adapted to clarify a relationship between the employer and the outside consulting company, and to inform employees of the condition.
