# US 2024/0265733 A1 · Personal care monitoring and feedback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0265733 A1
- Document ID: US20240265733A1
- Status: Published application
- Kind: A1
- Application number: 18/580,972
- Filed: 2022-07-22
- Published: 2024-08-08
- Earliest claimed: 2021-07-29
- Assignees: Conopco, Inc., d/b/a UNILEVER
- Applicants: Conopco, Inc., d/b/a UNILEVER
- Inventors: Anthony BROWN; Paul John CUNNINGHAM; Robert Lindsay TRELOAR; Michel François VALSTAR
- Web page: https://patentgrep.com/patent/US20240265733A1

## CPC

- G06V 40/174: {Facial expression recognition}
- G06V 40/176: {Dynamic expression}
- G06F 18/2413: based on distances to training or reference patterns
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/73: using feature-based methods
- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06V 20/44: {Event detection}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/30201: Face
- G06T 2207/30204: Marker
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A computer-implemented method for generating feedback for a user in performing personal care includes receiving a sequence of images of a face of a user and a personal care implement. The method also includes determining one or more implement parameters by tracking the personal care implement using the sequence of images. The method also includes determining one or more facial apparent emotional expressions of the user using the sequence of images. The method also includes analysing a personal care performance based on i) the one or more implement parameters and the one or more facial apparent emotional expressions of the user in a current personal care session and ii) the one or more implement parameters and the one or more facial apparent emotional expressions of the user from a previous personal care session, and determining one or more performance parameters or one or more personal care events.

## Claims

### Claim 1

A computer-implemented method for generating feedback for a user in performing personal care, the method comprising:
receiving a sequence of images of a face of a user and a personal care implement;
determining one or more implement parameters by tracking the personal care implement using the sequence of images;
determining one or more facial apparent emotional expressions of the user using the sequence of images;
analysing a personal care performance based on i) the one or more implement parameters and the one or more facial apparent emotional expressions of the user in a current personal care session and ii) the one or more implement parameters and the one or more facial apparent emotional expressions of the user from a previous personal care session, and determining one or more performance parameters or one or more personal care events based on the one or more implement parameters and the one or more facial apparent emotional expressions of the user;
generating feedback to the user based on (i) the one or more performance parameters or one or more personal care events and (ii) the personal care performance, the feedback to improve personal care and associated health of the user; and
determining a difference in one or more performance parameters associated with the current personal care session and one or more corresponding performance parameters associated with the previous personal care session, wherein the feedback provides an indication of the difference.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more facial apparent emotional expressions are any of: confidence, enjoyment, pain, frustration, confusion and happiness.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein tracking the personal care implement comprises tracking a position and an orientation of the personal care implement.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein tracking the position and orientation of the personal care implement comprises tracking the position and orientation of the personal care implement relative to a head of the user.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein tracking the personal care implement comprises tracking a fiducial marker coupled to an end of the personal care implement.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein:
the fiducial marker forms an integral part of the personal care implement; or
the fiducial marker comprises an attachment coupled to the personal care implement.

### Claim 7 (depends on 5)

The computer-implemented method of claim 5, wherein the fiducial marker comprises a substantially spherical marker having a plurality of coloured quadrants disposed around a longitudinal axis corresponding to a longitudinal axis of the personal care implement.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more implement parameters include one or more of: implement position; implement orientation; linear velocity; angular velocity; linear acceleration; angular acceleration; and implement path.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising:
identifying a suitable treatment in accordance with a detected performance parameter or personal care event;
wherein generating the feedback comprises generating instructions to the user to perform the suitable treatment.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the personal care comprises any of: a toothcare activity; a skincare activity; a hair care activity; and a shaving activity.

### Claim 11 (depends on 1)

A computer program product comprising computer readable instructions which when executed on a computer, causes the computer to perform the method of claim 1.

### Claim 12 (depends on 1)

A personal care monitoring system comprising a processor configured to perform the method of claim 1.

### Claim 13

A computer-implemented method comprising:
receiving a first image of a face of a user and a personal care implement;
receiving a second image of the face of the user and the personal care implement after receiving the first image;
determining a first implement parameter by tracking the personal care implement using the first image and the second image;
determine a first facial apparent emotional expression of the user using the first image and the second image;
receiving a third image of the face of the user and the personal care implement after receiving the second image;
receiving a fourth image of the face of the user and the personal care implement after receiving the third image;
determining a second implement parameter by tracking the personal care implement using the third image and the fourth image;
determine a second facial apparent emotional expression of the user using the third image and the fourth image;
determine a first performance parameter based on the first implement parameter and the first facial apparent emotional expression;
determine a second performance parameter based on the second implement parameter and the second facial apparent emotional expression; and
determine a difference between the first performance parameter and the second performance parameter.

### Claim 14 (depends on 13)

The method claim 13, further comprising generating feedback for the user based on the difference.

### Claim 15 (depends on 13)

The method of claim 13, further comprising identifying a suitable treatment based on at least one of the first performance parameter or the second performance parameter;
wherein generating the feedback comprises generating instructions to the user to perform the suitable treatment.

### Claim 16 (depends on 13)

The method of claim 13, wherein tracking the personal care implement comprises tracking a position and an orientation of the personal care implement.

### Claim 17 (depends on 13)

The method of claim 13, wherein tracking the personal care implement comprises tracking a fiducial marker coupled to an end of the personal care implement.

### Claim 18 (depends on 17)

The method of claim 17, wherein:
the fiducial marker forms an integral part of the personal care implement; or
the fiducial marker comprises an attachment coupled to the personal care implement.

### Claim 19 (depends on 17)

The method of claim 17, wherein the fiducial marker comprises a substantially spherical marker having a plurality of coloured quadrants disposed around a longitudinal axis corresponding to a longitudinal axis of the personal care implement.

### Claim 20 (depends on 13)

The method of claim 13, wherein the first implement parameter includes at least one of: an implement position; an implement orientation; a linear velocity; an angular velocity; a linear acceleration; an angular acceleration, or an implement path.
