# US 2023/0069282 A1 · Service request processing utilizing modified versions of images of computing devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0069282 A1
- Document ID: US20230069282A1
- Status: Published application
- Kind: A1
- Application number: 17/462,235
- Filed: 2021-08-31
- Published: 2023-03-02
- Earliest claimed: 2021-08-31
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: An Chung; Jimmy Henry Wiggers; Ravi Shukla; Jeffrey Scott Vah
- Web page: https://patentgrep.com/patent/US20230069282A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06T 3/00: Geometric image transformations in the plane of the image
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/82: using neural networks
- G06V 20/60: Type of objects
- G06T 2210/12: Bounding box
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An apparatus comprises a processing device configured to obtain a first image, to detect one or more designated types of objects in the first image, at least a given one of the designated types of objects comprising at least a given portion of a computing device associated with a service request and, responsive to detecting the given one of the designated types of objects, to identify features of the given portion of the computing device in the first image. The processing device is also configured to perform an augmentation of the identified features of the given portion of the computing device to generate a second image, the second image comprising a modified version of the first image that contains the given portion of the computing device and excludes one or more other portions of the first image, and to process the service request utilizing the second image.

## Claims

### Claim 1

An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to perform steps of:
obtaining a first image;
detecting one or more designated types of objects in the first image, at least a given one of the one or more designated types of objects comprising at least a given portion of a computing device associated with a service request;
responsive to detecting the given one of the one or more designated types of objects in the first image, identifying features of the given portion of the computing device in the first image;
performing an augmentation of the identified features of the given portion of the computing device to generate a second image, the second image comprising a modified version of the first image that contains the given portion of the computing device and excludes one or more other portions of the first image; and
processing the service request utilizing the second image.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein detecting the one or more designated types of objects in the first image utilizes an object detection machine learning model.

### Claim 3 (depends on 2)

The apparatus of claim 2 wherein the object detection machine learning model outputs a first value if the given one of the one or more designated types of objects are detected in the image and outputs a second value if the given one of the one or more designated types of objects are not detected in the image.

### Claim 4 (depends on 1)

The apparatus of claim 1 wherein the identified features of the given portion of the computing device comprises corners of the given portion of the computing device, and wherein identifying the features of the given portion of the computing device in first image utilizes a corners prediction machine learning model.

### Claim 5 (depends on 4)

The apparatus of claim 4 wherein the corners prediction machine learning model utilizes object detection to make predictions for each of a set of corner classes and outputs one or more predictions of locations of the corners of the given portion of the computing device in a designated coordinate format.

### Claim 6 (depends on 5)

The apparatus of claim 5 wherein the designated coordinate format comprises a normalized coordinate format.

### Claim 7 (depends on 5)

The apparatus of claim 5 wherein the designated coordinate format comprises a normalized coordinate format that specifies center coordinates of the given portion of the computing device in the first image and a bounding box width and height of the given portion of the computing device in the first image.

### Claim 8 (depends on 5)

The apparatus of claim 5 wherein the set of corner classes comprises four corner classes and each of the one or more predictions comprises coordinates of locations of four corners of the given portion of the computing device in the first image.

### Claim 9 (depends on 8)

The apparatus of claim 8 wherein performing the augmentation of the identified features of the given portion of the computing device to generate the second image comprises performing a trapezoidal augmentation that stretches each of the four corners of the given portion of the computing device in the first image to a predefined image resolution.

### Claim 10 (depends on 9)

The apparatus of claim 9 wherein the trapezoidal augmentation takes as input a first list comprising the predicted coordinates of the locations of the four corners of the given portion of the computing device in the first image and a second list comprising the predefined image resolution, and applies a transformation to the first list to produce the second image with the predefined image resolution specified in the second list.

### Claim 11 (depends on 1)

The apparatus of claim 1 wherein the given portion of the computing device comprises a screen of the computing device.

### Claim 12 (depends on 1)

The apparatus of claim 1 wherein the given portion of the computing device comprises a set of ports of the computing device.

### Claim 13 (depends on 1)

The apparatus of claim 1 wherein processing the service request utilizing the second image comprises classifying a type of damage associated with at least one component of the computing device based at least in part on one or more visual characteristics of the given portion of the computing device in the second image.

### Claim 14 (depends on 1)

The apparatus of claim 1 wherein processing the service request utilizing the second image comprises classifying a type of issue encountered on the computing device based at least in part on one or more visual characteristics of the given portion of the computing device in the second image.

### Claim 15

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform steps of:
obtaining a first image;
detecting one or more designated types of objects in the first image, at least a given one of the one or more designated types of objects comprising at least a given portion of a computing device associated with a service request;
responsive to detecting the given one of the one or more designated types of objects in the first image, identifying features of the given portion of the computing device in the first image;
performing an augmentation of the identified features of the given portion of the computing device to generate a second image, the second image comprising a modified version of the first image that contains the given portion of the computing device and excludes one or more other portions of the first image; and
processing the service request utilizing the second image.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein the identified features of the given portion of the computing device comprises corners of the given portion of the computing device, and wherein identifying the features of the given portion of the computing device in first image utilizes a corners prediction machine learning model to make predictions for each of a set of corner classes and outputs one or more predictions of locations of the corners of the given portion of the computing device in a designated coordinate format.

### Claim 17 (depends on 16)

The computer program product of claim 16 wherein the set of corner classes comprises four corner classes and each of the one or more predictions comprises coordinates of locations of four corners of the given portion of the computing device in the first image, and wherein performing the augmentation of the identified features of the given portion of the computing device to generate the second image comprises performing a trapezoidal augmentation that stretches each of the four corners of the given portion of the computing device in the first image to a predefined image resolution.

### Claim 18

A method comprising steps of:
obtaining a first image;
detecting one or more designated types of objects in the first image, at least a given one of the one or more designated types of objects comprising at least a given portion of a computing device associated with a service request;
responsive to detecting the given one of the one or more designated types of objects in the first image, identifying features of the given portion of the computing device in the first image;
performing an augmentation of the identified features of the given portion of the computing device to generate a second image, the second image comprising a modified version of the first image that contains the given portion of the computing device and excludes one or more other portions of the first image; and
processing the service request utilizing the second image;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

### Claim 19 (depends on 18)

The method of claim 18 wherein the identified features of the given portion of the computing device comprises corners of the given portion of the computing device, and wherein identifying the features of the given portion of the computing device in first image utilizes a corners prediction machine learning model to make predictions for each of a set of corner classes and outputs one or more predictions of locations of the corners of the given portion of the computing device in a designated coordinate format.

### Claim 20 (depends on 19)

The method of claim 19 wherein the set of corner classes comprises four corner classes and each of the one or more predictions comprises coordinates of locations of four corners of the given portion of the computing device in the first image, and wherein performing the augmentation of the identified features of the given portion of the computing device to generate the second image comprises performing a trapezoidal augmentation that stretches each of the four corners of the given portion of the computing device in the first image to a predefined image resolution.
