# US 2023/0069442 A1 · Method and system for fashion attribute detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0069442 A1
- Document ID: US20230069442A1
- Status: Published application
- Kind: A1
- Application number: 17/810,468
- Filed: 2022-07-01
- Published: 2023-03-02
- Earliest claimed: 2021-07-15
- Assignees: Tata Consultancy Services Limited
- Applicants: Tata Consultancy Services Limited
- Inventors: JAYAVARDHANA RAMA GUBBI LAKSHMINARASIMHA; GAURAB BHATTACHARYA; NIKHIL KILARI; BAGYALAKSHMI VASUDEVAN; BALAMURALIDHAR PURUSHOTHAMAN
- Web page: https://patentgrep.com/patent/US20230069442A1

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Traditional systems used for fashion attribute detection struggle to generate accurate predictions due to presence of large intra-class and relatively small inter-class variations in data related to the fashion attributes. The disclosure herein generally relates to image processing, and, more particularly, to a method and system for fashion attribute detection. The method proposes F-AttNet, an attribute extraction network to leverage the performance of fine-grained localized fashion attribute recognition. F-AttNet comprises Attentive Multi-scale Feature Encoder (AMF) blocks that encapsulate multi-scale fine-grained attribute information upon adaptive recalibration of channel weights. F-AttNet is designed by hierarchically stacking the AMF encoders to extract deep fine-grained information across multiple scales. A data model used by F-AttNet is trained using a novel γ-variant focal loss function for addressing the class imbalance problem by penalizing wrongly classified examples and incorporating separate importance to positive and negative instances.

## Claims

### Claim 1

A processor implemented method of fashion feature extraction, comprising:
collecting at least one image as input, via one or more hardware processors; and
processing the at least one image using a feature extraction network comprising a plurality of Attentive Multi-scale Feature (AMF) blocks implemented the via one or more hardware processors, using a data model, wherein processing the at least one image by the plurality of AMF blocks comprising:
extracting a plurality of features from the at least one image, by a first subnetwork of the AMF blocks;
identifying and extracting features belonging to different scales, from among the plurality of features extracted from the at least one image, by a second subnetwork of the AMF blocks, wherein the second subnetwork applies a convolution operation on the plurality of features;
assigning a unique weightage to each of a plurality of channels used for the convolution operation, based on a determined importance of each of the features belonging to the different scales, by a third subnetwork of the AMF blocks;
determining a rank for each of the extracted features belonging to the different scales, based on the unique weightage of corresponding channel, by the third subnetwork; and
generating one or more recommendations of the extracted features based on the determined rank of each of the extracted features.

### Claim 2 (depends on 1)

The method of claim 1, wherein extracting the plurality of features from the at least one image comprises concatenating a plurality of feature representations obtained from the at least one image by applying the convolution operation on the at least one image.

### Claim 3 (depends on 1)

The method of claim 1, wherein identifying and extracting features belonging to different scales by applying the convolution operation on the plurality of features comprises:
passing the features extracted from the at least one image through a plurality of paths;
encapsulating a plurality of multi scale fine-grained feature maps by applying the convolution operation on the features in each of the plurality of paths, wherein a stride length in each path is adaptively varied;
obtaining a plurality of fine-tuned features from the multi scale fine-grained feature maps, as an aggregated response; and
obtaining a summation of the aggregated response and the features extracted from the at least one image as the features belonging to the different scales.

### Claim 4 (depends on 1)

The method of claim 1, wherein an accuracy of the generated one or more recommendations of the extracted features is verified using a γ-variant focal loss function.

### Claim 5 (depends on 1)

The method of claim 1, wherein the data model is trained to perform the feature attribute extraction, further comprising:
determining value of a plurality of micro-metrics for a dataset, wherein the plurality of micro-metrics comprises micro-precision, micro-recall, and micro-F1 score;
determining an optimum value of each of a plurality of hyper parameters of a γ-variant focal loss function, based on the determined values of the plurality of micro-metrics; and
training a data model using the determined optimum value of the plurality of hyper parameters of the γ-variant focal loss function.

### Claim 6

A system for fashion feature extraction, comprising:
one or more hardware processors;
a communication interface; and
a memory storing a plurality of instructions, wherein the plurality of instructions cause the one or more hardware processors to:
collect at least one image as input; and
process the at least one image using a feature extraction network comprising a plurality of Attentive Multi-scale Feature (AMF) blocks implemented the via one or more hardware processors, using a data model, wherein processing the at least one image by the plurality of AMF blocks comprising:
extracting a plurality of features from the at least one image, by a first subnetwork of the AMF blocks;
identifying and extracting features belonging to different scales, from among the plurality of features extracted from the at least one image, by a second subnetwork of the AMF blocks, wherein the second subnetwork applies a convolution operation on the plurality of features;
assigning a unique weightage to each of a plurality of channels used for the convolution operation, based on a determined importance of each of the features belonging to the different scales, by a third subnetwork of the AMF blocks;
determining a rank for each of the extracted features belonging to the different scales, based on the unique weightage of corresponding channel, by the third subnetwork; and
generating one or more recommendations of the extracted features based on the determined rank of each of the extracted features.

### Claim 7 (depends on 6)

The system of claim 6, wherein the system extracts the plurality of features from the at least one image by concatenating a plurality of feature representations obtained from the at least one image by applying a convolution operation on the at least one image.

### Claim 8 (depends on 6)

The system of claim 6, wherein the system identifies and extracts features belonging to different scales by applying the convolution operation on the plurality of features, by:
passing the features extracted from the at least one image through a plurality of paths;
encapsulating a plurality of multi scale fine-grained feature maps by applying the convolution operation on the features in each of the plurality of paths, wherein a stride length in each path is adaptively varied;
obtaining a plurality of fine-tuned features from the multi scale fine-grained feature maps, as an aggregated response; and
obtaining a summation of the aggregated response and the features extracted from the at least one image as the features belonging to different scales.

### Claim 9 (depends on 6)

The system of claim 6, wherein the system verifies accuracy of the generated one or more recommendations of the extracted features using a γ-variant focal loss function.

### Claim 10 (depends on 6)

The system of claim 6, wherein the system trains the data model to perform the feature attribute extraction, by:
determining value of a plurality of micro-metrics for a dataset, wherein the plurality of micro-metrics comprises micro-precision, micro-recall, and micro-F1 score;
determining an optimum value of each of a plurality of hyper parameters of a γ-variant focal loss function, based on the determined values of the plurality of micro-metrics; and
training a data model using the determined optimum value of the plurality of hyper parameters of the γ-variant focal loss function.

### Claim 11

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
collecting at least one image as input, via one or more hardware processors;
processing the at least one image using a feature extraction network comprising a plurality of Attentive Multi-scale Feature (AMF) blocks implemented the using a data model, wherein processing the at least one image by the plurality of AMF blocks comprising:
extracting a plurality of features from the at least one image, by a first subnetwork of the AMF blocks;
identifying and extracting features belonging to different scales, from among the plurality of features extracted from the at least one image, by a second subnetwork of the AMF blocks, wherein the second subnetwork applies a convolution operation on the plurality of features;
assigning a unique weightage to each of a plurality of channels used for the convolution operation, based on a determined importance of each of the features belonging to the different scales, by a third subnetwork of the AMF blocks;
determining a rank for each of the extracted features belonging to the different scales, based on the unique weightage of corresponding channel, by the third subnetwork; and
generating one or more recommendations of the extracted features based on the determined rank of each of the extracted features.

### Claim 12 (depends on 11)

The one or more non-transitory machine-readable information storage mediums of claim 11, wherein extracting the plurality of features from the at least one image comprises concatenating a plurality of feature representations obtained from the at least one image by applying the convolution operation on the at least one image.

### Claim 13 (depends on 11)

The one or more non-transitory machine-readable information storage mediums of claim 11, wherein identifying and extracting features belonging to different scales by applying the convolution operation on the plurality of features comprises:
passing the features extracted from the at least one image through a plurality of paths;
encapsulating a plurality of multi scale fine-grained feature maps by applying the convolution operation on the features in each of the plurality of paths, wherein a stride length in each path is adaptively varied;
obtaining a plurality of fine-tuned features from the multi scale fine-grained feature maps, as an aggregated response; and
obtaining a summation of the aggregated response and the features extracted from the at least one image as the features belonging to the different scales.

### Claim 14 (depends on 11)

The one or more non-transitory machine-readable information storage mediums of claim 11, wherein an accuracy of the generated one or more recommendations of the extracted features is verified using a γ-variant focal loss function.

### Claim 15 (depends on 11)

The one or more non-transitory machine-readable information storage mediums of claim 11, wherein the data model is trained to perform the feature attribute extraction, comprising:
determining value of a plurality of micro-metrics for a dataset, wherein the plurality of micro-metrics comprises micro-precision, micro-recall, and micro-F1 score;
determining an optimum value of each of a plurality of hyper parameters of a γ-variant focal loss function, based on the determined values of the plurality of micro-metrics; and
training a data model using the determined optimum value of the plurality of hyper parameters of the γ-variant focal loss function.
