# US 2018/0322129 A1 · Identifying resources in user interfaces for feedback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322129 A1
- Document ID: US20180322129A1
- Status: Published application
- Kind: A1
- Application number: 15/632,395
- Filed: 2017-06-26
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: David A. Kern; Elliot N. Munger; Jared M. Green; Sven Merget; Bill L. Smith
- Web page: https://patentgrep.com/patent/US20180322129A1

## CPC

- G06F 16/3322: {using system suggestions (G06F16/3325 takes precedence)}
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 9/454: {Multi-language systems; Localisation; Internationalisation}
- G06F 16/144: {Query formulation}
- G06F 16/1734: {Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs}
- G06F 16/2425: {Iterative querying; Query formulation based on the results of a preceding query}
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/953: Querying, e.g. by the use of web search engines
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments relate to enabling users to provide feedback on resources displayed in user interfaces. A feedback tool is executed to allow a user to specify a resource displayed in a user interface. Context metadata about the user interface is captured and stored by the feedback tool. The context metadata corresponds to runtime activity such as loading executable modules, rendering the user interface, handling windowing/input events, etc. The tool is used to specify a resource displayed in the user interface. This triggers capture of the specified resource and the runtime context metadata. The resource and context metadata are searched for in a database of known resources and their respective properties and identifiers. If a match is found, then the source-level identity of the resource is known. The feedback is associated with the resource and used to directly supplement or modify the resource in a source code resource file.

## Claims

### Claim 1

A method implemented by one or more computing devices, the method comprising:
providing a feedback tool comprising an executable program configured to executed on user devices, each of the user devices having executable user interfaces, wherein each executable user interface comprises a respective plurality of resource references that reference resource strings in resource files on the user devices, and wherein when a user interface is executed a setting determines which resource file will be selected to supply resource strings according to the resource references in the user interface;
executing the feedback tool on the user devices, wherein the feedback tool, when executed on a user device, comprises a feedback function activatable responsive to user input;
responding to a first user input activating the feedback function by automatically capturing context metadata, the capturing comprising automatically monitoring a runtime execution environment and the metadata corresponding to actions of the runtime execution environment that are performed to execute a user interface;
receiving a second user input selecting a resource string displayed in the user interface, and based on the second user input copying the resource string, and storing the copy of the resource string in association with the context metadata; and
searching for the resource string in a database of resource strings, the searching qualified by one or more items in the context metadata, the resource strings obtained from the resource files; and
determining a resource identifier of the resource string based on the searching.

### Claim 2 (depends on 1)

A method according to claim 1, wherein the resource identifier corresponds to a resource reference in a resource file.

### Claim 3 (depends on 1)

A method according to claim 1, further comprising receiving feedback text inputted by a user based on the second user input, and storing the feedback text in association with the determined resource identifier.

### Claim 4 (depends on 3)

A method according to claim 3, wherein the resource strings comprise translations, and wherein feedback text comprises a candidate translation of the selected resource string.

### Claim 5 (depends on 1)

A method according to claim 1, wherein the user interface is displayed by executing one or more executable files compiled from source code comprising the determined resource identifier, wherein the executable files do not comprise the resource string, and wherein the resource string is loaded for display in the user interface based (i) the presence of the resource identifier in one of the one or more executable files and (ii) the presence of the resource string in one of the resource files.

### Claim 6 (depends on 1)

A method according to claim 1, further comprising a plurality of feedbacks inputted from a plurality of the feedback tools executed on a plurality of respective user devices, and constructing a new resource file comprised of the feedbacks and resource identifiers.

### Claim 7 (depends on 1)

A method according to claim 1, further comprising, based on the first and/or second user input, capturing at least a partial screenshot that includes the user interface, and storing the screenshot in association with the feedback.

### Claim 8 (depends on 1)

A method according to claim 1, further comprising determining an authorization level of a user identity of a user that inputted the feedback, and based on the authorization level automatically updating a resource file to include the feedback.

### Claim 9 (depends on 1)

A method according to claim 1, further comprising receiving votes from the user device directed to the feedback, and based on a count of the votes determining to automatically update a resource file with the feedback.

### Claim 10 (depends on 1)

A method according to claim 1, wherein the context metadata comprises one or more of: a name of a file or module loaded to execute the user interface, a culture or locale or language setting, a window or input event, a process name or identifier, a version number, or a bounding box.

### Claim 11

A computing device comprising:
processing hardware;
storage hardware storing an operating system, compiled applications comprised of code executable by the processing hardware, and an executable feedback tool;
an application comprising code to display a user interface comprised of resource strings;
the feedback tool comprising code configured to be executed by the processing hardware, the feedback tool, when executed, performing a process, while the application is executing, comprising:
receiving an indication of a resource string selected from the displayed user interface;
receiving feedback inputted in association with selection of the resource string;
capturing and storing context metadata corresponding to executing the application and/or loading the application for execution; and
inferring a source code identity of the resource string by searching, based on at least a portion of the context metadata, for the resource string in a store of resource strings prepared prior to execution of the application and the feedback tool.

### Claim 12 (depends on 11)

A computing device according to claim 11, wherein the feedback tool comprises an executable program separate from the applications, wherein the feedback tool is configured to enable feedback to be directed to any resource strings displayed by the applications executing concurrently with the feedback tool, and wherein the store of resource strings comprises identifiers of the respective resource strings and the inferred source code identity of the resource string comprises an identifier obtained from the store.

### Claim 13 (depends on 11)

A computing device according to claim 11, wherein the feedback tool comprises a feedback user interface configured to enable feedback to be enabled and disabled, and wherein the feedback tool comprises a feedback input component that enables text to be interactively inputted to the feedback tool and associated with a corresponding resource string to which the text is directed.

### Claim 14 (depends on 11)

A computing device according to claim 11, wherein the feedback tool is configured to associate feedback with resource strings of any arbitrary applications that execute on the computing device.

### Claim 15 (depends on 14)

A computing device according to claim 14, wherein the storage hardware stores a feedback relay component configured to send feedback and associations with respective resource strings via a network to a feedback service that collects feedback from a plurality of instances of the feedback tool on respective user computing devices and automatically incorporates the feedback into resource files configured for use with versions of applications from which the resource strings were obtained.

### Claim 16

A method performed by one or more computing devices, the method comprising:
receiving feedback items, each feedback item respectively comprising feedback strings, a resource string, and context metadata, each resource string having been captured by a feedback tool from an application other than the feedback tool, each context metadata captured from a runtime environment managing execution of the feedback tool and the application, and each feedback item's feedback string being associated with the feedback item's resource string based on selection, by the feedback tool, of the resource string from a user interface displayed by the application;
identifying source code level unique resource identifiers of the resource strings by searching a table for strings that match the resource strings and that at least partly match the context metadata, the table comprised of strings obtained from resource files of software projects, the table further comprised of unique resource identifiers of the respective strings in the table; and
storing associations between the feedback strings and the respectively corresponding identified unique resource identifiers.

### Claim 17 (depends on 16)

A method according to claim 16, wherein the context metadata comprises one or more of: a file name, a file path, a resource path, an environment variable, a bounding box, a process name, and a version number.

### Claim 18 (depends on 16)

A method according to claim 16, wherein the feedback tool displays, responsive to an input corresponding to providing feedback for a specific displayed resource string, a user interface comprising: a control to receive text inputted for a feedback string, a control to set a property of a feedback item, a display area that displays the corresponding resource string.

### Claim 19 (depends on 16)

A method according to claim 16, further comprising graphically highlighting a resource string or a display region thereof responsive to an interactive selection of the resource string or the display region.

### Claim 20 (depends on 16)

A method according to claim 16, further comprising automatically incorporating the feedback strings into a resource file stored on the computing device, the resource file configured to provide resource strings to an application such that the application incorporates, at runtime, the resource strings from the resource file, wherein a resource string displayed by the application from the resource file corresponds to a resource string selected for feedback by the feedback tool.