# US 2021/0387070 A1 · Method and apparatus for baseball strategy planning based on reinforcement learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0387070 A1
- Document ID: US20210387070A1
- Status: Published application
- Kind: A1
- Application number: 16/942,758
- Filed: 2020-07-29
- Published: 2021-12-16
- Earliest claimed: 2020-06-16
- Assignees: National Tsing Hua University
- Applicants: National Tsing Hua University
- Inventors: Wei-Yu Chiu; Song-Jen Chen
- Web page: https://patentgrep.com/patent/US20210387070A1

## CPC

- A63B 71/0605: {Decision makers and devices using detection means facilitating arbitration}
- A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user}
- A63F 13/812: Ball games, e.g. soccer or baseball
- G06N 20/00: Machine learning
- G06Q 10/06375: {Prediction of business process outcome or impact based on a proposed change}
- G06Q 99/00: Subject matter not provided for in other groups of this subclass
- A63B 2102/18: Baseball, rounders or similar games
- A63F 2003/00034: {Baseball}
- A63F 2011/0093: {characterised by the game theory or winning strategy}
- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A method and an apparatus for baseball strategy planning based on reinforcement learning are provided. The method includes steps below. Historical data of innings in past games of a team is collected. Multiple game states, multiple offensive and defensive actions, and multiple rewards corresponding to multiple offensive and defensive results are defined based on multiple offensive and defensive processes occurring during the game, and are used to establish a Q table. The Q table is updated according to multiple combinations of the game state, the offensive and defensive action, and the offensive and defensive result recorded in the historical data. According to a current game state, Q values of all offensive and defensive actions executable in the current game state recorded in the updated Q table are sorted, and the offensive and defensive action suitable for being executed in the current game state is recommended according to a sorting result.

## Claims

### Claim 1

A baseball strategy planning method based on reinforcement learning, adapted for an electronic apparatus having a processor, the method comprising:
collecting historical data of multiple innings in past games of a team;
defining multiple game states, multiple offensive and defensive actions, and multiple rewards corresponding to multiple offensive and defensive results according to multiple offensive and defensive processes occurring during the game, and using the game states, the offensive and defensive actions, and the rewards to establish a Q table;
updating the Q table according to multiple combinations of the game state, the offensive and defensive action, and the offensive and defensive result recorded in the historical data; and
sorting, according to a current game state, Q values of all offensive and defensive actions executable in the game state recorded in the updated Q table, and recommending the offensive and defensive action suitable for being executed in the game state according to a sorting result.

### Claim 2 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the game state comprises a base occupation status, a number of outs, or a strike/ball count.

### Claim 3 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the offensive and defensive action comprises multiple pitch types of a pitcher and multiple hitting actions of a hitter, and the hitting actions comprise a bunt, a hit, a sacrifice fly, or no swing.

### Claim 4 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the rewards corresponding to the offensive and defensive results comprise negative rewards representing losing a score, a base being advanced, and hitting by a hitter on a defensive side, a zero reward representing not losing a score on the defensive side, and positive rewards representing not being hit by the hitter, and striking out or putting out the hitter on the defensive side.

### Claim 5 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the rewards corresponding to the offensive and defensive results comprise positive rewards representing scoring, advancing a base, and hitting a ball on an offensive side, a zero reward representing not scoring on the offensive side, and negative rewards representing a hitter missing a ball, and being stricken out or put out on the offensive side.

### Claim 6 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the step of updating the Q table according to the multiple combinations of the game state, the offensive and defensive action, and the offensive and defensive result recorded in the historical data comprises:
for each of the game states, searching for an offensive and defensive result and a new game state obtained after executing multiple offensive and defensive actions in the game state recorded in the historical data, and using the offensive and defensive result and the new game state to calculate a reward obtained by executing each of the offensive and defensive actions in the game state; and
updating, by using the calculated rewards and Q values of executing multiple offensive and defensive actions in the new game state, a Q value of executing each of the offensive and defensive actions in the game state in the Q table.

### Claim 7 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein after the step of recommending the offensive and defensive action suitable for being executed in the game state according to the sorting result, the method further comprises:
receiving a selection of the recommended offensive and defensive action;
calculating a reward obtained by executing the selected offensive and defensive action in the game state according to an offensive and defensive result and a new game state obtained after executing the selected offensive and defensive action; and
updating, by using the calculated reward and Q values of executing multiple offensive and defensive actions in the new game state, a Q value of executing the selected offensive and defensive action in the game state in the Q table.

### Claim 8 (depends on 1)

The baseball strategy planning method based on reinforcement learning according to claim 1, wherein the Q values of all offensive and defensive actions executable in the game state comprise Q values of executing the offensive and defensive actions by multiple players capable of executing the offensive and defensive actions.

### Claim 9

A baseball strategy planning apparatus based on reinforcement learning, comprising:
a data retrieval device connected an external device;
a storage device storing a computer program; and
a processor coupled to the data retrieval device and the storage device and configured to load and execute the computer program to:
collect, by the data retrieval device, historical data of multiple innings in past games of a team from the external device;
define multiple game states, multiple offensive and defensive actions, and multiple rewards corresponding to multiple offensive and defensive results according to multiple offensive and defensive processes occurring during the game, and using the game states, the offensive and defensive actions, and the rewards to establish a Q table;
update the Q table according to multiple combinations of the game state, the offensive and defensive action, and the offensive and defensive result recorded in the historical data; and
sort, according to a current game state, Q values of all offensive and defensive actions executable in the game state recorded in the updated Q table, and recommend the offensive and defensive action suitable for being executed in the game state according to a sorting result.

### Claim 10 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the game state comprises a base occupation status, a number of outs, or a strike/ball count.

### Claim 11 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the offensive and defensive action comprises multiple pitch types of a pitcher and multiple hitting actions of a hitter, and the hitting actions comprise a bunt, a hit, a sacrifice fly, or no swing.

### Claim 12 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the rewards corresponding to the offensive and defensive results comprise negative rewards representing losing a score, a base being advanced, and hitting by a hitter on a defensive side, a zero reward representing not losing a score on the defensive side, and positive rewards representing not being hit by the hitter, and striking out or putting out the hitter on the defensive side.

### Claim 13 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the rewards corresponding to the offensive and defensive results comprise positive rewards representing scoring, advancing a base, and hitting a ball on an offensive side, a zero reward representing not scoring on the offensive side, and negative rewards representing a hitter missing a ball, and being stricken out or put out on the offensive side.

### Claim 14 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the processor is configured to:
for each of the game states, search for an offensive and defensive result and a new game state obtained after executing multiple offensive and defensive actions in the game state recorded in the historical data, and use the offensive and defensive result and the new game state to calculate a reward obtained by executing each of the offensive and defensive actions in the game state; and
update, by using the calculated rewards and Q values of executing multiple offensive and defensive actions in the new game state, a Q value of executing each of the offensive and defensive actions in the game state in the Q table.

### Claim 15 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the processor is further configured to:
receive a selection of the recommended offensive and defensive action;
calculate a reward obtained by executing the selected offensive and defensive action in the game state according to an offensive and defensive result and a new game state obtained after executing the selected offensive and defensive action; and
update, by using the calculated reward and Q values of executing multiple offensive and defensive actions in the new game state, a Q value of executing the selected offensive and defensive action in the game state in the Q table.

### Claim 16 (depends on 9)

The baseball strategy planning apparatus based on reinforcement learning according to claim 9, wherein the Q values of all offensive and defensive actions executable in the game state comprise Q values of executing the offensive and defensive actions by multiple players capable of executing the offensive and defensive actions.
