# US 2023/0267250 A1 · Method of RF Analog Circuits Electronic Design Automation Based on GCN and Deep Reinforcement Learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267250 A1
- Document ID: US20230267250A1
- Status: Published application
- Kind: A1
- Application number: 17/653,880
- Filed: 2022-03-08
- Published: 2023-08-24
- Earliest claimed: 2022-02-24
- Assignees: Mitsubishi Electric Research Laboratories, Inc.
- Applicants: Mitsubishi Electric Research Laboratories, Inc.
- Inventors: Rui Ma; Mouhacine Benosman; Weidong Cao
- Web page: https://patentgrep.com/patent/US20230267250A1

## CPC

- G06F 30/323: Translation or migration, e.g. logic to logic, hardware description language [HDL] translation or netlist translation
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06F 30/373: Design optimisation
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/065: Analogue means
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- G06F 2117/12: Sizing, e.g. of transistors or gates
- G06F 2119/06: Power analysis or power optimisation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method is provided for generating device parameters of circuits using a pretrained reinforcement learning (RL) agent composed of a graph neural network (GNN) and a fully connected neural network (FCNN). The method is performed by steps including acquiring inputs with respect to a set of desired specifications or one desired specification of a circuit, device parameters, a fixed topology of the circuit and providing the inputs to the RL agent. The desired circuit description includes a graph modeling the topology of the circuit and device parameters of the circuit, and the desired specifications include gain, bandwidth, phase margin, power consumption, output power and power efficiency. The pretrained RL agent performs steps including transmitting an action selected from a set of actions to an environment module, updating the device parameters of the circuit according to the selected action using a data processor of the environment module, obtaining a current specification of the circuit by simulating a netlist of the circuit, acquiring a reward from the environment module, and generating the updated device parameters of the circuit.

## Claims

### Claim 1

A computer-implemented method for generating device parameters of circuits using a pretrained reinforcement learning (RL) agent composed of a graph neural network (GNN) and a fully connected neural network (FCNN), wherein the method uses a processor coupled with a memory storing instructions implementing the method, wherein the instructions, when executed by the processor, carry out at steps of the method, comprising:
acquiring, via an interface, inputs with respect to a set of desired specifications or one desired specification of a circuit, device parameters, a fixed topology of the circuit and providing the inputs to the RL agent, wherein the desired circuit description includes a graph modeling the topology of the circuit and device parameters of the circuit, wherein the desired specifications include gain, bandwidth, phase margin, power consumption, output power and power efficiency, wherein the pretrained RL agent is configured to perform steps of:
transmitting an action selected from a set of actions to an environment module;
updating the device parameters of the circuit with respect to a circuit desired specification according to the selected action using a data processor of the environment module, wherein the action changes each of current device parameters by a minimum unit value of each device parameter;
obtaining a current specification of the circuit by simulating a netlist of the circuit based on the updated device parameters using a circuit simulator of the environment module, wherein the environment module includes the netlist of the circuit;
acquiring a reward from the environment module, wherein the reward is computed based on a difference between the current specification and the desired specification, wherein the steps of the transmitting, updating, obtaining and acquiring are continued until the reward reaches to a threshold value or a number of steps reach a preset value; and
generating the updated device parameters of the circuit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the circuit is modeled by a graph G(V, E), wherein each node V is represented by a device, wherein an edge E represents a connection between devices.

### Claim 3 (depends on 1)

The method of claim 1, where in the power supply (VP), ground (VGND), and other DC bias voltages in the circuit topology are represented as extra nodes V.

### Claim 4 (depends on 1)

The method of claim 1, wherein the reward is calculated by a weighted sum of all desired specification by assigning different value of the weighting factor.

### Claim 5 (depends on 1)

The method of claim 1, wherein the Graph Neural Network can be implemented by different types such as Graph Convolutional Network or Grpah Attention Network (GNN).

### Claim 6 (depends on 1)

The method of claim 1, wherein the FCNN embeds the desired specifications of the circuit, and GNN embeds the circuit topology and parameters.

### Claim 7

A computer-implemented method for training a RL agent network generating device parameters of circuits, wherein the method uses a processor coupled with a memory storing instructions implementing the method, wherein the instructions, when executed by the processor, carry out at steps of the method, comprising:
acquiring inputs with respect to a desired circuit specification of a circuit, a device parameter, a fixed topology of the circuit, a final state corresponding to a maximum step, wherein the desired circuit specification includes a gain, bandwidth, phase margin, power consumption, output power and power efficiency, wherein the RL agent network is configured to perform steps of:
transmitting an action selected from a set of actions to an environment module;
updating the device parameters of the circuit with respect to a circuit specification according to the selected action using a data processor of the environment module, wherein the action changes each of current device parameters by a minimum unit value of each device parameter;
obtaining a current specification of the circuit by simulating a netlist of the circuit based on the updated device parameters using a circuit simulator of the environment module, wherein the environment module includes the netlist of the circuit;
acquiring a reward from the environment module, wherein the reword is computed based on a difference between the current specification and the desired specification, wherein the steps of the transmitting, updating, obtaining and acquiring are continued until the reward reaches to a threshold value or a number of steps reach a preset value; and
generating the satisfied updated device parameters via the interface.

### Claim 8 (depends on 7)

The method of claim 7, wherein the circuit is modeled by a graph G(V, E), wherein each node V is represented by a device, wherein an edge E represents a connection between devices.

### Claim 9 (depends on 7)

The method of claim 7, where in the power supply (VP), ground (VGND), and other DC bias voltages in the circuit topology are represented as extra nodes V.

### Claim 10 (depends on 7)

The method of claim 7, wherein the reward is calculated by a weighted sum of all desired specification by assigning different value of the weighting factor.

### Claim 11 (depends on 7)

The method of claim 7, wherein the Graph Neural Network can be implemented by different types such as Graph Convolutional Network or Grpah Attention Network (GNN).

### Claim 12 (depends on 7)

The method of claim 7, wherein the FCNN embeds the desired specifications of the circuit, and GNN embeds the circuit topology and parameters.
