# US 2019/0171604 A1 · System Having a Hybrid Threading Processor, a Hybrid Threading Fabric Having Configurable Computing Elements, and a Hybrid Interconnection Network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171604 A1
- Document ID: US20190171604A1
- Status: Published application
- Kind: A1
- Application number: 16/176,434
- Filed: 2018-10-31
- Published: 2019-06-06
- Earliest claimed: 2017-10-31
- Assignees: Micron Technology, Inc.
- Applicants: Micron Technology, Inc.
- Inventors: Tony M. Brewer
- Web page: https://patentgrep.com/patent/US20190171604A1

## CPC

- G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)}
- G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15})
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)}
- G06F 13/1684: {using multiple buses}
- G06F 13/4027: {using bus bridges (G06F13/4022 takes precedence)}
- G06F 15/7889: {Reconfigurable logic implemented as a co-processor (instruction execution using a coprocessor G06F9/3877)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Representative apparatus, method, and system embodiments are disclosed for configurable computing. In a representative embodiment, a system includes: a first, interconnection network; a processor coupled to the interconnection network; a host interface coupled to the interconnection network; and at least one configurable circuit cluster coupled to the interconnection network, the configurable circuit cluster including: a plurality of configurable circuits arranged in an array; a second, asynchronous packet network coupled to each configurable circuit of the plurality of configurable circuits of the array; a third, synchronous network coupled to each configurable circuit of the plurality of configurable circuits of the array; a memory interface circuit coupled to the asynchronous packet network and to the interconnection network; and a dispatch interface circuit coupled to the asynchronous packet network and to the interconnection network.

## Claims

### Claim 1

A system comprising:
a first, interconnection network;
a processor coupled to the interconnection network;
a host interface coupled to the interconnection network; and
at least one configurable circuit cluster coupled to the interconnection network, the configurable circuit cluster comprising:
a plurality of configurable circuits arranged in an array;
a second, asynchronous packet network coupled to each configurable circuit of the plurality of configurable circuits of the array;
a third, synchronous network coupled to each configurable circuit of the plurality of configurable circuits of the array;
a memory interface circuit coupled to the asynchronous packet network and to the interconnection network; and
a dispatch interface circuit coupled to the asynchronous packet network and to the interconnection network.

### Claim 2 (depends on 1)

The system of claim 1, wherein the interconnection network comprises:
a first plurality of crossbar switches having a Folded Clos configuration and a plurality of direct, mesh connections at interfaces with system endpoints.

### Claim 3 (depends on 2)

The system of claim 2, wherein the asynchronous packet network comprises:
a second plurality of crossbar switches, each crossbar switch coupled to at least one configurable circuit of the plurality of configurable circuits of the array and to another crossbar switch of the second plurality of crossbar switches.

### Claim 4 (depends on 3)

The system of claim 3, wherein the synchronous network comprises:
a plurality of direct point-to-point connections coupling adjacent configurable circuits of the array of the plurality of configurable circuits of the configurable circuit cluster.

### Claim 5 (depends on 1)

The system of claim 1, wherein each configurable circuit of the plurality of configurable circuits comprises:
a configurable computation circuit;
a control circuit coupled to the configurable computation circuit, the control circuit comprising: a memory control circuit; a thread control circuit; and a plurality of control registers;
a first memory circuit coupled to the configurable computation circuit;
a plurality of synchronous network inputs coupled to the configurable computation circuit and to the synchronous network;
a plurality of synchronous network outputs coupled to the configurable computation circuit and to the synchronous network;
an asynchronous network input queue coupled to the asynchronous packet network;
an asynchronous network output queue coupled to the asynchronous packet network;
a second, configuration memory circuit coupled to the configurable computation circuit, to the control circuitry, to the synchronous network inputs, and to the synchronous network outputs, the configuration memory circuit comprising:
a first, instruction memory storing a plurality of data path configuration instructions to configure a data path of the configurable computation circuit; and
a second, instruction and instruction index memory storing a plurality of spoke instructions and data path configuration instruction indices for selection of a master synchronous input of the synchronous network inputs.

### Claim 6 (depends on 5)

The system of claim 5, wherein the second, instruction and instruction index memory further stores a plurality of spoke instructions and data path configuration instruction indices for selection of a current data path configuration instruction for the configurable computation circuit.

### Claim 7 (depends on 5)

The system of claim 5, wherein the second, instruction and instruction index memory further stores a plurality of spoke instructions and data path configuration instruction indices for selection of a next data path configuration instruction for a next configurable computation circuit.

### Claim 8 (depends on 5)

The system of claim 5, wherein the second, instruction and instruction index memory further stores a plurality of spoke instructions and data path configuration instruction indices for selection of a synchronous network output of the plurality of synchronous network outputs.

### Claim 9 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a configuration memory multiplexer coupled to the first, instruction memory and to the second, instruction and instruction index memory.

### Claim 10 (depends on 9)

The system of claim 9, wherein when a selection input of the configuration memory multiplexer has a first setting, the current data path configuration instruction is selected using an instruction index from the second, instruction and instruction index memory.

### Claim 11 (depends on 10)

The system of claim 10, wherein when the selection input of the configuration memory multiplexer has a second setting different from the first setting, the current data path configuration instruction is selected using an instruction index from the master synchronous input.

### Claim 12 (depends on 5)

The system of claim 5, wherein the second, instruction and instruction index memory further stores a plurality of spoke instructions and data path configuration instruction indices for configuration of portions of the configurable circuit independently from the current data path instruction.

### Claim 13 (depends on 12)

The system of claim 12, wherein a selected spoke instruction and data path configuration instruction index of the plurality of spoke instructions and data path configuration instruction indices is selected according to a modulo spoke count.

### Claim 14 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a conditional logic circuit coupled to the configurable computing circuit.

### Claim 15 (depends on 14)

The system of claim 14, wherein depending upon an output from the configurable computing circuit, the conditional logic circuit is adapted to provide a first next instruction or instruction index for execution by a next configurable circuit or provide a different, second next instruction or instruction index for execution by the next configurable circuit.

### Claim 16 (depends on 14)

The system of claim 14, wherein depending upon an output from the configurable computing circuit, the conditional logic circuit is adapted to modify the next data path instruction index provided on a selected output of the plurality of synchronous network outputs.

### Claim 17 (depends on 14)

The system of claim 14, wherein depending upon an output from the configurable computing circuit, the conditional logic circuit is adapted to provide conditional branching by modifying the next data path instruction or next data path instruction index provided on a selected output of the plurality of synchronous network outputs.

### Claim 18 (depends on 14)

The system of claim 14, wherein the conditional logic circuit, when enabled, is adapted to provide conditional branching by ORing the output from the configurable computing circuit with the least significant bit of either the next data path instruction or the next data path instruction index to designate the next data path instruction or next data path instruction index.

### Claim 19 (depends on 5)

The system of claim 5, wherein the plurality of synchronous network inputs comprise:
a plurality of input registers coupled to a plurality of communication lines of a synchronous network; and
an input multiplexer coupled to the plurality of input registers and to the second, instruction and instruction index memory for selection of the master synchronous input.

### Claim 20 (depends on 5)

The system of claim 5, wherein the plurality of synchronous network outputs comprise:
a plurality of output registers coupled to a plurality of communication lines of the synchronous network; and
an output multiplexer coupled to the configurable computing circuit for selection of an output from the configurable computing circuit.

### Claim 21 (depends on 5)

The system of claim 5, wherein the plurality of synchronous network outputs comprise:
an asynchronous fabric state machine coupled to the asynchronous network input queue and to the asynchronous network output queue, the asynchronous fabric state machine adapted to decode an input data packet received from the asynchronous packet network and to assemble an output data packet for transmission on the asynchronous packet network.

### Claim 22 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a direct, pass-through connection between the plurality of input registers and the plurality of output registers.

### Claim 23 (depends on 22)

The system of claim 22, wherein the direct, pass-through connection of a first configurable circuit of the plurality configurable circuits provides a direct, point-to-point connection for data transmission on the synchronous network of data received from a second configurable circuit received of the plurality configurable circuits on the synchronous network to a third configurable circuit of the plurality configurable circuits.

### Claim 24 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
an arithmetic, logical and bit operation circuit adapted to perform at least one integer operation selected from the group consisting of: signed and unsigned addition, absolute value, negate, logical NOT, add and negate, subtraction A−B, reverse subtraction B−A, signed and unsigned greater than, signed and unsigned greater than or equal to, signed and unsigned less than, signed and unsigned less than or equal to, comparison of equal or not equal to, logical AND operation, logical OR operation, logical XOR operation, logical NAND operation, logical NOR operation, logical NOT XOR operation, logical AND NOT operation, logical OR NOT operation, and an interconversion between integer and floating point.

### Claim 25 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
an arithmetic, logical and bit operation circuit adapted to perform at least one floating point operation selected from the group consisting of: signed and unsigned addition, absolute value, negate, logical NOT, add and negate, subtraction A−B, reverse subtraction B−A, signed and unsigned greater than, signed and unsigned greater than or equal to, signed and unsigned less than, signed and unsigned less than or equal to, comparison of equal or not equal to, logical AND operation, logical OR operation, logical XOR operation, logical NAND operation, logical NOR operation, logical NOT XOR operation, logical AND NOT operation, logical OR NOT operation, an interconversion between integer and floating point, and combinations thereof.

### Claim 26 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a multiply and shift operation circuit adapted to perform at least one integer operation selected from the group consisting of: multiply, shift, pass an input, signed and unsigned multiply, signed and unsigned shift right, signed and unsigned shift left, bit order reversal, a permutation, an interconversion between integer and floating point, and combinations thereof.

### Claim 27 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a multiply and shift operation circuit adapted to perform at least floating point operation selected from the group consisting of: multiply, shift, pass an input, signed and unsigned multiply, signed and unsigned shift right, signed and unsigned shift left, bit order reversal, a permutation, an interconversion between integer and floating point, and combinations thereof.

### Claim 28 (depends on 5)

The system of claim 5, wherein the dispatch interface circuit is adapted to receive a work descriptor packet over the first interconnection network, and in response to the work descriptor packet, to generate one or more data and control packets to the plurality of configurable circuits to configure the plurality of configurable circuits for execution of a selected computation.

### Claim 29 (depends on 5)

The system of claim 5, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a flow control circuit coupled to the asynchronous network output queue, the flow control circuit adapted to generate a stop signal when a predetermined threshold has been reached in the asynchronous network output queue.

### Claim 30 (depends on 29)

The system of claim 29, wherein in response to the stop signal, each asynchronous network output queue stops outputting data packets on the asynchronous packet network.

### Claim 31 (depends on 29)

The system of claim 29, wherein in response to the stop signal, each configurable computation circuit stops executing upon completion of its current instruction.

### Claim 32 (depends on 5)

The system of claim 5, wherein a first plurality of configurable circuits of the array of a plurality of configurable circuits are coupled in a first predetermined sequence through the synchronous network to form a first synchronous domain; and wherein a second plurality of configurable circuits of the array of a plurality of configurable circuits are coupled in a second predetermined sequence through the synchronous network to form a second synchronous domain.

### Claim 33 (depends on 32)

The system of claim 32, wherein the first synchronous domain is adapted to generate a continuation message to the second synchronous domain transmitted through the asynchronous packet network.

### Claim 34 (depends on 32)

The system of claim 32, wherein the second synchronous domain is adapted to generate a completion message to the first synchronous domain transmitted through the asynchronous packet network.

### Claim 35 (depends on 5)

The system of claim 5, wherein the plurality of control registers store a completion table having a first, data completion count.

### Claim 36 (depends on 35)

The system of claim 35, wherein the plurality of control registers further store the completion table having a second, iteration count.

### Claim 37 (depends on 35)

The system of claim 35, wherein the plurality of control registers further store a loop table having a plurality of thread identifiers and, for each thread identifier, a next thread identifier for execution following execution of a current thread.

### Claim 38 (depends on 37)

The system of claim 37, wherein the plurality of control registers further store, in the loop table, an identification of a first iteration and an identification of a last iteration.

### Claim 39 (depends on 35)

The system of claim 35, wherein the control circuit is adapted to queue a thread for execution when, for its thread identifier, its completion count has decremented to zero and its thread identifier is the next thread.

### Claim 40 (depends on 35)

The system of claim 35, wherein the control circuit is adapted to queue a thread for execution when, for its thread identifier, its completion count indicates completion of any data dependencies.

### Claim 41 (depends on 35)

The system of claim 35, wherein the completion count indicates a predetermined number of completion messages to be received, per selected thread of a plurality of threads, prior to execution of the selected thread.

### Claim 42 (depends on 35)

The system of claim 35, wherein the plurality of control registers further store a completion table having a plurality of types of thread identifiers, with each type of thread identifier indicating a loop level for loop and nested loop execution.

### Claim 43 (depends on 35)

The system of claim 35, wherein the plurality of control registers further store a completion table having a loop count of an active number of loop threads, and wherein in response to receipt of an asynchronous fabric message returning a thread identifier to a thread identifier pool, the control circuit decrements the loop count and, when the loop count reaches zero, transmits an asynchronous fabric completion message.

### Claim 44 (depends on 35)

The system of claim 35, wherein the plurality of control registers further store a top of thread identifiers stack to allow each type of thread identifier access to private variables for a selected loop.

### Claim 45 (depends on 5)

The system of claim 5, wherein the control circuit further comprises:
a continuation queue; and
a reenter queue.

### Claim 46 (depends on 45)

The system of claim 45, wherein the continuation queue stores one or more thread identifiers for computation threads having completion counts allowing execution and do not yet have an assigned thread identifier.

### Claim 47 (depends on 45)

The system of claim 45, wherein the reenter queue stores one or more thread identifiers for computation threads having completion counts allowing execution and having an assigned thread identifier.

### Claim 48 (depends on 45)

The system of claim 45, wherein any thread having a thread identifier in the reenter queue is executed prior to execution of any thread having a thread identifier in the continuation queue.

### Claim 49 (depends on 5)

The system of claim 5, wherein the control circuit further comprises:
a priority queue, wherein any thread having a thread identifier in the priority queue is executed prior to execution of any thread having a thread identifier in the continuation queue or in the reenter queue.

### Claim 50 (depends on 5)

The system of claim 5, wherein the control circuit further comprises:
a run queue, wherein any thread having a thread identifier in the run queue is executed upon an occurrence of a spoke count for the thread identifier.

### Claim 51 (depends on 5)

The system of claim 5, wherein the control circuit is adapted to self-schedule a computation thread for execution.

### Claim 52 (depends on 5)

The system of claim 5, wherein the control circuit is adapted to order computation threads for execution.

### Claim 53 (depends on 5)

The system of claim 5, wherein the control circuit is adapted to order loop computation threads for execution.

### Claim 54 (depends on 5)

The system of claim 5, wherein the control circuit is adapted to commence execution of computation threads in response to one or more completion signals from data dependencies.

### Claim 55 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core adapted to execute a received instruction; and
a core control circuit coupled to the processor core, the core control circuit adapted to automatically schedule an instruction for execution by the processor core in response to a received work descriptor data packet or in response to a received event data packet.

### Claim 56 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core adapted to execute a fiber create instruction; and
a core control circuit coupled to the processor core, the core control circuit adapted to automatically schedule the fiber create instruction for execution by the processor core and generate one or more work descriptor data packets to another, second processor or the configurable circuit cluster for execution of a corresponding plurality of execution threads.

### Claim 57 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core adapted to execute a fiber create instruction; and
a core control circuit coupled to the processor core, the core control circuit adapted to schedule the fiber create instruction for execution by the processor core, to reserve a predetermined amount of memory space in a thread control memory to store return arguments, and to generate one or more work descriptor data packets to another, second processor or the configurable circuit cluster for execution of a corresponding plurality of execution threads.

### Claim 58 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit comprising:
an interconnection network interface;
a core control memory coupled to the interconnection network interface;
an execution queue coupled to the core control memory;
a control logic and thread selection circuit coupled to the execution queue and to the core control memory; and
an instruction cache coupled to the control logic and thread selection circuit and to the processor core.

### Claim 59 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit comprising:
an interconnection network interface;
a thread control memory coupled to the interconnection network interface;
a network response memory coupled to the interconnection network interface;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory;
an instruction cache coupled to the control logic and thread selection circuit and to the processor core; and
a command queue coupled to the processor core and to the interconnection network interface.

### Claim 60 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit comprising:
a thread control memory comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a program count register storing a received program count, a data cache, and a general purpose register storing a received argument;
an execution queue coupled to the thread control memory; and
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to automatically place the thread identifier in the execution queue, and to periodically select the thread identifier for execution of an instruction of the execution thread by the processor core, the processor core using data stored in the data cache or general purpose register.

### Claim 61 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit comprising:
a thread control memory comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a program count register storing a received program count, and thread state registers storing a valid state or a paused state for each thread identifier of the plurality of thread identifiers;
an execution queue coupled to the thread control memory; and
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to automatically place the thread identifier in the execution queue when it has a valid state, and for as long as the valid state remains, to periodically select the thread identifier for execution of an instruction of the execution thread by the processor core until completion of the execution thread.

### Claim 62 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit comprising:
a thread control memory comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a program count register storing a received program count, and thread state registers storing a valid state or a paused state for each thread identifier of the plurality of thread identifiers;
an execution queue coupled to the thread control memory; and
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to automatically place the thread identifier in the execution queue when it has a valid state, and for as long as the valid state remains, to periodically select the thread identifier for execution of an instruction of the execution thread by the processor core, and to pause thread execution by not returning the thread identifier to the execution queue when it has a pause state.

### Claim 63 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core adapted to execute a plurality of instructions; and
a core control circuit coupled to the processor core, the core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a work descriptor data packet, to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, a data cache, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to place the thread identifier in the execution queue, to select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread; and
an instruction cache coupled to the processor core and to the control logic and thread selection circuit to receive the initial program count and provide to the processor core a corresponding instruction for execution, of the plurality of instructions.

### Claim 64 (depends on 1)

The system of claim 1, wherein the processor comprises:
a core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a work descriptor data packet and to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, a data cache, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to automatically place the thread identifier in the execution queue, to periodically select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread; and
an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide a corresponding instruction for execution; and
a processor core coupled to the instruction cache of the core control circuit, the processor core adapted to execute the corresponding instruction.

### Claim 65 (depends on 1)

The system of claim 1, wherein the processor comprises:
a core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a work descriptor data packet, to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to place the thread identifier in the execution queue, to select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread;
an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide a corresponding instruction for execution; and
a command queue; and
a processor core coupled to the instruction cache and the command queue of the core control circuit, the processor core adapted to execute the corresponding instruction.

### Claim 66 (depends on 1)

The system of claim 1, wherein the processor comprises:
a core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a work descriptor data packet, to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to place the thread identifier in the execution queue, to select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread, and
an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide a corresponding instruction for execution; and
a processor core coupled to the instruction cache of the core control circuit, the processor core adapted to execute the corresponding instruction.

### Claim 67 (depends on 1)

The system of claim 1, wherein the processor comprises:
a core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a call work descriptor data packet, to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument, and to encode a work descriptor packet for transmission to other processing elements;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a network response memory coupled to the interconnection network interface;
a control logic and thread selection circuit coupled to the execution queue, to the thread control memory, and to the instruction cache, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to place the thread identifier in the execution queue, to select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread;
an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide a corresponding instruction for execution; and
a command queue storing one or more commands for generation of one or more work descriptor packets; and
a processor core coupled to the instruction cache and the command queue of the core control circuit, the processor core adapted to execute the corresponding instruction.

### Claim 68 (depends on 1)

The system of claim 1, wherein the processor comprises:
a processor core; and
a core control circuit coupled to the processor core.

### Claim 69 (depends on 68)

The system of claim 68, wherein the core control circuit comprises:
an interconnection network interface coupleable to an interconnection network, the interconnection network interface adapted to receive a work descriptor data packet and to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument, the interconnection network interface further adapted to receive an event data packet and to decode the received event data packet into an event identifier and any received argument.

### Claim 70 (depends on 69)

The system of claim 69, wherein the interconnection network interface is further adapted to generate a point-to-point event data message or to generate a broadcast event data message.

### Claim 71 (depends on 69)

The system of claim 69, wherein the core control circuit comprises:
a control logic and thread selection circuit coupled to the interconnection network interface, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread.

### Claim 72 (depends on 71)

The system of claim 71, wherein the core control circuit comprises:
a thread control memory having a plurality of registers, the plurality of registers comprising:
a thread identifier pool register storing a plurality of thread identifiers;
a thread state register;
a program count register storing a received initial program count; and
a general purpose register storing a received argument.

### Claim 73 (depends on 72)

The system of claim 72, wherein the control logic and thread selection circuit is further adapted to return a corresponding thread identifier of the selected thread to the thread identifier pool register in response to the execution of a return instruction by the processor core.

### Claim 74 (depends on 72)

The system of claim 72, wherein the control logic and thread selection circuit is further adapted to clear the registers of the thread control memory indexed by the corresponding thread identifier of the selected thread in response to the execution of a return instruction by the processor core.

### Claim 75 (depends on 72)

The system of claim 72, wherein the thread control memory further comprises one or more registers selected from the group consisting of:
a pending fiber return count register; a return argument buffer or register; a thread return argument link list register; a custom atomic transaction identifier register; a data cache; an event state register; an event mask register; and combinations thereof

### Claim 76 (depends on 72)

The system of claim 72, wherein the interconnection network interface is further adapted to store the execution thread having the initial program count and any received argument in the thread control memory using a thread identifier as an index to the thread control memory.

### Claim 77 (depends on 72)

The system of claim 72, wherein the core control circuit further comprises:
an execution queue coupled to the thread control memory, the execution queue storing one or more thread identifiers.

### Claim 78 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to place the thread identifier in the execution queue, to select the thread identifier for execution from the execution queue, and to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread for execution by the processor core.

### Claim 79 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to successively select a next thread identifier from the execution queue for execution of a single instruction of a corresponding execution thread.

### Claim 80 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to perform a round-robin or barrel selection of a next thread identifier from the execution queue, of the plurality of thread identifiers, each for execution of a single instruction of a corresponding execution thread until completion of the execution thread.

### Claim 81 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to assign a valid status or a pause status to a thread identifier.

### Claim 82 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to assign a priority status to a thread identifier.

### Claim 83 (depends on 77)

The system of claim 77, wherein the control logic and thread selection circuit is further adapted to, following execution of a corresponding instruction, to return the corresponding thread identifier to the execution queue with an assigned valid status and an assigned priority.

### Claim 84 (depends on 77)

The system of claim 77, wherein the execution queue further comprises:
a first priority queue; and
a second priority queue.

### Claim 85 (depends on 84)

The system of claim 84, wherein the control logic and thread selection circuit further comprises:
thread selection control circuitry coupled to the execution queue, the thread selection control circuitry adapted to select a thread identifier from the first priority queue at a first frequency and to select a thread identifier from the second priority queue at a second frequency, the second frequency lower than the first frequency.

### Claim 86 (depends on 85)

The system of claim 85, wherein the thread selection control circuitry is adapted to determine the second frequency as a skip count from selection of a thread identifier from the first priority queue.

### Claim 87 (depends on 77)

The system of claim 77, wherein the core control circuit further comprises:
a network command queue coupled to the processor core.

### Claim 88 (depends on 87)

The system of claim 87, wherein the processor core is adapted to execute a fiber create instruction to generate one or more commands to the network command queue for the interconnection network interface to generate one or more call work descriptor packets to another processor core or to a hybrid threading fabric circuit.

### Claim 89 (depends on 88)

The system of claim 88, wherein in response to execution of a fiber create instruction by the processor core, the core control circuit is adapted to reserve a predetermined amount of memory space in the general purpose registers or return argument registers.

### Claim 90 (depends on 77)

The system of claim 77, wherein in response to the generation of one or more call work descriptor packets to another processor core or to a hybrid threading fabric, the core control circuit is adapted to store a thread return count in the thread return register.

### Claim 91 (depends on 90)

The system of claim 90, wherein in response to receipt of a return data packet, the core control circuit is adapted to decrement the thread return count stored in the thread return register.

### Claim 92 (depends on 92)

The system of claim 92, wherein in response to the thread return count in the thread return register being decremented to zero, the core control circuit is adapted to change a paused status to a valid status for a corresponding thread identifier for subsequent execution of a thread return instruction for completion of the created fibers or threads.

### Claim 93 (depends on 71)

The system of claim 71, wherein the core control circuit further comprises:
an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide a corresponding instruction for execution by the processor core.

### Claim 94 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to assign an initial valid state to the execution thread.

### Claim 95 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to assign a pause state to the execution thread in response to the processor core executing a memory load instruction or a memory store instruction.

### Claim 96 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to end execution of a selected thread in response to the execution of a return instruction by the processor core.

### Claim 97 (depends on 71)

The system of claim 71, wherein the interconnection network interface is further adapted to generate a return work descriptor packet in response to the execution of a return instruction by the processor core.

### Claim 98 (depends on 68)

The system of claim 68, wherein the core control circuit further comprises:
a network response memory.

### Claim 99 (depends on 98)

The system of claim 98, wherein the network response memory comprises:
a memory request register; and
a thread identifier and transaction identifier register.

### Claim 100 (depends on 99)

The system of claim 99, wherein the network response memory further comprises one or more registers selected from the group consisting of:
a request cache line index register; a bytes register; a general purpose register index and type register; and combinations thereof.

### Claim 101 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to use an event mask stored in the event mask register to respond to a received event data packet.

### Claim 102 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to determine an event number corresponding to a received event data packet.

### Claim 103 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to change the status of a thread identifier from pause to valid in response to a received event data packet to resume execution of a corresponding execution thread.

### Claim 104 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to change the status of a thread identifier from pause to valid in response to an event number of a received event data packet to resume execution of a corresponding execution thread.

### Claim 105 (depends on 71)

The system of claim 71, wherein the interconnection network interface comprises:
an input queue;
a packet decoder circuit coupled to the input queue, to the control logic and thread selection circuit, and to the thread control memory;
an output queue; and
a packet encoder circuit coupled to the output queue, to the network response memory, and to the network command queue.

### Claim 106 (depends on 71)

The system of claim 71, wherein the core control circuit further comprises:
data path control circuitry adapted to control access size over the first interconnection network.

### Claim 107 (depends on 71)

The system of claim 71, wherein the core control circuit further comprises:
data path control circuitry adapted to increase or decrease memory load access size in response to time averaged usage levels.

### Claim 108 (depends on 71)

The system of claim 71, wherein the core control circuit further comprises:
data path control circuitry adapted to increase or decrease memory store access size in response to time averaged usage levels.

### Claim 109 (depends on 71)

The system of claim 71, wherein the control logic and thread selection circuit is further adapted to increase a size of a memory load access request to correspond to a cache line boundary of the data cache.

### Claim 110 (depends on 71)

The system of claim 71, wherein the core control circuit further comprises:
system call circuitry adapted to generate one or more system calls to a host processor.

### Claim 111 (depends on 110)

The system of claim 110, wherein the system call circuitry further comprises:
a plurality of system call credit registers storing a predetermined credit count to modulate a number of system calls in any predetermined period of time.

### Claim 112 (depends on 71)

The system of claim 71, wherein the core control circuit is further adapted, in response to a request from a host processor, to generate a command to a command queue for the interconnection network interface to copy and transmit all data from the thread control memory corresponding to a selected thread identifier for monitoring thread state.

### Claim 113 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a waiting or nonwaiting fiber join instruction.

### Claim 114 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a fiber join all instruction.

### Claim 115 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a non-cached read or load instruction to designate a general purpose register for storage of data received from a memory.

### Claim 116 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a non-cached write or store instruction to designate data in a general purpose register for storage in a memory.

### Claim 117 (depends on 68)

The system of claim 68, wherein the core control circuit is adapted to assign a transaction identifier to any load or store request to memory and to correlate the transaction identifier with a thread identifier.

### Claim 118 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a first thread priority instruction to assign a first priority to an execution thread having a corresponding thread identifier.

### Claim 119 (depends on 118)

The system of claim 118, wherein the processor core is adapted to execute a second thread priority instruction to assign a second priority to an execution thread having a corresponding thread identifier.

### Claim 120 (depends on 68)

The system of claim 68, wherein the processor core is adapted to execute a custom atomic return instruction to complete an executing thread of a custom atomic operation.

### Claim 121 (depends on 68)

The system of claim 68, wherein in conjunction with a memory controller, the processor core is adapted to execute a floating point atomic memory operation.

### Claim 122 (depends on 68)

The system of claim 68, wherein in conjunction with a memory controller, the processor core is adapted to execute a custom atomic memory operation.

### Claim 123 (depends on 1)

The system of claim 1, wherein data communication over the interconnection network occurs using a global, virtual address space across all nodes independent of a physical address space.

### Claim 124 (depends on 1)

The system of claim 1, wherein data communication over the interconnection network occurs using virtual node identifiers.

### Claim 125 (depends on 1)

The system of claim 1, further comprising:
one or more memory circuits operable in a plurality of modes, the plurality of modes comprising: private and not shared; shared and interleaved; shared and noninterleaved.

### Claim 126 (depends on 1)

The system of claim 1, wherein data communication over the interconnection network occurs using a split header and payload configuration for pipelining a plurality of communications to a plurality of different destinations.

### Claim 127 (depends on 1)

The system of claim 1, wherein the interconnection network is adapted to use the split header and payload configuration for delayed payload switching.

### Claim 128 (depends on 1)

The system of claim 1, wherein the interconnection network is adapted to use a single header to route a plurality of data payloads as a sequential data burst.

### Claim 129 (depends on 1)

The system of claim 1, wherein the interconnection network is adapted to interleave a first, acknowledgment message to a destination in an unused header field of a second message to the destination.

### Claim 130 (depends on 1)

The system of claim 1, wherein the interconnection network is adapted for power gating or clock gating based on load requirements.

### Claim 131 (depends on 1)

The system of claim 1, wherein each configurable circuit of the plurality of configurable circuits further comprises:
a plurality of delay registers to synchronize data transmission and reception for thread execution.

### Claim 132 (depends on 1)

The system of claim 1, wherein the at least one configurable circuit cluster further comprises:
a dispatch interface adapted to receive a work descriptor packet and to assign the work descriptor packet to a selected configurable circuit of the plurality of configurable circuits based on load balancing.

### Claim 133 (depends on 1)

The system of claim 1, wherein the at least one configurable circuit cluster further comprises:
a dispatch interface adapted to receive one or more completion messages from a configurable circuit of the plurality of configurable circuits and to generate a return work descriptor packet having return values.

### Claim 134 (depends on 1)

The system of claim 1, wherein each configurable circuit of the plurality of configurable circuits is further adapted to utilize an execution mask for selection of valid data elements.

### Claim 135

A system comprising:
a first, interconnection network;
a processor coupled to the interconnection network;
a host interface coupled to the interconnection network; and
at least one configurable circuit cluster coupled to the interconnection network, the configurable circuit cluster comprising:
a plurality of configurable circuits arranged in an array, each configurable circuit, comprising:
a configurable computation circuit;
a first memory circuit coupled to the configurable computation circuit;
a plurality of synchronous network inputs and outputs coupled to the configurable computation circuit;
an asynchronous network input queue and an asynchronous network output queue;
a second, configuration memory circuit coupled to the configurable computation circuit, to the control circuitry, to the synchronous network inputs, and to the synchronous network outputs, the second, configuration memory comprising:
a first, instruction memory storing a plurality of data path configuration instructions to configure a data path of the configurable computation circuit; and
a second, instruction and instruction index memory storing: a plurality of spoke instructions and data path configuration instruction indices for selection of a master synchronous input of the synchronous network inputs, for selection of a current data path configuration instruction for the configurable computation circuit, and for selection of a next data path instruction or next data path instruction index for a next configurable computation circuit;
and
a control circuit coupled to the configurable computation circuit, the control circuit comprising:
a memory control circuit;
a plurality of control registers, wherein the plurality of control registers store a completion table having a first, data completion count; and
a thread control circuit adapted to queue a thread for execution.

### Claim 136

A system comprising:
a first, interconnection network;
a host interface coupled to the interconnection network;
at least one configurable circuit cluster coupled to the interconnection network, the configurable circuit cluster comprising a plurality of configurable circuits arranged in an array; and
a processor coupled to the interconnection network, the processor comprising:
a processor core adapted to execute a plurality of instructions; and
a core control circuit coupled to the processor core, the core control circuit comprising:
an interconnection network interface coupleable to an interconnection network to receive a work descriptor data packet, to decode the received work descriptor data packet into an execution thread having an initial program count and any received argument;
a thread control memory coupled to the interconnection network interface and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing the received program count, a data cache, and a general purpose register storing the received argument;
an execution queue coupled to the thread control memory;
a control logic and thread selection circuit coupled to the execution queue and to the thread control memory, the control logic and thread selection circuit adapted to assign an available thread identifier to the execution thread, to place the thread identifier in the execution queue, to select the thread identifier for execution, to access the thread control memory using the thread identifier as an index to select the initial program count for the execution thread; and
an instruction cache coupled to the processor core and to the control logic and thread selection circuit to receive the initial program count and provide to the processor core a corresponding instruction for execution, of the plurality of instructions.
