# US 2021/0060416 A1 · Information processing system, storage medium storing information processing program, information processing apparatus, and information processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0060416 A1
- Document ID: US20210060416A1
- Status: Published application
- Kind: A1
- Application number: 16/749,245
- Filed: 2020-01-22
- Published: 2021-03-04
- Earliest claimed: 2019-08-30
- Assignees: Nintendo Co., Ltd.
- Applicants: Nintendo Co., Ltd.
- Inventors: Atsushi UENO; Shota SHIMADA
- Web page: https://patentgrep.com/patent/US20210060416A1

## CPC

- A63F 13/24: Constructional details thereof, e.g. game controllers with detachable joystick handles
- A63F 13/21: characterised by their sensors, purposes or types
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/218: using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
- A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console
- A63F 13/235: using a wireless connection, e.g. infrared or piconet
- A63F 13/245: specially adapted to a particular type of game, e.g. steering wheels
- A63F 13/285: Generating tactile feedback signals via the game input device, e.g. force feedback
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/03: Arrangements for converting the position or the displacement of a member into a coded form
- G06F 3/0362: with detection of one-dimensional [1D] translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
- A63F 2300/1012: involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
- A63F 2300/1025: details of the interface with the game device, e.g. USB version detection
- A63F 2300/1037: being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
- A63F 2300/1043: being characterized by constructional details
- A63F 2300/105: using inertial sensors, e.g. accelerometers, gyroscopes
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A non-limiting example information processing system includes a first input apparatus including a strain sensor, a second input apparatus including a motion sensor, and an information processing apparatus. The strain sensor provides an output corresponding to a force applied to at least a portion of the first input apparatus. The motion sensor provides an output corresponding to a motion of the second input apparatus. The information processing apparatus includes a computer that executes obtaining strain data corresponding to the output of the strain sensor and motion data corresponding to the output of the motion sensor, and executing first control on an object disposed in a virtual space based on the strain data, and second control on the object based on the motion data, the second control being different from the first control.

## Claims

### Claim 1

An information processing system comprising a first input apparatus including a strain sensor, a second input apparatus including a motion sensor and different from the first input apparatus, and an information processing apparatus, wherein
the strain sensor provides an output corresponding to a force applied to at least a portion of the first input apparatus,
the motion sensor provides an output corresponding to a motion of the second input apparatus, and
the information processing apparatus includes a computer that executes
obtaining strain data corresponding to the output of the strain sensor and motion data corresponding to the output of the motion sensor, and
executing first control on an object disposed in a virtual space based on the strain data, and second control on the object based on the motion data, the second control being different from the first control.

### Claim 2 (depends on 1)

The information processing system according to claim 1, wherein
the first input apparatus further includes a holding portion, and
the strain sensor outputs the strain data corresponding to a force applied to the holding portion of the first input apparatus.

### Claim 3 (depends on 1)

The information processing system according to claim 1, wherein
the first input apparatus is at least partially elastically deformable in response to an external force applied thereto,
the strain sensor provides an output corresponding to deformation of the first input apparatus, and
the first control is executed based on deformation of the first input apparatus indicated by the strain data, and the second control is executed based on the motion data while the deformation of the first input apparatus is detected based on the strain data.

### Claim 4 (depends on 1)

The information processing system according to claim 1, wherein
the second control is executed in different ways between when the force applied to the first input apparatus satisfies a predetermined condition and when the force applied to the first input apparatus does not satisfy the predetermined condition.

### Claim 5 (depends on 4)

The information processing system according to claim 4, wherein
the second control is performed so that the object is moved based on the motion data when the force applied to the first input apparatus satisfies the predetermined condition, and the object is not moved, irrespective of the motion data having been obtained, when the force applied to the first input apparatus does not satisfy the predetermined condition.

### Claim 6 (depends on 1)

The information processing system according to claim 1, wherein
the second input apparatus is attached to a user's body,
the motion sensor outputs the motion data corresponding to a motion of the second input apparatus caused by the user's stamping-foot action, and
a process of moving the object in the virtual space based on the motion data corresponding to a motion of the second input apparatus caused by the user's stamping-foot action, is executed as the second control.

### Claim 7 (depends on 5)

The information processing system according to claim 5, wherein
the first input apparatus further includes a motion sensor which outputs motion data corresponding to a motion of the first input apparatus,
a process of changing an orientation of the object based on the motion data output from the first input apparatus, is executed as third control, and
an in-game merit is given to a user when the object has a predetermined orientation at a predetermined location in the virtual space.

### Claim 8 (depends on 1)

The information processing system according to claim 1, wherein
a deformation process of deforming the object based on the strain data, is executed as the first control, and
a process of changing a way of deforming the object based on the motion data output from the second input apparatus, is executed as the second control.

### Claim 9 (depends on 1)

The information processing system according to claim 1, wherein
a process of moving the object in a first direction based on the strain data, is executed as the first control, and
a process of moving the object in a second direction different from the first direction, based on the motion data, is executed as the second control.

### Claim 10 (depends on 9)

The information processing system according to claim 9, wherein
the first input apparatus further includes a holding portion,
the second input apparatus is attached to a portion of a user's body,
the strain sensor outputs the strain data corresponding to a force applied to the holding portion of the first input apparatus, and
the motion sensor outputs the motion data corresponding to a motion of the portion of the user's body to which the second input apparatus is attached.

### Claim 11 (depends on 9)

The information processing system according to claim 9, wherein
a shape of a target object disposed in the virtual space is changed based on a positional relationship between the object and the target object.

### Claim 12 (depends on 11)

The information processing system according to claim 11, wherein
a process of moving the object along an outer periphery of the target object, based on the motion data, is executed as the second control,
a process of moving the object in a direction including at least a directional component toward inside of the target object, based on the strain data, is executed as the first control, and
a process of changing the shape of the target object is executed based on the positional relationship between the object and the target object.

### Claim 13 (depends on 9)

The information processing system according to claim 9, wherein
a process of continuing to move the object in the first direction, based on the amount of a force applied to the holding portion of the first input apparatus indicated by the strain data, is executed as the first control.

### Claim 14 (depends on 11)

The information processing system according to claim 11, wherein
the executing the first control and the second control includes assessing the shape of the target object at the end of a predetermined time limit.

### Claim 15 (depends on 1)

The information processing system according to claim 1, wherein
the first control is executed when the strain data indicates that the amount of the force applied to the first input apparatus is greater than a first threshold, and the first control is executed when the strain data does not indicate that the amount of the force applied to the first input apparatus is greater than the first threshold, and indicates that the amount of the force applied to the first input apparatus is greater than a second threshold smaller than the first threshold after the amount of the force applied to the first input apparatus has exceeded the first threshold.

### Claim 16 (depends on 1)

The information processing system according to claim 1, wherein
the first control is executed on the object, based on the amount of a force applied to the first input apparatus at a timing that a motion occurs on the motion sensor of the second input apparatus.

### Claim 17 (depends on 16)

The information processing system according to claim 16, wherein
the first control is executed, based on the amount of a force having been applied to the first input apparatus during a period of time from a timing that a motion occurs on the motion sensor of the second input apparatus to a predetermined time before, when the amount of a force applied to the first input apparatus at the timing that a motion occurs on the motion sensor of the second input apparatus is smaller than the amount of deformation of the first input apparatus having occurred during the period of time from the timing that a motion occurs on the motion sensor of the second input apparatus to the predetermined time before.

### Claim 18

A non-transitory computer-readable storage medium having stored therein an information processing program executable by a computer of an information processing apparatus for performing a process using an output of a first input apparatus including a strain sensor, and an output of a second input apparatus including a motion sensor and different from the first input apparatus, wherein
the strain sensor provides an output corresponding to a force applied to at least a portion of the first input apparatus,
the motion sensor provides an output corresponding to a motion of the second input apparatus, and
the information processing program causes the computer to execute
obtaining strain data corresponding to the output of the strain sensor and motion data corresponding to the output of the motion sensor, and
executing first control on an object disposed in a virtual space based on the strain data, and second control on the object based on the motion data, the second control being different from the first control.

### Claim 19

An information processing apparatus for performing a process using an output of a first input apparatus including a strain sensor, and an output of a second input apparatus including a motion sensor and different from the first input apparatus, wherein
the strain sensor provides an output corresponding to a force applied to at least a portion of the first input apparatus,
the motion sensor provides an output corresponding to a motion of the second input apparatus, and
the information processing apparatus executes
obtaining strain data corresponding to the output of the strain sensor and motion data corresponding to the output of the motion sensor, and
executing first control on an object disposed in a virtual space based on the strain data, and second control on the object based on the motion data, the second control being different from the first control.

### Claim 20

An information processing method for performing a process using an output of a first input apparatus including a strain sensor, and an output of a second input apparatus including a motion sensor and different from the first input apparatus, wherein
the strain sensor provides an output corresponding to a force applied to at least a portion of the first input apparatus;
the motion sensor provides an output corresponding to a motion of the second input apparatus; and
the method comprises
obtaining strain data corresponding to the output of the strain sensor and motion data corresponding to the output of the motion sensor; and
executing first control on an object disposed in a virtual space based on the strain data, and second control on the object based on the motion data, the second control being different from the first control.
