# US 11,307,730 B2 · Pervasive 3D graphical user interface configured for machine learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,730 B2 - Document ID: US11307730B2 - Status: Granted - Kind: B2 - Application number: 16/164,928 - Filed: 2018-10-19 - Published: 2020-04-23 - Granted: 2022-04-19 - Earliest claimed: 2018-10-19 - Applicants: Wen-Chieh Geoffrey Lee - Inventors: Wen-Chieh Geoffrey Lee - Web page: https://patentgrep.com/patent/US11307730B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation - G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines - G06F 18/2431: Multiple classes - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 15/20: Perspective computation - G06V 10/764: using classification, e.g. of video objects - G06V 20/64: Three-dimensional [3D] objects - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30241: Trajectory - G06V 10/94: Hardware or software architectures specially adapted for image or video understanding - G06V 2201/06: Recognition of objects for industrial automation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A three-dimensional graphical user interface (3D GUI) configured to be used by a computer, a display system, an electronic system, or an electro-mechanical system. The 3D GUI provides an enhanced user-engaging experience while enabling a user to manipulate the motion of an object of arbitrary size and a multiplicity of independent degrees of freedom, using sufficient degrees of freedom to represent the motion. The 3D GUI includes the functionality of machine learning (ML), which uses support vector machine (SVM) and/or convolutional neural network (CNN) to provide intelligent control of robot kinematics and computer graphics as well as the ability of the user to more quickly learn the more subtle applications of 3D computer graphics. ## Claims ### Claim 1 A system comprising: a memory and at least one processor coupled to the memory; wherein said system is configured to display a three-dimensional (3D) graphical user interface (3D GUI) on a 3D graphical rendering device; wherein said 3D GUI is configured to enhance a user's experience of engaging with 3D scenery represented on said 3D graphical rendering device, said 3D scenery comprising a plurality of 3D objects whose properties are represented by a subset of 3D graphical vectors within a set of such vectors that have more than three degrees of freedom, said set of vectors having a volume in a 3D space, or properties associated with a set of data measured by an instrument; wherein said subset of 3D graphical vectors or said set of data measured by said instrument are represented in a coordinate system that has at least three dimensions; wherein said system is configured to enable its operator to adjust a perspective angle within a two-and-a-half-dimensional (2.5D) coordinate system embedded in said 3D graphical rendering device; wherein said perspective angle is an angle of intersection with the X and Z axes in said 2.5D coordinate system, or an angle of intersection with the X and Y axes in said 2.5D coordinate system, or an angle of intersection with any of two axes of a 2.5D coordinate system that are orthogonal to one another; wherein said system is configured to enable said operator to apply an artificial intelligence (AI) functionality that uses a machine learning process to classify into several separable classes, visually or by other sensory or cognitive modes, a plurality of said 3D subset of graphical vectors or said set of data measured by said instrument; wherein said 3D subset of graphical vectors or said set of data measured by said instrument are associated with a feature vector space and are used by said system to denote said 3D objects; wherein separate properties associated with motion vectors, graphical vectors, or relative distances between elements of a set of said 3D objects or a set of data that is measured by said instrument, are classified into the same class by said system; wherein each of said separate properties is controlled by a relationship, wherein said relationship is expressed as a function of the magnitude of said perspective angle, and said magnitude of said perspective angle is adjusted for each said class. ### Claim 2 (depends on 1) The system of claim 1: wherein said system is configured so that said process of classifying motion vectors, graphical vectors, relative distances among said 3D objects, or said set of data that is measured by said instrument, reaches an optimal condition at a specific perspective angle. ### Claim 3 (depends on 1) The system of claim 1: wherein said process of classifying reaches an optimal condition by designating an initial 3D position of a world space camera, followed by series of translational motion vectors and rotational motion vectors of said world space camera. ### Claim 4 (depends on 1) The system of claim 1: wherein said process of classifying reaches an optimal condition by using a neural network and machine learning as a trajectory analyzing process by said system without allowing any 3D object to conduct any realistic movement during said trajectory analyzing process. ### Claim 5 (depends on 1) The system of claim 1: wherein a leading index of performance used by said 3D graphical rendering device or means of 3D rendering is defined, wherein said leading index of performance comprises a metric that allows said system to seek an optimal performance by assessing the influence of a small variation of said perspective angle on the movement of an object presented therein, wherein the magnitude of said small variation is large enough for said system to identify its impact on a performance relating to said 3D rendering, but not large enough to cause physiological perturbations to a viewer. ### Claim 6 (depends on 1) The system of claim 1: wherein a transformation from a 3D coordinate system to a 2.5D coordinate system is made by using a “kernel trick,” whereby the transformation from a 3D system representation of a graphical object to a 2.5D system representation of the same said graphical object is the result of manipulating the respective perspective angle appropriately; wherein the process of said manipulating, is accomplished using the kernel functions K1 and K2 as shown in the following equations: K(x·x′)=K1(x·x′)+K2(x·x′) K(x·x′)=(x·x′) K(x·x′)=K1(x·x′)×K2(x·x′) K(x·x′)=xrAx′ K(x·x′)=K1(ƒ(x)·ƒ(x′)). ### Claim 7 (depends on 1) The system of claim 1 wherein said relationship between differential motions in a three-dimensional (3D) space and their corresponding motions in a 2.5-dimensional (2.5D) projected space is expressed in differential form as follows: dXdisplaying device=dX(2.5D)+dZ(2.5D)·cos(δ)−Z(2.5D)sin(δ)dδ dYdisplaying device=dY(2.5D)+dZ(2.5D)−sin(δ)−Z(2.5D)·cos(δ)dδ wherein said differentials express the variation of the coordinates of a graphical vector (Xdisplaying device, Ydisplaying device) in a 2.5D space that is projected on the screen of a displaying device when a perspective angle δ as visualized by the user of said displaying device undergoes a variation dci. ### Claim 8 (depends on 7) The system of claim 7 wherein said relationship, when using the functionalities of AI, takes into consideration the effects of occlusion wherein intervening objects block the line of sight as the angle of perspective undergoes said variation dδ and whereby an optimal perspective can be obtained. ### Claim 9 (depends on 8) The system of claim 8 further comprising an electro-mechanical system coupled to said system, wherein said electro-mechanical system is a maneuverable aircraft configured to receive external signals and, using an internal AI functionality, to convert them into appropriate responses that control the path of the maneuverable aircraft through a three-dimensional (3D) space comprising a multiplicity of 3D objects. ### Claim 10 (depends on 1) The system of claim 1 further comprising an electro-mechanical system coupled to said system, wherein said electromechanical system is a mobile, maneuverable robot that is externally controllable by said operator of said system or is under the influence of internal sensors, wherein said mobile, maneuverable robot's visual space is controllably altered as said robot undergoes external rotations, thereby creating accurate visual images of its position, internally, for autonomous operation, or, externally, for control by said operator, wherein said visual images are continually altered corresponding to associated changes in a perspective angle. ### Claim 11 (depends on 1) The system of claim 1 wherein reducing the dimensionality of the coordinate system from 3D to 2.5D, without any deterioration of the functionality of a neural network, reduces the power consumption of the system.