Yesterday, the United States Patent and Trademark Office officially published a patent application from Apple, which pertains to a potential input device that artists can use while wearing the Vision Pro.
Apple notes in their patent background that computer-generated environments are environments where at least some objects displayed for a user's viewing are generated using an electronic device such as a computer system. Virtual objects are generated by computers and are displayed for viewing by a user of an electronic device such as a head-mounted display (HMD). However, creating life-size virtual objects can be complex, time consuming, and can introduce undesirable results. This is what Apple's invention is to address.
Methods for Generating Virtual Objects and Sound
Apple's invention generally relates to electronic devices, and more particularity to electronic devices such as computer systems that are used for generating virtual objects based on input from an input device, movement of the input device, translation of the movement of the input device to movement of a virtual pointer, and optionally an elasticity model.
In some examples, an electronic device includes processing circuitry that is configured to present a virtual pointer element that is offset from a portion of the input device (e.g., controller) and/or a hand of a user which is used to generate virtual objects in a three-dimensional environment.
In some examples, the virtual object is generated in accordance with translation of movement of the input device to the movement of the virtual pointer element when generating the virtual object.
In some examples,the virtual object is generated in accordance with an elasticity model that translates the movement of the input device to movement of the virtual pointer element when generating the virtual object.
In some examples, the elasticity model implements one or more branches of physics which is applied to the translation of a movement of the input device to a movement of the virtual pointer element when generating the virtual objects.
For example, while generating virtual objects, the elasticity model is applied to the translation of the movement of the input device to the movement of the virtual pointer element which causes a delay in the virtual pointer element while generating the virtual objects. Accordingly, in some examples, at least a portion of the generated virtual objects are offset in position based on the translation relative to a position indicated by the movement of the input device.
In some examples, by applying the elasticity model to translate the movement of the input device to the movement of the virtual pointer, greater motor control of the virtual pointer element can be achieved, which in turn reduces undesirable handshaking artifacts appearing in the generated virtual object.
In some examples, the electronic device generates large scale virtual objects directly (e.g., optionally without generating virtual objects in small scale and then resizing to large scale), optionally in addition to reducing undesirable handshaking artifacts (e.g., reducing inconsistencies in a generated virtual object that are due to a user's hand shaking or other unintended movements while holding the input device).
In some examples, the electronic device generates virtual objects at distances beyond a reach of a user, optionally such that, using the electronic device, a user can generate a virtual object at a first position while the user is located at a second position that is in physical and/or virtual space beyond the reach of the user.
In some examples, sound can be generated in accordance with the movement of the input device where one or more characteristic of sound (e.g., volume, intensity, pitch) are adjusted based on the characteristics of the movement of the virtual pointer element (e.g., speed) and/or the material properties of the virtual pointer element.In this way, incorporating sound with the movement of the input device and/or incorporating sound while the virtual objects are generated can provide a more realistic and more immersive virtual experience.
Apple's patent FIG. 3D above illustrates an input device projecting a virtual pointer element towards a virtual easel in a 3D environment to generate virtual art; FIG. 4 illustrates an example block diagram illustrating respective inputs for an elasticity model.
Further to Apple's patent FIG. 3D above, a paintball with a mass and density is applied to the virtual pointer element #106. The input device #104 is used to fling the virtual pointer element against a display surface of the virtual easel #310 to create virtual art #312a-312b. Because the virtual pointer element is represented by a paintball with a mass, virtual representation #304 forms a curvature shape and represents the relationship between the input device and the paintball (e.g., virtual pointer element #106). When the input device is used to fling the paintball toward the surface of the virtual easel, the paintball moves along flight path #308 and contacts the display surface of the virtual easel to generate the virtual art.
In some examples,the elasticity model implements physics which is applied to the translation of the movement of the input device to movement of the virtual pointer element (e.g., paintball).
In some examples, the elasticity model includes Newton's laws of motion which can be used to determine a flight path #308 (e.g., trajectory) of the paintball as the paintball is flung toward the virtual easel.
Trajectory (e.g., flight path) is the path followed by a moving object (e.g., paintball) under the action of gravity. Using Newton's equations of motion, the flight path #308 and the range of trajectory (e.g., the distance from the virtual pointer element to the virtual easel) can be determine based on parameters such as, but not limited to, a velocity of the paintball, angle of launch, and an initial height.
Accordingly, to fling a paintball from a distance so that it can reach a surface of the virtual easel a user of the input device can adjust the velocity at which the paintball is flung, adjust a launch angle at which the paintball is flung, and/or adjust a height at which the paintball is flung.
To assist the artist in their creations, Apple has added 3D sounds to this system. Apple notes that in some examples, sound can be generated based on an event or action associated with the virtual pointer element #106. For example, sound is generated in response to the virtual pointer element drawing a virtual object, the virtual pointer element being stretched, and/or the virtual pointer element contacting or colliding with another object.
For example, with reference to FIG. 3A, while moving the input device #104 and using the virtual pointer element #106 to spray paint the virtual object #306,the generated audio sound is rendered to resemble the sound effect of aerosol being released from a pressurized container.A host of other sounds could be generated depending on what the artist is doing.
Translated from: patentlyapple
English 
Chinese