Recently the U.S. Patent and Trademark Office officially granted Apple a patent that relates to smartglasses. One feature that could be included is a geographic location sensor such as a satellite navigation system sensor and may include an orientation sensor such as an inertial measurement unit. The smartglasses could also double as a sleep mask or pair of meditation glasses.
Head-mounted Systems With Sensor For Eye Monitoring
A head-mounted device may have a head-mounted housing that is worn on the head of a user. The head-mounted housing may have displays that overlap a user's eyes or may not contain displays. Speakers may be provided in the housing to provide audio output to the user's ears.
The head-mounted housing may have transparent lenses that overlap the eyes of the user or may have opaque structures that prevent ambient light from reaching the user's eyes. Opaque structures may, for example, be used in a head-mounted device such as a sleep mask or pair of meditation glasses that block ambient light. Opaque structures covering the user's eyes may also be used in head-mounted devices such as virtual reality goggles that have forward-facing cameras that capture images of the real world.
During use of a head-mounted device, it may be desirable to monitor eye movements. For example, eye movements may provide information about whether the user is awake or asleep. Eye movement data may also supply information about the direction of a user's gaze. Information on the user's gaze (direction of viewing) may be used as input to the device, may be used to help efficiently display foveated content on a display, may be used to determine which virtual and/or real objects in the user's field of view are currently being viewed by the user to provide the device with context (e.g., so that a user may request more information about the currently viewed object, so that the device can automatically supply such information, etc.), and/or may otherwise be used by the head-mounted device.
A head-mounted device may include one or more gaze tracking systems such as systems based on image sensors that detect and process eye glints (eye reflections arising when the eye is illuminated by light-emitting diodes or other light-sources near the eye) and/or that detect and process images of the user's eye (e.g., retinal images, images of the user's pupil, etc.). Gaze tracking systems such as these may operate at infrared and/or visible wavelengths.
To supplement these devices and/or to help reduce bulk, complexity, and/or weight, eye monitoring can also be performed by gathering electrical measurements of the user's skin near the user's eyes. These measurements are sometimes referred to as electrooculograms. The human eye exhibits a standing potential between the cornea and retina. Electooculographic measurements made from eye monitoring electrodes near a user's eye can be used to detect the user's eye motions and direction of gaze. In some configurations, electrooculograms can be influenced by eye blinks and movement of facial muscles, so electrooculograms may also reveal information on a user's eye blinks and facial expressions.
Apple's patent FIGS. 5 and 6 below are front views of an eye showing illustrative eye monitoring electrode patterns for a head-mounted device; FIG. 7 is a top view of a left portion of an illustrative head-mounted device with eye monitoring electrodes; FIG. 10 is a diagram showing operations involved in using an electronic device with electrooculographic capabilities.
More specifically, FIG. 7 above is a top view of a left-hand portion of an illustrative electronic device (smartglasses). As shown in FIG. 7, support structures 26 may be configured so that electrodes 46 are located adjacent to the locations of the user's eyes such as eye box 30. When structures 26 are worn on a user's head, electrodes 46 may press against the user's face to gather electrooculography measurements.
In some configurations, device #10 does not contain a display (e.g., no display is used to provide an image to eye box #30). In this type of arrangement, portion #26L may be transparent (e.g., portion 26L may be a transparent lens in a pair of glasses such as a pair of sunglasses) or portion 26L may be opaque (e.g., portion 26L may be formed from one or more opaque materials). In configurations such as these in which portion 26L is opaque(e.g., when device 10 is a pair of meditation glasses or a sleep mask)portions 26L may be configured to block some or all of the ambient light surrounding the glasses so that little or no ambient light reaches eye box.
Regardless of whether or not eye box is illuminated by ambient light and/or whether a user's gaze is being tracked by an image-sensor based gaze tracking sensor, the electrooculography sensor formed with electrodes #46 may measure eye movements. For example, if a user is sleeping while wearing a sleep mask, the electrooculography sensor formed using the electrodes 46 can measure the user's eye movements (e.g., to monitor a user's sleep cycles such as a non-rapid-eye-movement sleep cycle, a rapid eye movement sleep cycle, etc.). These measurements may be made even if the user's eyes are closed.
Lastly, the head-mounted device / smartglasses may have a geographic location sensor such as a satellite navigation system sensor and may include an orientation sensor such as an inertial measurement unit.
The direction of a user's gaze relative to the head mounted device may be measured using an electrooculographic sensor and, if desired, optional image-sensor-based gaze tracking systems. By using geographic location information, orientation information, and gaze information, and by using pattern recognition techniques and/or on-line database look-up techniques, the head-mounted device may identify objects in the user's environment and can identify which of these objects is currently being viewed by the user.
Translated from: patentlyapple
English 
Chinese