DETAILED DESCRIPTION
Generally, techniques related to an optical computer system and use thereof are described. In an example, an optical computer system includes a multi-purpose optical device, an imager, and an image sensor. The imager displays an image to the multi-purpose optical device. The image is optically processed through diffractive optical layers of the multi-purpose optical device and the diffracted light from the multi-purpose optical device is detected by the image sensor. The multi-purpose optical device is configured to store data and perform one or more compute operations. For instance, at least one of the diffractive optical layers include an optical data storage portion that stores the data. Further, some or all of the diffractive optical layers include optical compute portions. In turn, each of the optical compute portions includes diffraction elements that diffract light to a next optical compute operations and the light diffraction collectively through the optical compute portions corresponds to an optical compute operation. Based on the information encoded in the image, data can be read from the optical data storage and/or the optical compute operation can be invoked. For instance, when the image encodes a data location on the optical data storage portion, light travels through the diffractive optical layers to that location and out from the multi-purpose device and is detected by the image sensor. The image sensor converts the detected light into a read of the data stored at that data location. In comparison, when the image encodes an input to the compute operation, light travels through the optical compute portions and out from the multi-purpose device and is detected by the image sensor. Here, the image sensor converts the detected light into an output of optically performing the operation on the input.
To illustrate, consider an example of an optical computer system implemented as an optical disk drive. In this illustrative example, the multi-purpose optical device represents an optical disk or card, and the imager and image sensor represent a reader head. More specifically, the multi-purpose optical device includes a plurality of diffractive optical layers made out of polymeric material, such as plastic, transparent to light having a wavelength between 400 nm and 700 nm. These layers are attached with transparent optical glue in a stack arrangement. Each of the diffractive optical layers includes an optical data storage portion and an optical compute portion arranged side-by-side. In turn, each optical data storage portion stores data holographically, meaning the data is stored as collection interference patterns. Each of the optical compute portions corresponds to a layer of an artificial neural network and its diffraction elements are arranged to optically perform the equivalent transformation of the corresponding neural network layer. The artificial neural network is pre-trained to perform a compute operation. The various optical compute portions are arranged such that light is diffracted through these portions in a manner equivalent to data transformations between the neural network layers. The imager includes an array of monochromatic light sources, such as an array of micro light emitting diodes (.mu.LEDs), beam manipulation optics, and an optical modulator mounted on an actuating arm. The image sensor includes an array of CCD or CMOS sensors also mounted on the actuating arm. Upon a first image encoding the location of data to be read from a particular optical data portion, the actuating arm moves the imager and the image sensor to a position parallel to the optical data storage portions and the first image is emitted as a light from the imager. The light is diffracted through the optical data storage portions, including through the data location. The image sensor detects the diffracted light that is output from the multi-purpose optical device and converts the diffracted light to read data. Upon a second image encoding input for the compute operation, the actuating arm moves the imager and the image sensor to a position parallel to the optical compute portions and the second image is emitted as a light from the imager. The light is diffracted through the optical compute portions. The image sensor detects the diffracted light that is output from the multi-purpose optical device and converts the diffracted light to an output of the compute operation.
The optical computer system provides various technical advantages according to embodiments of the present disclosure. For example, high data storage can be achieved in a small footprint. In particular, one terabit of data can be stored in one square millimeter within each of the diffractive optical layers. In addition, data reads and compute operations can be performed at the speed of light. Hence, such an optical computer system can free up other computing resources to perform other operations.
In the interest of clarity of explanation, various embodiments of the present disclosure are described in connection with a video game system that includes an optical computer system. However, the embodiments are not limited as such and similarly apply to any computer system that integrates or interfaces with such an optical system, including a personal computer, a mobile device, or a datacenter. Also in the interest of clarity of explanation, various embodiments of the present disclosure are described in connection with a multi-purpose optical device that stores data holographically and that optically perform compute operations of a pre-trained artificial neural network. However, the embodiments are not limited as such and similarly apply to other types of optical storage including, for instance, pit-based storage, and/or to other types of artificial intelligence models.
FIG. 1 illustrates an example video game console 110 that includes an optical computer system 114, according to embodiments of the present disclosure. As illustrated, the video game console 110 interfaces with a display 120 to present interactive content of a video game to a video game player 130. The video game console 110 executes the video game. Some or all of the video game data can be stored on the optical computer system 114 (e.g., on optical data storage portions thereof). Additionally or alternatively, some or all of the operations of the video game can be executed on the optical computer system 114 (e.g., on optical compute portions thereof).
In an example, the video game console 110 includes the optical computer system 114 along with other computational resources. As illustrated in FIG. 1, these computational resources include a graphics processing unit (GPU) 118, although other components are possible such as a memory, a central processing unit (CPU), and the like as further illustrated in FIG. 9. The optical computer system 114 provides optical data storage, such as holographic data storage, and optical compute operations, such as operations of an AI model implemented as a collection of diffractive optical layers. Hence, data can be read from and compute operations can be performed by the optical computer system 114 at the speed of light. In addition, some of the data storage and some of the compute operations can be moved from the other computational resources to the optical computer system 114, thereby freeing up these resources for other tasks.
In an illustration, the optical computer system 114 is used for two purposes. First, to store video game data of the video game. In this way, depending on a context of the video game, the relevant video game data can be read from the optical computer system 114 and rendered by the GPU 118 on the display 120. Second, the optical computer system 114 is set-up to perform speech recognition. In this way, natural language utterances of the video game player 130 can be translated into text that, in turn, can be translated into game commands and/or displayable text on the display 120 or displays of other video game players. In particular, a neural network can be pre-trained for speech recognition on an offline system. And the trained neural network can be implemented as the collection of diffractive optical layers of the optical computer system 114.
In the illustrative example, audio input 113 is sent from a computational resource to the optical computer system 114 over an application programming interface (API) 112. The audio input is converted into an image for display by an imager to a multi-purpose optical device of the optical computer system 114. The diffracted light out from the multi-purpose optical device can be detected by an image sensor of the optical computer system 114 and converted to an output text 115. This output text 115 can be sent back to the same or different computational resource over the API 112 or a different API.
Similarly, a context of the video game can be sent over an API to the optical computer system 114. In response, the context is converted into an image for display by the imager to the multi-purpose optical device. The diffracted light can be detected by the image sensor and converted into video game data. This video game data can be sent to the GPU 118 as graphics input 117 over an API 116. The GPU 118 renders the video game content as graphics output 119 that is then sent over the same or a different API for presentation at the display 120.
According to this illustrative example, some of the operations of the video game console 110 can be allocated to the optical computer system 114, including video game data reads and speech recognition. Such operations can be run in parallel with operations performed by other computational resources of the video game console 110.
In the interest of clarity of explanation, various embodiments of the present disclosure are described in connection with data storage and with compute operations for speech recognition as an application on an optical computer system. However, the embodiments are not limited as such and apply to other possible purposes that can be implemented as diffractive optical layers, including other types of compute operations (e.g., object detection, speech synthetization, and other applications). Generally, a compute operation of an application can be defined as a set of interrelated operations. Each of such operations can be implemented as an diffractive optical layer of a multi-purpose optical device and the different diffractive optical layers can be arranged depending on the dependencies between the sub-operations.
In addition, the optical computer system can include a plurality of multi-purpose optical devices, one for each supported application. A single multi-purpose optical device that supports a plurality of applications can also or alternatively be used. In this case, each diffractive optical layer can contain one or more optical compute portions corresponding to one of the supported applications. For instance, if speech recognition and object detection were to be supported as a ten-layer neural network and a fifteen-layer neural network, respectively, the multi-purpose optical device can include fifteen diffractive optical layers. Ten of these diffractive optical layers can have optical compute portions for the two applications side-by-side, and remaining five of the diffractive optical layers can have optical computer portions for only the object detection application.
In other words, a multi-purpose optical device of an optical computer system represents an optical device configured for multiple purposes. In one example, the purposes are for data storage and a particular application. In another example, the purposes are for different applications without data storage. In yet another example, the purposes are for a plurality of data storages and a plurality of applications.
Furthermore, although FIG. 1 illustrates an optical computer system on a video game console, such a system can be implemented differently. For instance, the optical computer system can be installed on a server communicatively coupled with the video game console and/or can be part of an optical disk for a video game, where the optical disk can be inserted in the video game console.
FIG. 2 illustrates an example optical computer system 200, according to embodiments of the present disclosure. The optical computer system 200 is an example of the optical computer system 114 of FIG. 1. As illustrated, the optical computer system 200 includes a transmissive multi-purpose optical device 210, an imager 220, and an image sensor 230. The transmissive multi-purpose optical device 210 includes a plurality of diffractive optical layers. In response to input data 222, the imager 220 displays an image 224 to an input side of the transmissive multi-purpose optical device 210. Light forming the image travels through this device 210 and is diffracted through its diffractive optical layers. Because of the transmissivity, the diffracted light 212 is transmitted out from an output side of the transmissive multi-purpose optical device 210 to the image sensor 230. The output side can be opposite to or, more generally, different from the input side. The image sensor 230 converts the diffracted light 212 into output data 232.
Generally, depending on the input data 222, a specific image 224 is generated to invoke one of the purposes of the transmissive multi-purpose optical device 210. In an example, the transmissive multi-purpose optical device 210 is usable for two purposes: one, for data storage and two, for compute operations of a speech recognition application. On one hand, if data storage were to be invoked, the input data 222 corresponds to a read command that identifies an address of the data on the transmissive multi-purpose optical device 210. The imager 210 generates an image 224 that contains lighting information for the retrieval of data encoded at the optical data storage position (e.g. the address) or that optically encodes this address. Upon display of this image 224, the image 224 (or light representing the image 224) diffracts through the transmissive multi-purpose optical device 210 and the image sensor 230 can read and output the data stored at the address. In an example of holographic data storage, diffractive elements are at a region X, which contains multiple addresses Ys. The address encoding is used as a key, where the image 224 is a reference light interference pattern (Y). A single spatial portion (X region) of the multi-purpose optical device can have multiple stored data points (Zs). These data points are the interference pattern of the address reference pattern (Y) combined with the interference pattern generated from the data (Z) that is stored. On the other hand, if speech recognition where to be invoked, the input data 222 corresponds to audio input. The imager 210 generates an image 224 that optically encodes the audio input. Upon display of this image 224, the image 224 (or light representing the image 224) diffracts through optical compute portions at the different diffractive optical layers. The diffracted light 212 out from the transmissive multi-purpose optical device 210 represents an output of performing the speech recognition and is sensed by the image sensor 230.