E-mail Alert
RSS
| Citation: |
Xiang S Y, Song Z W, Zhang Y H, et al. Progress in the research of optical neural networks[J]. Opto-Electron Eng, 2024, 51(7): 240101. doi: 10.12086/oee.2024.240101
|
Progress in the research of optical neural networks
-
Abstract
In the era of massive data and information, electronic computer processing systems face increasingly greater demands on computing power and energy consumption. Bottlenecks such as the "memory wall" and "power wall" inherent in the traditional von Neumann architecture, coupled with the slowing down or even invalidation of Moore's Law, have posed significant challenges to electronic chips in terms of computing speed and power consumption. Utilizing optical computing as an alternative to traditional electronic computing represents one of the most promising avenues to address current challenges in computing power and power consumption. This review systematically summarized the research progress of optical neural network architectures and algorithms in both on-chip integration and free space, and described typical research efforts in detail. Then, the advantages and disadvantages of these two types of optical neural networks and the training strategies of optical neural networks were discussed and compared. Finally, the potential challenges that optical neural networks may encounter were discussed in depth, and a forward-looking perspective on their future development was offered.-
Keywords:
- optical neural network /
- on-chip integration /
- free space /
- brain-like computing
-
-
References
[1] McCarthy J, Minsky M L, Rochester N, et al. A proposal for the Dartmouth summer research project on artificial intelligence: August 31, 1955[J]. AI Mag, 2006, 27(4): 12−14. doi: 10.1609/aimag.v27i4.1904 [2] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436−444. doi: 10.1038/nature14539 [3] Moore G E. Cramming more components onto integrated circuits[J]. Electronics, 1965, 38 (8): 114–117. [4] Mead C. Neuromorphic electronic systems[J]. Proc IEEE, 1990, 78(10): 1629−1636. doi: 10.1109/5.58356 [5] Amunts K, Ebell C, Muller J, et al. The human brain project: creating a European research infrastructure to decode the human brain[J]. Neuron, 2016, 92(3): 574−581. doi: 10.1016/j.neuron.2016.10.046 [6] Insel T R, Landis S C, Collins F S. The NIH BRAIN initiative[J]. Science, 2013, 340(6133): 687−688. doi: 10.1126/science.1239276 [7] Martin C L, Chun M. The BRAIN initiative: building, strengthening, and sustaining[J]. Neuron, 2016, 92(3): 570−573. doi: 10.1016/j.neuron.2016.10.039 [8] Ngai J. BRAIN 2.0: transforming neuroscience[J]. Cell, 2022, 185(1): 4−8. doi: 10.1016/j.cell.2021.11.037 [9] Okano H, Sasaki E, Yamamori T, et al. Brain/MINDS: a Japanese national brain project for marmoset neuroscience[J]. Neuron, 2016, 92(3): 582−590. doi: 10.1016/j.neuron.2016.10.018 [10] Poo M M. Whereto the mega brain projects?[J]. Natl Sci Rev, 2014, 1(1): 12−14. doi: 10.1093/nsr/nwt019 [11] Poo M M, Du J L, Ip N Y, et al. China brain project: basic neuroscience, brain diseases, and brain-Inspired computing[J]. Neuron, 2016, 92(3): 591−596. doi: 10.1016/j.neuron.2016.10.050 [12] 蒲慕明, 徐波, 谭铁牛. 脑科学与类脑研究概述[J]. 中国科学院院刊, 2016, 31(7): 725−736 doi: 10.16418/j.issn.1000-3045.2016.07.001 Poo M M, Xu B, Tan T N. Brain science and brain-inspired intelligence technolog—an overview[J]. Bull Chin Acad Sci, 2016, 31(7): 725−736 doi: 10.16418/j.issn.1000-3045.2016.07.001 [13] 黄铁军, 施路平, 唐华锦, 等. 多媒体技术研究: 2015——类脑计算的研究进展与发展趋势[J]. 中国图象图形学报, 2016, 21(11): 1411−1424. doi: 10.11834/jig.20161101 Huang T J, Shi L P, Tang H J, et al. Research on multimedia technology 2015——advances and trend of brain-like computing[J]. J Image Graphics, 2016, 21(11): 1411−1424. doi: 10.11834/jig.20161101 [14] 项水英, 宋紫薇, 高爽, 等. 光神经形态计算研究进展与展望(特邀)[J]. 光子学报, 2021, 50(10): 1020001. doi: 10.3788/gzxb20215010.1020001 Xiang S Y, Song Z W, Gao S, et al. Progress and prospects of photonic neuromorphic computing (Invited)[J]. Acta Photonica Sin, 2021, 50(10): 1020001. doi: 10.3788/gzxb20215010.1020001 [15] Painkras E, Plana L A, Garside J, et al. SpiNNaker: a 1-W 18-core system-on-chip for massively-parallel neural network simulation[J]. IEEE J Solid-State Circuits, 2013, 48(8): 1943−1953. doi: 10.1109/JSSC.2013.2259038 [16] Benjamin B V, Gao P R, McQuinn E, et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations[J]. Proc IEEE, 2014, 102(5): 699−716. doi: 10.1109/JPROC.2014.2313565 [17] Merolla P A, Arthur J V, Alvarez-Icaza R, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface[J]. Science, 2014, 345(6197): 668−673. doi: 10.1126/science.1254642 [18] Schemmel J, Brüderle D, Grübl A, et al. A wafer-scale neuromorphic hardware system for large-scale neural modeling[C]//2010 IEEE International Symposium on Circuits and Systems (ISCAS), 2010: 1947–1950. https://doi.org/10.1109/ISCAS.2010.5536970. [19] Ma D, Shen J C, Gu Z H, et al. Darwin: a neuromorphic hardware co-processor based on spiking neural networks[J]. J Syst Archit, 2017, 77: 43−51. doi: 10.1016/j.sysarc.2017.01.003 [20] Davies M, Srinivasa N, Lin T H, et al. Loihi: a neuromorphic manycore processor with on-chip learning[J]. IEEE Micro, 2018, 38(1): 82−99. doi: 10.1109/MM.2018.112130359 [21] Orchard G, Frady E P, Rubin D B D, et al. Efficient neuromorphic signal processing with loihi 2[C]//2021 IEEE Workshop on Signal Processing Systems (SiPS), 2021: 254–259. https://doi.org/10.1109/SiPS52927.2021.00053. [22] Shi L P, Pei J, Deng N, et al. Development of a neuromorphic computing system[C]//2015 IEEE International Electron Devices Meeting (IEDM), 2015: 4.3.1–4.3.4. https://doi.org/10.1109/IEDM.2015.7409624. [23] Liu Z S, Chen S, Qu P Y, et al. SUSHI: ultra-high-speed and ultra-low-power neuromorphic chip using superconducting single-flux-quantum circuits[C]//Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, 2023: 614–627. [24] Miller D. Device requirements for optical interconnects to silicon chips[J]. Proc IEEE, 2009, 97(7): 1166−1185. doi: 10.1109/JPROC.2009.2014298 [25] Nahmias M A, De Lima T F, Tait A N, et al. Photonic multiply-accumulate operations for neural networks[J]. IEEE J Sel Top Quantum Electron, 2020, 26(1): 7701518. doi: 10.1109/JSTQE.2019.2941485 [26] Tait A N, Nahmias M A, Tian Y, et al. Photonic neuromorphic signal processing and computing[M]//Naruse M. Nanophotonic Information Physics: Nanointelligence and Nanophotonic Computing. Berlin: Springer, 2014: 183–222. https://doi.org/10.1007/978-3-642-40224-1_8. [27] Shastri B J, Chang J, Tait A N, et al. Ultrafast optical techniques for communication networks and signal processing[M]//Wabnitz S, Eggleton B J. All-Optical Signal Processing: Data Communication and Storage Applications. Cham: Springer, 2015: 469–503. https://doi.org/10.1007/978-3-319-14992-9_15. [28] Hopfield J J. Neural networks and physical systems with emergent collective computational abilities[J]. Proc Natl Acad Sci, 1982, 79(8): 2554−2558. doi: 10.1073/pnas.79.8.2554 [29] Liu J, Wu Q H, Sui X, et al. Research progress in optical neural networks: theory, applications and developments[J]. PhotoniX, 2021, 2(1): 5. doi: 10.1186/s43074-021-00026-0 [30] Tsai F C F, O’Brien C J, Petrović N S, et al. Analysis of optical channel cross talk for free-space optical interconnects in the presence of higher-order transverse modes[J]. Appl Opt, 2005, 44(30): 6380−6387. doi: 10.1364/AO.44.006380 [31] Hu W H, Li X J, Yang J K, et al. Crosstalk analysis of aligned and misaligned free-space optical interconnect systems[J]. J Opt Soc Am A, 2010, 27(2): 200−205. doi: 10.1364/JOSAA.27.000200 [32] Xiang S Y, Wen A J, Pan W. Emulation of spiking response and spiking frequency property in VCSEL-based photonic neuron[J]. IEEE Photonics J, 2016, 8(5): 1−9. doi: 10.1109/JPHOT.2016.2614104 [33] Xiang S Y, Zhang H, Guo X X, et al. Cascadable neuron-like spiking dynamics in coupled VCSELs subject to orthogonally polarized optical pulse injection[J]. IEEE J Sel Top Quantum Electron, 2017, 23(6): 1−7. doi: 10.1109/jstqe.2017.2678170 [34] Xiang S Y, Zhang Y H, Guo X X, et al. Photonic generation of neuron-like dynamics using VCSELs subject to double polarized optical injection[J]. J Lightwave Technol, 2018, 36(19): 4227−4234. doi: 10.1109/JLT.2018.2818195 [35] Zhang Y H, Xiang S Y, Gong J K, et al. Spike encoding and storage properties in mutually coupled vertical-cavity surface-emitting lasers subject to optical pulse injection[J]. Appl Opt, 2018, 57(7): 1731. doi: 10.1364/AO.57.001731 [36] Zhang Y H, Xiang S Y, Guo X X, et al. Polarization-resolved and polarization- multiplexed spike encoding properties in photonic neuron based on VCSEL-SA[J]. Sci Rep, 2018, 8(1): 16095. doi: 10.1038/s41598-018-34537-x [37] Zhang Y, Xiang S, Guo X, et al. All-optical inhibitory dynamics in photonic neuron based on polarization mode competition in a VCSEL with an embedded saturable absorber[J]. Opt Lett, 2019, 44(7): 1548−1551. doi: 10.1364/OL.44.001548 [38] Xiang S Y, Ren Z X, Zhang Y H, et al. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on a VCSEL-SA[J]. Opt Lett, 2020, 45(5): 1104−1107. doi: 10.1364/OL.383942 [39] Xiang S Y, Gong J K, Zhang Y H, et al. Numerical implementation of wavelength-dependent photonic spike timing dependent plasticity based on VCSOA[J]. IEEE J Quantum Electron, 2018, 54(6): 8100107. doi: 10.1109/jqe.2018.2879484 [40] Song Z W, Xiang S Y, Cao X Y, et al. Experimental demonstration of photonic spike-timing-dependent plasticity based on a VCSOA[J]. Sci China Inf Sci, 2022, 65(8): 182401. doi: 10.1007/s11432-021-3350-9 [41] Xiang S Y, Han Y N, Guo X X, et al. Real-time optical spike-timing dependent plasticity in a single VCSEL with dual-polarized pulsed optical injection[J]. Sci China Inf Sci, 2020, 63(6): 160405. doi: 10.1007/s11432-020-2820-y [42] Xiang S Y, Zhang Y H, Gong J K, et al. STDP-based unsupervised spike pattern learning in a photonic spiking neural network With VCSELs and VCSOAs[J]. IEEE J Sel Top Quantum Electron, 2019, 25(6): 1700109. doi: 10.1109/JSTQE.2019.2911565 [43] Xiang S Y, Ren Z X, Song Z W, et al. Computing primitive of fully VCSEL-based all-optical spiking neural network for supervised learning and pattern classification[J]. IEEE Trans Neural Networks Learn Syst, 2021, 32(6): 2494−2505. doi: 10.1109/TNNLS.2020.3006263 [44] Fu C T, Xiang S Y, Han Y N, et al. Multilayer photonic spiking neural networks: generalized supervised learning algorithm and network optimization[J]. Photonics, 2022, 9(4): 217. doi: 10.3390/photonics9040217 [45] Zhang Y H, Xiang S Y, Guo X X, et al. The winner-take-all mechanism for all-optical systems of pattern recognition and max-pooling operation[J]. J Lightwave Technol, 2020, 38(18): 5071−5077. doi: 10.1109/JLT.2020.3000670 [46] Han Y N, Xiang S Y, Ren Z X, et al. Delay-weight plasticity-based supervised learning in optical spiking neural networks[J]. Photonics Res, 2021, 9(4): B119−B127. doi: 10.1364/PRJ.413742 [47] Song Z W, Xiang S Y, Ren Z X, et al. Photonic spiking neural network based on excitable VCSELs-SA for sound azimuth detection[J]. Opt Express, 2020, 28(2): 1561−1573. doi: 10.1364/OE.381229 [48] Song Z W, Xiang S Y, Ren Z X, et al. Spike sequence learning in a photonic spiking neural network consisting of VCSELs-SA with supervised training[J]. IEEE J Sel Top Quantum Electron, 2020, 26(5): 1700209. doi: 10.1109/jstqe.2020.2975564 [49] Wang S H, Xiang S Y, Han G Q, et al. Photonic associative learning neural network based on VCSELs and STDP[J]. J Lightwave Technol, 2020, 38(17): 4691−4698. doi: 10.1109/JLT.2020.2995083 [50] Zhang Y H, Xiang S Y, Guo X X, et al. A modified supervised learning rule for training a photonic spiking neural network to recognize digital patterns[J]. Sci China Inf Sci, 2021, 64(2): 122403. doi: 10.1007/s11432-020-3040-1 [51] Gao S, Xiang S Y, Song Z W, et al. All-optical Sudoku solver with photonic spiking neural network[J]. Opt Commun, 2021, 495: 127068. doi: 10.1016/j.optcom.2021.127068 [52] Gao S, Xiang S Y, Song Z W, et al. Motion detection and direction recognition in a photonic spiking neural network consisting of VCSELs-SA[J]. Opt Express, 2022, 30(18): 31701−31713. doi: 10.1364/OE.465653 [53] Xiang S Y, Ren Z X, Zhang Y H, et al. Training a multi-layer photonic spiking neural network with modified supervised learning algorithm based on photonic STDP[J]. IEEE J Sel Top Quantum Electron, 2021, 27(2): 7500109. doi: 10.1109/JSTQE.2020.3005589 [54] Zhang Y H, Xiang S Y, Han Y N, et al. BP-based supervised learning algorithm for multilayer photonic spiking neural network and hardware implementation[J]. Opt Express, 2023, 31(10): 16549−16559. doi: 10.1364/OE.487047 [55] Song Z W, Xiang S Y, Zhao S H, et al. A multi-layer photonic spiking neural network with a modified backpropagation algorithm for nonlinear classification[J]. Opt Commun, 2023, 546: 129806. doi: 10.1016/j.optcom.2023.129806 [56] Xiang S Y, Zhang T R, Han Y N, et al. Neuromorphic speech recognition with photonic convolutional spiking neural networks[J]. IEEE J Sel Top Quantum Electron, 2023, 29(6): 7600507. doi: 10.1109/JSTQE.2023.3240248 [57] Han Y N, Xiang S Y, Zhang Y N, et al. An all-MRR-based photonic spiking neural network for spike sequence learning[J]. Photonics, 2022, 9(2): 120. doi: 10.3390/photonics9020120 [58] Zhang Y N, Xiang S Y, Han Y N, et al. Supervised learning and pattern recognition in photonic spiking neural networks based on MRR and phase-change materials[J]. Opt Commun, 2023, 549: 129870. doi: 10.1016/j.optcom.2023.129870 [59] Song Z W, Xiang S Y, Zhao S T, et al. A hybrid-integrated photonic spiking neural network framework based on an MZI array and VCSELs-SA[J]. IEEE J Sel Top Quantum Electron, 2023, 29(2): 8300211. doi: 10.1109/JSTQE.2022.3200942 [60] Zheng D Z, Xiang S Y, Guo X X, et al. Experimental demonstration of coherent photonic neural computing based on a Fabry–Perot laser with a saturable absorber[J]. Photonics Res, 2023, 11(1): 65−71. doi: 10.1364/PRJ.471950 [61] Song Z W, Xiang S Y, Guo X X, et al. Nonlinear neural computation in an integrated FP-SA spiking neuron subject to incoherent dual-wavelength optical pulse injections[J]. Sci China Inf Sci, 2023, 66(12): 229405. doi: 10.1007/s11432-022-3749-3 [62] Xiang S Y, Shi Y C, Guo X X, et al. Hardware-algorithm collaborative computing with photonic spiking neuron chip based on an integrated Fabry–Perot laser with a saturable absorber[J]. Optica, 2023, 10(2): 162−171. doi: 10.1364/OPTICA.468347 [63] Guo X X, Xiang S Y, Zhang Y H, et al. Hardware implementation of multi-layer photonic spiking neural network with three cascaded photonic spiking neurons[J]. J Lightwave Technol, 2023, 41(20): 6533−6541. doi: 10.1109/JLT.2023.3287647 [64] Han Y N, Xiang S Y, Gao S, et al. Experimental demonstration of delay-weight learning and pattern classification with a FP-SA-based photonic spiking neuron chip[J]. J Lightwave Technol, 2024, 42(5): 1497−1503. doi: 10.1109/JLT.2023.3322628 [65] Zhang Y H, Xiang S Y, Guo X X, et al. Spiking information processing in a single photonic spiking neuron chip with double integrated electronic dendrites[J]. Photonics Res, 2023, 11(12): 2033−2041. doi: 10.1364/PRJ.499767 [66] Gao S, Xiang S Y, Song Z W, et al. Hardware implementation of ultra-fast obstacle avoidance based on a single photonic spiking neuron[J]. Laser Photonics Rev, 2023, 17(12): 2300424. doi: 10.1002/lpor.202300424 [67] Xiang S Y, Gao S, Shi Y C, et al. Experimental demonstration of a photonic spiking neuron based on a DFB laser subject to side-mode optical pulse injection[J]. Sci China Inf Sci, 2024, 67(3): 132402. doi: 10.1007/s11432-023-3810-9 [68] Gao S, Xiang S Y, Zheng D Z, et al. Cascadable excitability and inhibition in DFB laser-based photonic spiking neurons[J]. Opt Commun, 2024, 554: 130207. doi: 10.1016/j.optcom.2023.130207 [69] Zhang Y N, Xiang S Y, Song Z W, et al. Evolution of neuron-like spiking response and spike-based all-optical XOR operation in a DFB with saturable absorber[J]. J Lightwave Technol, 2024, 42(6): 2026−2035. doi: 10.1109/JLT.2023.3331252 [70] Yu C Y, Xiang S Y, Zhang Y N, et al. Neuromorphic convolution with a spiking DFB-SA laser neuron based on rate coding[J]. Opt Express, 2023, 31(26): 43698−43711. doi: 10.1364/OE.499085 [71] Han Y N, Xiang S Y, Song Z W, et al. Pattern recognition in multi-synaptic photonic spiking neural networks based on a DFB-SA chip[J]. Opto-Electron Sci, 2023, 2(9): 230021−230021. doi: 10.29026/oes.2023.230021 [72] Xiang S Y, Shi Y C, Zhang Y H, et al. Photonic integrated neuro-synaptic core for convolutional spiking neural network[J]. Opto-Electron Adv, 2023, 6(11): 230140. doi: 10.29026/oea.2023.230140 [73] Hurtado A, Henning I D, Adams M J. Optical neuron using polarisation switching in a 1550nm-VCSEL[J]. Opt Express, 2010, 18(24): 25170−25176. doi: 10.1364/OE.18.025170 [74] Hurtado A, Schires K, Henning I D, et al. Investigation of vertical cavity surface emitting laser dynamics for neuromorphic photonic systems[J]. Appl Phys Lett, 2012, 100(10): 103703. doi: 10.1063/1.3692726 [75] Robertson J, Deng T, Javaloyes J, et al. Controlled inhibition of spiking dynamics in VCSELs for neuromorphic photonics: theory and experiments[J]. Opt Lett, 2017, 42(8): 1560−1563. doi: 10.1364/OL.42.001560 [76] Hurtado A, Javaloyes J. Controllable spiking patterns in long-wavelength vertical cavity surface emitting lasers for neuromorphic photonics systems[J]. Appl Phys Lett, 2015, 107(24): 241103. doi: 10.1063/1.4937730 [77] Deng T, Robertson J, Hurtado A. Controlled propagation of spiking dynamics in vertical-cavity surface-emitting lasers: towards neuromorphic photonic networks[J]. IEEE J Sel Top Quantum Electron, 2017, 23(6): 1800408. doi: 10.1109/JSTQE.2017.2685140 [78] Robertson J, Hejda M, Bueno J, et al. Ultrafast optical integration and pattern classification for neuromorphic photonics based on spiking VCSEL neurons[J]. Sci Rep, 2020, 10(1): 6098. doi: 10.1038/s41598-020-62945-5 [79] Robertson J, Wade E, Kopp Y, et al. Toward neuromorphic photonic networks of ultrafast spiking laser neurons[J]. IEEE J Sel Top Quantum Electron, 2020, 26(1): 7700715. doi: 10.1109/jstqe.2019.2931215 [80] Robertson J, Kirkland P, Alanis J A, et al. Ultrafast neuromorphic photonic image processing with a VCSEL neuron[J]. Sci Rep, 2022, 12(1): 4874. doi: 10.1038/s41598-022-08703-1 [81] Robertson J, Kirkland P, Di Caterina G, et al. VCSEL-based photonic spiking neural networks for ultrafast detection and tracking[J]. Neuromorph Comput Eng, 2024, 4(1): 014010. doi: 10.1088/2634-4386/ad2d5c [82] Chen Z J, Sludds A, Davis R, et al. Deep learning with coherent VCSEL neural networks[J]. Nat Photonics, 2023, 17(8): 723−730. doi: 10.1038/s41566-023-01233-w [83] Wang J W, Sciarrino F, Laing A, et al. Integrated photonic quantum technologies[J]. Nat Photonics, 2020, 14(5): 273−284. doi: 10.1038/s41566-019-0532-1 [84] Tait A N, De Lima T F, Zhou E, et al. Neuromorphic photonic networks using silicon photonic weight banks[J]. Sci Rep, 2017, 7(1): 7430. doi: 10.1038/s41598-017-07754-z [85] Mehrabian A, Al-Kabani Y, Sorger V J, et al. PCNNA: a photonic convolutional neural network accelerator[C]//2018 31st IEEE International System-on-Chip Conference (SOCC), 2018: 169–173. https://doi.org/10.1109/SOCC.2018.8618542. [86] Ma P Y, Tait A N, De Lima T F, et al. Photonic independent component analysis using an on-chip microring weight bank[J]. Opt Express, 2020, 28(2): 1827−1844. doi: 10.1364/OE.383603 [87] Bangari V, Marquez B A, Miller H, et al. Digital electronics and analog photonics for convolutional neural networks (DEAP-CNNs)[J]. IEEE J Sel Top Quantum Electron, 2020, 26(1): 7701213. doi: 10.1109/JSTQE.2019.2945540 [88] Sunny F, Mirza A, Nikdast M, et al. CrossLight: a cross-layer optimized silicon photonic neural network accelerator[C]//2021 58th ACM/IEEE Design Automation Conference (DAC), 2021: 1069–1074. https://doi.org/10.1109/DAC18074.2021.9586161. [89] Ohno S, Tang R, Toprasertpong K, et al. Si microring resonator crossbar array for on-chip inference and training of the optical neural network[J]. ACS Photonics, 2022, 9(8): 2614−2622. doi: 10.1021/acsphotonics.1c01777 [90] Xu S F, Wang J, Yi S C, et al. High-order tensor flow processing using integrated photonic circuits[J]. Nat Commun, 2022, 13(1): 7970. doi: 10.1038/s41467-022-35723-2 [91] Bai B W, Yang Q P, Shu H W, et al. Microcomb-based integrated photonic processing unit[J]. Nat Commun, 2023, 14(1): 66. doi: 10.1038/s41467-022-35506-9 [92] Reck M, Zeilinger A, Bernstein H J, et al. Experimental realization of any discrete unitary operator[J]. Phys Rev Lett, 1994, 73(1): 58−61. doi: 10.1103/PhysRevLett.73.58 [93] Clements W R, Humphreys P C, Metcalf B J, et al. Optimal design for universal multiport interferometers[J]. Optica, 2016, 3(12): 1460−1465. doi: 10.1364/OPTICA.3.001460 [94] Shen Y C, Harris N C, Skirlo S, et al. Deep learning with coherent nanophotonic circuits[J]. Nat Photonics, 2017, 11(7): 441−446. doi: 10.1038/nphoton.2017.93 [95] George J K, Nejadriahi H, Sorger V J. Towards on-chip optical FFTs for convolutional neural networks[C]//2017 IEEE International Conference on Rebooting Computing (ICRC), 2017: 1–4. https://doi.org/10.1109/ICRC.2017.8123675. [96] Fang M Y S, Manipatruni S, Wierzynski C, et al. Design of optical neural networks with component imprecisions[J]. Opt Express, 2019, 27(10): 14009−14029. doi: 10.1364/OE.27.014009 [97] Zhang T, Wang J, Dan Y H, et al. Efficient training and design of photonic neural network through neuroevolution[J]. Opt Express, 2019, 27(26): 37150−37163. doi: 10.1364/OE.27.037150 [98] Shokraneh F, Geoffroy-gagnon S, Liboiron-Ladouceur O. The diamond mesh, a phase-error- and loss-tolerant field-programmable MZI-based optical processor for optical neural networks[J]. Opt Express, 2020, 28(16): 23495−23508. doi: 10.1364/OE.395441 [99] Shokraneh F, Geoffroy-Gagnon S, Liboiron-Ladouceur O. Towards phase-error- and loss-tolerant programmable MZI-based optical processors for optical neural networks[C]//2020 IEEE Photonics Conference (IPC), 2020: 1–2. https://doi.org/10.1109/IPC47351.2020.9252466. [100] Tian Y, Zhao Y, Liu S P, et al. Scalable and compact photonic neural chip with low learning-capability-loss[J]. Nanophotonics, 2022, 11(2): 329−344. doi: 10.1515/nanoph-2021-0521 [101] Zhu H H, Zou J, Zhang H, et al. Space-efficient optical computing with an integrated chip diffractive neural network[J]. Nat Commun, 2022, 13(1): 1044. doi: 10.1038/s41467-022-28702-0 [102] Shi Y, Ren J Y, Chen G Y, et al. Nonlinear germanium-silicon photodiode for activation and monitoring in photonic neuromorphic networks[J]. Nat Commun, 2022, 13(1): 6048. doi: 10.1038/s41467-022-33877-7 [103] Wu B, Liu S J, Cheng J W, et al. Real-valued optical matrix computing with simplified MZI mesh[J]. Intell Comput, 2023, 2: 0047. doi: 10.34133/icomputing.0047 [104] Wright C D, Liu Y W, Kohary K I, et al. Arithmetic and biologically-inspired computing using phase-change materials[J]. Adv Mater, 2011, 23(30): 3408−3413. doi: 10.1002/adma.201101060 [105] Kuzum D, Jeyasingh R G D, Lee B, et al. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing[J]. Nano Lett, 2012, 12(5): 2179−2186. doi: 10.1021/nl201040y [106] Cheng Z G, Ríos C, Pernice W H P, et al. On-chip photonic synapse[J]. Sci Adv, 2017, 3(9): e1700160. doi: 10.1126/sciadv.1700160 [107] Chakraborty I, Saha G, Roy K. Photonic in-memory computing primitive for spiking neural networks using phase-change materials[J]. Phys Rev Appl, 2019, 11(1): 014063. doi: 10.1103/PhysRevApplied.11.014063 [108] Feldmann J, Youngblood N, Wright C D, et al. All-optical spiking neurosynaptic networks with self-learning capabilities[J]. Nature, 2019, 569(7755): 208−214. doi: 10.1038/s41586-019-1157-8 [109] Feldmann J, Youngblood N, Karpov M, et al. Parallel convolutional processing using an integrated photonic tensor core[J]. Nature, 2021, 589(7840): 52−58. doi: 10.1038/s41586-020-03070-1 [110] Zhou W, Dong B W, Farmakidis N, et al. In-memory photonic dot-product engine with electrically programmable weight banks[J]. Nat Commun, 2023, 14(1): 2887. doi: 10.1038/s41467-023-38473-x [111] Vandoorne K, Mechet P, Van Vaerenbergh T, et al. Experimental demonstration of reservoir computing on a silicon photonics chip[J]. Nat Commun, 2014, 5(1): 3541. doi: 10.1038/ncomms4541 [112] Xu X Y, Tan M X, Corcoran B, et al. 11 TOPS photonic convolutional accelerator for optical neural networks[J]. Nature, 2021, 589(7840): 44−51. doi: 10.1038/s41586-020-03063-0 [113] Ashtiani F, Geers A J, Aflatouni F. An on-chip photonic deep neural network for image classification[J]. Nature, 2022, 606(7914): 501−506. doi: 10.1038/s41586-022-04714-0 [114] Fu T Z, Zang Y B, Huang Y Y, et al. Photonic machine learning with on-chip diffractive optics[J]. Nat Commun, 2023, 14(1): 70. doi: 10.1038/s41467-022-35772-7 [115] Meng X Y, Zhang G J, Shi N N, et al. Compact optical convolution processing unit based on multimode interference[J]. Nat Commun, 2023, 14(1): 3000. doi: 10.1038/s41467-023-38786-x [116] Lin X, Rivenson Y, Yardimci N T, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406): 1004−1008. doi: 10.1126/science.aat8084 [117] Chang J L, Sitzmann V, Dun X, et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification[J]. Sci Rep, 2018, 8(1): 12324. doi: 10.1038/s41598-018-30619-y [118] Bueno J, Maktoobi S, Froehly L, et al. Reinforcement learning in a large-scale photonic recurrent neural network[J]. Optica, 2018, 5(6): 756−760. doi: 10.1364/OPTICA.5.000756 [119] Lu L D, Zhu L Q, Zhang Q K, et al. Miniaturized diffraction grating design and processing for deep neural network[J]. IEEE Photonics Technol Lett, 2019, 31(24): 1952−1955. doi: 10.1109/LPT.2019.2948626 [120] Yan T, Wu J M, Zhou T K, et al. Fourier-space diffractive deep neural network[J]. Phys Rev Lett, 2019, 123(2): 023901. doi: 10.1103/PhysRevLett.123.023901 [121] Chen H, Feng J N, Jiang M W, et al. Diffractive deep neural networks at visible wavelengths[J]. Engineering, 2021, 7(10): 1483−1491. doi: 10.1016/j.eng.2020.07.032 [122] Zhou T K, Lin X, Wu J M, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit[J]. Nat Photonics, 2021, 15(5): 367−373. doi: 10.1038/s41566-021-00796-w [123] Goi E, Chen X, Zhang Q M, et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip[J]. Light Sci Appl, 2021, 10(1): 40. doi: 10.1038/S41377-021-00483-Z [124] Fujita T, Sakaguchi H, Zhang J, et al. Magneto-optical diffractive deep neural network[J]. Opt Express, 2022, 30(20): 36889−36899. doi: 10.1364/OE.470513 [125] Duan Z Y, Chen H, Lin X. Optical multi-task learning using multi-wavelength diffractive deep neural networks[J]. Nanophotonics, 2023, 12(5): 893−903. doi: 10.1515/nanoph-2022-0615 [126] Chen Y T, Nazhamaiti M, Xu H, et al. All-analog photoelectronic chip for high-speed vision tasks[J]. Nature, 2023, 623(7985): 48−57. doi: 10.1038/s41586-023-06558-8 [127] Zuo Y, Li B H, Zhao Y J, et al. All-optical neural network with nonlinear activation functions[J]. Optica, 2019, 6(9): 1132−1137. doi: 10.1364/OPTICA.6.001132 [128] Hamerly R, Bernstein L, Sludds A, et al. Large-scale optical neural networks based on photoelectric multiplication[J]. Phys Rev X, 2019, 9(2): 021032. doi: 10.1103/PhysRevX.9.021032 [129] Sludds A, Bernstein L, Hamerly R, et al. A scalable optical neural network architecture using coherent detection[J]. Proc SPIE, 2020, 11299: 112990H. doi: 10.1117/12.2546940 [130] Rafayelyan M, Dong J, Tan Y Q, et al. Large-scale optical reservoir computing for spatiotemporal chaotic systems prediction[J]. Phys Rev X, 2020, 10(4): 041037. doi: 10.1103/PhysRevX.10.041037 [131] Xu Z H, Zhou T K, Ma M Z, et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence[J]. Science, 2024, 384(6692): 202−209. doi: 10.1126/science.adl1203 [132] Qian C, Lin X, Lin X B, et al. Performing optical logic operations by a diffractive neural network[J]. Light Sci Appl, 2020, 9(1): 59. doi: 10.1038/s41377-020-0303-2 [133] Wu C M, Yu H S, Lee S, et al. Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network[J]. Nat Commun, 2021, 12(1): 96. doi: 10.1038/s41467-020-20365-z [134] Liu C, Ma Q, Luo Z J, et al. A programmable diffractive deep neural network based on a digital-coding metasurface array[J]. Nat Electron, 2022, 5(2): 113−122. doi: 10.1038/s41928-022-00719-9 [135] Gu J Q, Zhao Z, Feng C H, et al. ROQ: a noise-aware quantization scheme towards robust optical neural networks with low-bit controls[C]//2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020: 1586–1589. https://doi.org/10.23919/DATE48585.2020.9116521. [136] Mourgias-Alexandris G, Moralis-Pegios M, Tsakyridis A, et al. Noise-resilient and high-speed deep learning with coherent silicon photonics[J]. Nat Commun, 2022, 13(1): 5572. doi: 10.1038/s41467-022-33259-z [137] Kirtas M, Oikonomou A, Passalis N, et al. Quantization-aware training for low precision photonic neural networks[J]. Neural Networks, 2022, 155: 561−573. doi: 10.1016/j.neunet.2022.09.015 [138] Feng C H, Gu J Q, Zhu H Q, et al. A compact butterfly-style silicon photonic–electronic neural chip for hardware-efficient deep learning[J]. ACS Photonics, 2022, 9(12): 3906−3916. doi: 10.1021/acsphotonics.2c01188 [139] Zhan Y C, Zhang H, Lin H X, et al. Physics-aware analytic-gradient training of photonic neural networks[J]. Laser Photonics Rev, 2024, 18(4): 2300445. doi: 10.1002/lpor.202300445 [140] Hughes T W, Minkov M, Shi Y, et al. Training of photonic neural networks through in situ backpropagation and gradient measurement[J]. Optica, 2018, 5(7): 864−871. doi: 10.1364/OPTICA.5.000864 [141] Zhou T K, Fang L, Yan T, et al. In situ optical backpropagation training of diffractive optical neural networks[J]. Photonics Res, 2020, 8(6): 940−953. doi: 10.1364/PRJ.389553 [142] Zheng Z Y, Duan Z Y, Chen H, et al. Dual adaptive training of photonic neural networks[J]. Nat Mach Intell, 2023, 5(10): 1119−1129. doi: 10.1038/s42256-023-00723-4 [143] Wu T W, Menarini M, Gao Z H, et al. Lithography-free reconfigurable integrated photonic processor[J]. Nat Photonics, 2023, 17(8): 710−716. doi: 10.1038/s41566-023-01205-0 [144] Pai S, Sun Z H, Hughes T W, et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks[J]. Science, 2023, 380(6643): 398−404. doi: 10.1126/science.ade8450 -
Overview
The era of big data has placed greater demands on the computing power and speed of electronic computer processing systems. Issues such as the "memory wall" and "power wall" inherent in the traditional von Neumann architecture, coupled with the slowing down or even invalidation of Moore's Law, have posed significant challenges to electronic chips in terms of computing speed and power consumption. Utilizing optical computing as an alternative to traditional electronic computing represents one of the most promising avenues to address current challenges in computing power and power consumption.
This review systematically summarized the research progress of optical neural network (ONN) architectures and algorithms in both on-chip integration and in free space, and described typical research efforts in detail. In terms of on-chip integrated ONNs, the research progress of ONNs based on semiconductor lasers, silicon micro-ring resonators, Mach-Zehnder interferometers, and phase change materials was presented. Meanwhile, progress in research on free-space-based ONNs, including diffractive deep neural networks and metasurface-based ONNs, was summarized. Then, the advantages and disadvantages of these two types of ONNs were discussed and compared. The free-space-based ONNs have excellent parallel computing capabilities and are suitable for large-scale computing tasks. But they suffer from large volume and high complexity. In contrast, on-chip integrated ONNs have the advantages of scalability, high power efficiency, compact footprint, and high programmability. However, how to ensure accuracy and robustness in the process of large-scale integration to better cope with increasingly complex and large-scale computing tasks is still an urgent problem to be solved. In addition, training is an important step in the construction of neural networks and determines the performance of the entire system. Therefore, the research progress of the in-situ training method and the hardware-aware offline training method used in ONNs was introduced.
At last, the potential challenges that ONNs may encounter were discussed in depth, and a forward-looking perspective on their future development was offered. From the material and devices, to the system architecture, and ONNs are presenting a multi-level, cross-domain, and comprehensive development pattern for the algorithm implementation. By thoroughly exploring the potential of photon properties and deeply integrating them with artificial intelligence algorithms, the broad prospects and infinite possibilities of ONNs in building new intelligent computing systems can be demonstrated. Advances in ONNs can promote the development of the new computing paradigm of photonic brain-like computing, leading computing technology toward a more efficient and intelligent future.
-
Access History
Figures(12)
Tables(1)
Article Metrics
Export File
Citation
Xiang S Y, Song Z W, Zhang Y H, et al. Progress in the research of optical neural networks[J]. Opto-Electron Eng, 2024, 51(7): 240101. doi: 10.12086/oee.2024.240101
Format
Content
DownLoad:
-
Figure 1.
The development history of artificial intelligence
-
Figure 2.
National strategy overview and hardware implementation of brain-like intelligence[5-12,15-23]
- Figure 3.
-
Figure 4.
Schematic diagram of a 4×4 MRR array
-
Figure 5.
Research progress in photonic neural networks based on MRR arrays[84-85,87-91]
-
Figure 6.
4×4 MZI network. (a) Triangle structure; (b) Rectangular structure
-
Figure 7.
Research progress in photonic neural networks based on MZI mesh[94,96-97,99-101,103]
-
Figure 8.
Photonic neural network based on PCM. (a) Principle and experiment of all-optical spiking neurosynaptic network[108]; (b) Integrated photonic hardware accelerator architecture based on photonic tensor cores[109]; (c) In-memory photonic–electronic computing platform based on non-volatile electronically reprogrammable PCM memory cells[110]
-
Figure 9.
Research progress in diffractive optical neural network[116-125]
-
Figure 10.
Photonic neural network based on metasurfaces. (a) The principle and experiment of optical logic operations performed by a diffractive neural network based on a compound Huygens’ metasurface[132]; (b) Optical convolutional neural network based on the phase-change metasurface mode converter as a photonic computing core[133]; (c) A programmable diffractive deep neural network based on a multi-layer digital-coding metasurface array[134]
-
Figure 11.
Research progress in training optical neural networks[89,135-144]
- Figure .
