近年來，針對低功耗終端裝置，許多研究提出了基於脈衝神經網路 (SNN) 的高效能硬體，這類硬體研究集中在實現預訓練模型的推論功能；然而，預訓練模型無法適應不斷變化的環境，因此需要使用本地資料進行重新訓練，神經網絡的訓練需要大量高精度的運算，對於具有功耗限制的終端裝置來說，執行訓練過程是相對困難的，此外，如果將用戶資料上傳雲端伺服器進行重新訓練，可能會有安全及隱私的疑慮。
因此，本論文提出了一個基於卷積SNN的高效能學習系統，該系統可以有效率地執行空間時間反向傳播 (STBP) 演算法，首先，我們針對STBP演算法提出了四個改善方案，使其更容易被硬體實現，且可以達到比原始演算法更高的準確性、更少的脈衝數量和更簡單的反向傳播運算；此外，我們將STBP演算法的浮點數運算量化為低精度整數運算，以在不影響模型準確性的情況下，最小化硬體實現成本；最後，基於改良的STBP演算法，我們提出了一個整合三個訓練階段 (前向、反向及權重更新) 的硬體架構，可以有效地處理整個STBP演算法，利用前向和反向運算的輸入資料稀疏性，減少整體運算能耗和延遲，此架構還具有高度可擴展性，可以組合多個核心支援更寬更深的SNN模型。
我們使用40奈米製程實作了基於該架構的晶片，量測結果顯示，在90％的輸入資料稀疏性下，此晶片可以達到的訓練與推論每瓦每秒運算效率分別為3TOPS/W和7.7TOPS/W，以及達到每個突觸運算僅1.89pJ/SOP的能耗；我們也開發了一個包含多個晶片的系統，我們使用此系統進行SNN訓練，在MNIST資料集上達到了99.1％的準確性，且在SVHN和CIFAR10資料集上展示了首個晶片上訓練的結果，此外，此系統的能源效率比典型商用GPU平台的能源效率平均高了36倍。

In recent years, there has been extensive research on energy-efficient spiking neural network (SNN) hardware for low-power end-point devices. Most hardware studies are about inference-only engines, but the pre-trained models cannot adapt to changing environments. Therefore, there is a growing need for re-training with locally collected data. Training neural networks requires high numerical precision and computing power, so it is arduous for end-point devices with constrained power budgets to perform the training process. In addition, it is also unsuitable and insecure to disclose sensitive user data for cloud-based re-training.
Therefore, this dissertation introduces an energy-efficient convolutional SNN learning system incorporating the scalable hardware architecture and the spatio-temporal back-propagation (STBP) algorithm. We modify the STBP algorithm using four hardware-based enhancing schemes to attain higher accuracy, lower spike counts, and more simple back-propagation compared with the original STBP algorithm. We also convert floating-point operations of the STBP algorithm to low-bitwidth integer operations, in order to minimize hardware costs without sacrificing classification accuracy. Building upon the modified STBP algorithm, we propose a unified hardware architecture encompassing three data-flow modes, i.e., the \emph{forward}, \emph{backward}, and \emph{update} modes, to efficiently process the entire algorithm. Additionally, the architecture exploits input sparsity for both forward and backward computations to reduce the overall processing energy and delay. It also provides multi-core scalability to accommodate wider and deeper models in a multi-core system.
A 40-nm prototype chip has been implemented, achieving peak training and inference efficiencies of 3TOPS/W and 7.7TOPS/W, respectively, at 90\% input sparsity and an energy per synaptic operation (SOP) metric of 1.89pJ/SOP. Based on the chip, our multi-core prototype system achieves a competitive accuracy of 99.1\% on MNIST. We also present the first on-chip training results on SVHN and CIFAR10, demonstrating an average of 36$\times$ improvement in efficiency compared with a typical commercial GPU platform.

[1] Y. Wu, L. Deng, G. Li, J. Zhu, Y. Xie, and L. Shi, “Direct training for spiking
neural networks: Faster, larger, better,” in Proceedings of the AAAI Conference
on Artificial Intelligence, vol. 33, Jul. 2019, pp. 1311–1318. [Online]. Available:
https://ojs.aaai.org/index.php/AAAI/article/view/3929
[2] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv, 2018.
[3] OpenAI, “Gpt-4 technical report,” 2023.
[4] M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi,
N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul,
J. Tse, G. Venkataramanan, Y.-H. Weng, A. Wild, Y. Yang, and H. Wang, “Loihi: A
neuromorphic manycore processor with on-chip learning,” IEEE Micro, vol. 38, no. 1,
pp. 82–99, 2018.
[5] A. Basu, L. Deng, C. Frenkel, and X. Zhang, “Spiking neural network integrated circuits:
A review of trends and future directions,” in 2022 IEEE Custom Integrated Circuits Conference
(CICC), 2022, pp. 1–8.
[6] G. K. Chen, R. Kumar, H. E. Sumbul, P. C. Knag, and R. K. Krishnamurthy, “A 4096-
neuron 1m-synapse 3.8pj/sop spiking neural network with on-chip stdp learning and
sparse weights in 10nm finfet cmos,” in 2018 IEEE Symposium on VLSI Circuits, 2018,
pp. 255–256.
[7] C. Frenkel, J.-D. Legat, and D. Bol, “Morphic: A 65-nm 738k-synapse/mm2 quad-core
binary-weight digital neuromorphic processor with stochastic spike-driven online learning,”
IEEE Transactions on Biomedical Circuits and Systems, vol. 13, no. 5, pp. 999–
1010, 2019.
[8] S. K. Esser, P. A. Merolla, J. V. Arthur, A. S. Cassidy, R. Appuswamy, A. Andreopoulos,
D. J. Berg, J. L. McKinstry, T. Melano, D. R. Barch, C. di Nolfo, P. Datta,
A. Amir, B. Taba, M. D. Flickner, and D. S. Modha, “Convolutional networks
for fast, energy-efficient neuromorphic computing,” Proceedings of the National
Academy of Sciences, vol. 113, no. 41, pp. 11 441–11 446, 2016. [Online]. Available:
https://www.pnas.org/doi/abs/10.1073/pnas.1604850113
[9] L. Deng, G. Wang, G. Li, S. Li, L. Liang, M. Zhu, Y. Wu, Z. Yang, Z. Zou, J. Pei, Z. Wu,
X. Hu, Y. Ding, W. He, Y. Xie, and L. Shi, “Tianjic: A unified and scalable chip bridging
spike-based and continuous neural computation,” IEEE Journal of Solid-State Circuits,
vol. 55, no. 8, pp. 2228–2246, 2020.
[10] S. Moradi, N. Qiao, F. Stefanini, and G. Indiveri, “A scalable multicore architecture with
heterogeneous memory structures for dynamic neuromorphic asynchronous processors
(dynaps),” IEEE Transactions on Biomedical Circuits and Systems, vol. 12, no. 1, pp.
106–122, 2018.
[11] V. P. Nambiar, J. Pu, Y. K. Lee, A. Mani, T. Luo, L. Yang, E. K. Koh, M. M. Wong, F. Li,
W. L. Goh, and A. T. Do, “0.5v 4.8 pj/sop 0.93uw leakage/core neuromorphic processor
with asynchronous noc and reconfigurable lif neuron,” in 2020 IEEE Asian Solid-State
Circuits Conference (A-SSCC), 2020, pp. 1–4.
[12] S.-G. Cho, E. Beigné, and Z. Zhang, “A 2048-neuron spiking neural network accelerator
with neuro-inspired pruning and asynchronous network on chip in 40nm cmos,” in 2019
IEEE Custom Integrated Circuits Conference (CICC), 2019, pp. 1–4.
[13] J. Zhang, D. Huo, J. Zhang, C. Qian, Q. Liu, L. Pan, Z. Wang, N. Qiao, K.-T. Tang, and
H. Chen, “22.6 anp-i: A 28nm 1.5pj/sop asynchronous spiking neural network processor
enabling sub-o.1 μj/sample on-chip learning for edge-ai applications,” in 2023 IEEE
International Solid- State Circuits Conference (ISSCC), 2023, pp. 21–23.
[14] C. Frenkel and G. Indiveri, “Reckon: A 28nm sub-mm2 task-agnostic spiking recurrent
neural network processor enabling on-chip learning over second-long timescales,” in
2022 IEEE International Solid- State Circuits Conference (ISSCC), vol. 65, 2022, pp.
1–3.
[15] J. Park, J. Lee, and D. Jeon, “7.6 a 65nm 236.5nj/classification neuromorphic processor
with 7.5% energy overhead on-chip learning using direct spike-only feedback,” in 2019
IEEE International Solid- State Circuits Conference - (ISSCC), 2019, pp. 140–142.
[16] C. Frenkel, J.-D. Legat, and D. Bol, “A 28-nm convolutional neuromorphic processor enabling
online learning with spike-based retinas,” in 2020 IEEE International Symposium
on Circuits and Systems (ISCAS), 2020, pp. 1–5.
[17] J. K. Kim, P. Knag, T. Chen, and Z. Zhang, “A 640m pixel/s 3.65mw sparse event-driven
neuromorphic object recognition processor with on-chip learning,” in 2015 Symposium
on VLSI Circuits (VLSI Circuits), 2015, pp. C50–C51.
[18] P. Knag, J. K. Kim, T. Chen, and Z. Zhang, “A sparse coding neural network asic with onchip
learning for feature extraction and encoding,” IEEE Journal of Solid-State Circuits,
vol. 50, no. 4, pp. 1070–1079, 2015.
[19] C. Frenkel, M. Lefebvre, J.-D. Legat, and D. Bol, “A 0.086-mm2 12.7-pj/sop 64ksynapse
256-neuron online-learning digital spiking neuromorphic processor in 28-nm
cmos,” IEEE Transactions on Biomedical Circuits and Systems, vol. 13, no. 1, pp. 145–
158, 2019.
[20] D. Wang, P. K. Chundi, S. J. Kim, M. Yang, J. P. Cerqueira, J. Kang, S. Jung, S. Kim, and
M. Seok, “Always-on, sub-300-nw, event-driven spiking neural network based on spikedriven
clock-generation and clock- and power-gating for an ultra-low-power intelligent
device,” in 2020 IEEE Asian Solid-State Circuits Conference (A-SSCC), 2020, pp. 1–4.
[21] P.-Y. Chuang, P.-Y. Tan, C.-W. Wu, and J.-M. Lu, “A 90nm 103.14 tops/w binaryweight
spiking neural network cmos asic for real-time object classification,” in 2020
57th ACM/IEEE Design Automation Conference (DAC), 2020, pp. 1–6.
[22] J. L. Molin, A. Eisape, C. S. Thakur, V. Varghese, C. Brandli, and R. Etienne-Cummings,
“Low-power, low-mismatch, highly-dense array of vlsi mihalas-niebur neurons,” in 2017
IEEE International Symposium on Circuits and Systems (ISCAS), 2017, pp. 1–4.
[23] B. Yan, Q. Yang, W.-H. Chen, K.-T. Chang, J.-W. Su, C.-H. Hsu, S.-H. Li, H.-Y. Lee,
S.-S. Sheu, M.-S. Ho, Q. Wu, M.-F. Chang, Y. Chen, and H. Li, “Rram-based spiking
nonvolatile computing-in-memory processing engine with precision-configurable in situ
nonlinear activation,” in 2019 Symposium on VLSI Technology, 2019, pp. T86–T87.
[24] A. Neckar, S. Fok, B. V. Benjamin, T. C. Stewart, N. N. Oza, A. R. Voelker, C. Eliasmith,
R. Manohar, and K. Boahen, “Braindrop: A mixed-signal neuromorphic architecture
with a dynamical systems-based programming model,” Proceedings of the IEEE, vol.
107, no. 1, pp. 144–164, 2019.
[25] F. N. Buhler, P. Brown, J. Li, T. Chen, Z. Zhang, and M. P. Flynn, “A 3.43tops/w
48.9pj/pixel 50.1nj/classification 512 analog neuron sparse coding neural network with
on-chip learning and classification in 40nm cmos,” in 2017 Symposium on VLSI Circuits,
2017, pp. C30–C31.
[26] S. Brink, S. Nease, P. Hasler, S. Ramakrishnan, R. Wunderlich, A. Basu, and B. Degnan,
“A learning-enabled neuron array ic based upon transistor channel models of biological
phenomena,” IEEE Transactions on Biomedical Circuits and Systems, vol. 7, no. 1, pp.
71–81, 2013.
[27] C. Mayr, J. Partzsch, M. Noack, S. Hänzsche, S. Scholze, S. Höppner, G. Ellguth, and
R. Schüffny, “A biological-realtime neuromorphic system in 28 nm cmos using lowleakage
switched capacitor circuits,” IEEE Transactions on Biomedical Circuits and Systems,
vol. 10, no. 1, pp. 243–254, 2016.
[28] P.-Y. Tan, C.-W. Wu, and J.-M. Lu, “An improved stbp for training high-accuracy and
low-spike-count spiking neural networks,” in 2021 Design, Automation & Test in Europe
Conference & Exhibition (DATE), 2021, pp. 575–580.
[29] P.-Y. Tan and C.-W. Wu, “A low-bitwidth integer-stbp algorithm for efficient training
and inference of spiking neural networks,” in Proceedings of the 28th Asia and
South Pacific Design Automation Conference, ser. ASPDAC ’23. New York, NY,
USA: Association for Computing Machinery, 2023, p. 651–656. [Online]. Available:
https://doi.org/10.1145/3566097.3567875
[30] ——, “A 40-nm 1.89-pj/sop scalable convolutional spiking neural network learning core
with on-chip spatiotemporal back-propagation,” IEEE Transactions on Very Large Scale
Integration (VLSI) Systems, vol. 31, no. 12, pp. 1994–2007, 2023.
[31] M. Taylor et al., “The problem of stimulus structure in the behavioural theory of perception,”
South African Journal of Psychology, vol. 3, pp. 23–45, 1973.
[32] W. Levy and O. Steward, “Temporal contiguity requirements for long-term associative
potentiation/depression in the hippocampus,” Neuroscience, vol. 8, no. 4, pp.
791–797, 1983. [Online]. Available: https://www.sciencedirect.com/science/article/pii/
0306452283900106
[33] Y. Dan and M. ming Poo, “Hebbian depression of isolated neuromuscular synapses
in vitro,” Science, vol. 256, no. 5063, pp. 1570–1573, 1992. [Online]. Available:
https://www.science.org/doi/abs/10.1126/science.1317971
[34] D. Debanne, B. H. Gähwiler, and S. M. Thompson, “Asynchronous pre- and postsynaptic
activity induces associative long-term depression in area ca1 of the rat hippocampus in
vitro.” Proceedings of the National Academy of Sciences, vol. 91, no. 3, pp. 1148–1152,
1994. [Online]. Available: https://www.pnas.org/doi/abs/10.1073/pnas.91.3.1148
[35] H. Markram, J. Lübke, M. Frotscher, and B. Sakmann, “Regulation of synaptic efficacy
by coincidence of postsynaptic aps and epsps,” Science, vol. 275, no. 5297, pp.
213–215, 1997. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.
275.5297.213
[36] G. qiang Bi and M. ming Poo, “Synaptic modifications in cultured hippocampal
neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type,”
Journal of Neuroscience, vol. 18, no. 24, pp. 10 464–10 472, 1998. [Online]. Available:
https://www.jneurosci.org/content/18/24/10464
[37] Y. Hao, X. Huang, M. Dong, and B. Xu, “A biologically plausible supervised learning
method for spiking neural networks using the symmetric stdp rule,” Neural Networks,
vol. 121, pp. 387–395, 2020. [Online]. Available: https://www.sciencedirect.com/
science/article/pii/S0893608019302680
[38] R. V. Florian, “Reinforcement learning through modulation of spike-timing-dependent
synaptic plasticity,” Neural computation, vol. 19, no. 6, pp. 1468–1502, 2007.
[39] J. Wu, E. Yılmaz, M. Zhang, H. Li, and K. C. Tan, “Deep spiking neural networks
for large vocabulary automatic speech recognition,” Frontiers in Neuroscience,
vol. 14, 2020. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.
2020.00199
[40] Q. Chen, C. Sun, C. Gao, X. Fang, and H. Luan, “Framefire: Enabling efficient spiking
neural network inference for video segmentation,” in 2023 IEEE 5th International
Conference on Artificial Intelligence Circuits and Systems (AICAS), 2023, pp. 1–5.
[41] J. Pei, L. Deng, S. Song, M. Zhao, Y. Zhang, S. Wu, G. Wang, Z. Zou, Z. Wu, W. He et al.,
“Towards artificial general intelligence with hybrid tianjic chip architecture,” Nature,
vol. 572, no. 7767, pp. 106–111, 2019.
[42] K. Kumarasinghe, N. Kasabov, and D. Taylor, “Brain-inspired spiking neural networks
for decoding and understanding muscle activity and kinematics from electroencephalography
signals during hand movements,” Scientific reports, vol. 11, no. 1, p. 2486, 2021.
[43] Y. Zeng, D. Zhao, F. Zhao, G. Shen, Y. Dong, E. Lu, Q. Zhang, Y. Sun, Q. Liang, Y. Zhao,
Z. Zhao, H. Fang, Y. Wang, Y. Li, X. Liu, C. Du, Q. Kong, Z. Ruan, and W. Bi, “Braincog:
A spiking neural network based, brain-inspired cognitive intelligence engine for braininspired
ai and brain simulation,” Patterns, vol. 4, no. 8, p. 100789, 2023. [Online].
Available: https://www.sciencedirect.com/science/article/pii/S2666389923001447
[44] R. Batllori, C. Laramee, W. Land, and J. Schaffer, “Evolving spiking neural networks
for robot control,” Procedia Computer Science, vol. 6, pp. 329–334, 2011, complex
adaptive sysytems. [Online]. Available: https://www.sciencedirect.com/science/article/
pii/S1877050911005254
[45] M. Hopkins, G. Pineda-Garcia, P. A. Bogdan, and S. B. Furber, “Spiking neural networks
for computer vision,” Interface Focus, vol. 8, no. 4, p. 20180007, 2018.
[46] Y. Li, H. Shen, and D. Hu, “A spiking neural network for brain-computer interface of
four classes motor imagery,” in Human Brain and Artificial Intelligence, X. Ying, Ed.
Singapore: Springer Nature Singapore, 2023, pp. 148–160.
[47] G. Zhan, Z. Song, T. Fang, Y. Zhang, S. Le, X. Zhang, S. Wang, Y. Lin, J. Jia, L. Zhang,
and X. Kang, “Applications of spiking neural network in brain computer interface,” in
2021 9th International Winter Conference on Brain-Computer Interface (BCI), 2021, pp.
1–6.
[48] F. Rosenblatt, “The perceptron: a probabilistic model for information storage and organization
in the brain.” Psychological review, vol. 65, no. 6, pp. 386–408, 1958.
[49] S. Haykin, Neural networks: a comprehensive foundation. Prentice Hall PTR, 1998.
[50] L. Lapicque, “Recherches quantitatives sur l’excitation electrique des nerfs traitee
comme une polarization,” Journal de Physiologie et de Pathologie generale, vol. 9, pp.
620–635, 1907.
[51] A. L. Hodgkin and A. F. Huxley, “A quantitative description of membrane current and its
application to conduction and excitation in nerve,” The Journal of physiology, vol. 117,
no. 4, pp. 500–544, 1952.
[52] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Transactions on neural networks,
vol. 14, no. 6, pp. 1569–1572, 2003.
[53] L. R. Medsker and L. Jain, “Recurrent neural networks,” Design and Applications, vol. 5,
no. 64-67, p. 2, 2001.
[54] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,
vol. 9, no. 8, pp. 1735–1780, 1997.
[55] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and
Y. Bengio, “Learning phrase representations using rnn encoder-decoder for statistical
machine translation,” arXiv preprint arXiv:1406.1078, 2014.
[56] P. Diehl and M. Cook, “Unsupervised learning of digit recognition using spike-timingdependent
plasticity,” Frontiers in Computational Neuroscience, vol. 9, 2015. [Online].
Available: https://www.frontiersin.org/articles/10.3389/fncom.2015.00099
[57] S. R. Kheradpisheh, M. Ganjtabesh, S. J. Thorpe, and T. Masquelier, “Stdp-based
spiking deep convolutional neural networks for object recognition,” Neural Networks,
vol. 99, pp. 56–67, 2018. [Online]. Available: https://www.sciencedirect.com/science/
article/pii/S0893608017302903
[58] P. Falez, P. Tirilly, and I. M. Bilasco, “Improving stdp-based visual feature learning with
whitening,” in 2020 International Joint Conference on Neural Networks (IJCNN), 2020,
pp. 1–8.
[59] C. Lee, G. Srinivasan, P. Panda, and K. Roy, “Deep spiking convolutional neural network
trained with unsupervised spike-timing-dependent plasticity,” IEEE Transactions
on Cognitive and Developmental Systems, vol. 11, no. 3, pp. 384–394, 2019.
[60] J. Liu, H. Huo, W. Hu, and T. Fang, “Brain-inspired hierarchical spiking neural network
using unsupervised stdp rule for image classification,” in Proceedings of the 2018 10th
International Conference on Machine Learning and Computing, ser. ICMLC ’18. New
York, NY, USA: Association for Computing Machinery, 2018, p. 230–235. [Online].
Available: https://doi.org/10.1145/3195106.3195115
[61] P. Ferré, F. Mamalet, and S. J. Thorpe, “Unsupervised feature learning with winnertakes-
all based stdp,” Frontiers in Computational Neuroscience, vol. 12, 2018. [Online].
Available: https://www.frontiersin.org/articles/10.3389/fncom.2018.00024
[62] A. Tavanaei and A. S. Maida, “Bio-inspired spiking convolutional neural network
using layer-wise sparse coding and STDP learning,” CoRR, vol. abs/1611.03000, 2016.
[Online]. Available: http://arxiv.org/abs/1611.03000
[63] N. Caporale and Y. Dan, “Spike timing–dependent plasticity: A hebbian learning rule,”
Annual Review of Neuroscience, vol. 31, no. 1, pp. 25–46, 2008, pMID: 18275283.
[Online]. Available: https://doi.org/10.1146/annurev.neuro.31.060407.125639
[64] P. U. Diehl, D. Neil, J. Binas, M. Cook, S.-C. Liu, and M. Pfeiffer, “Fast-classifying,
high-accuracy spiking deep networks through weight and threshold balancing,” in 2015
International Joint Conference on Neural Networks (IJCNN), 2015, pp. 1–8.
[65] B. Rueckauer, I.-A. Lungu, Y. Hu, M. Pfeiffer, and S.-C. Liu, “Conversion
of continuous-valued deep networks to efficient event-driven networks for image
classification,” Frontiers in Neuroscience, vol. 11, 2017. [Online]. Available:
https://www.frontiersin.org/articles/10.3389/fnins.2017.00682
[66] A. Sengupta, Y. Ye, R. Wang, C. Liu, and K. Roy, “Going deeper in spiking neural
networks: Vgg and residual architectures,” Frontiers in Neuroscience, vol. 13, 2019.
[Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.2019.00095
[67] B. Han, G. Srinivasan, and K. Roy, “Rmp-snn: Residual membrane potential neuron for
enabling deeper high-accuracy and low-latency spiking neural network,” in Proceedings
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June
2020.
[68] T. Bu, J. Ding, Z. Yu, and T. Huang, “Optimized potential initialization for
low-latency spiking neural networks,” Proceedings of the AAAI Conference on
Artificial Intelligence, vol. 36, no. 1, pp. 11–20, Jun. 2022. [Online]. Available:
https://ojs.aaai.org/index.php/AAAI/article/view/19874
[69] Y. Li, S. Deng, X. Dong, R. Gong, and S. Gu, “A free lunch from ann: Towards efficient,
accurate spiking neural networks calibration,” in Proceedings of the 38th International
Conference on Machine Learning, ser. Proceedings of Machine Learning Research,
M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 6316–6325.
[Online]. Available: https://proceedings.mlr.press/v139/li21d.html
[70] Y. Hu, H. Tang, and G. Pan, “Spiking deep residual networks,” IEEE Transactions on
Neural Networks and Learning Systems, vol. 34, no. 8, pp. 5200–5205, 2023.
[71] C. Stöckl and W. Maass, “Optimized spiking neurons can classify images with high accuracy
through temporal coding with two spikes,” Nature Machine Intelligence, vol. 3,
no. 3, pp. 230–238, 2021.
[72] S. M. Bohte, J. N. Kok, and H. La Poutré, “Error-backpropagation in temporally encoded
networks of spiking neurons,” Neurocomputing, vol. 48, no. 1, pp. 17–37, 2002. [Online].
Available: https://www.sciencedirect.com/science/article/pii/S0925231201006580
[73] F. Zenke and S. Ganguli, “SuperSpike: Supervised Learning in Multilayer Spiking
Neural Networks,” Neural Computation, vol. 30, no. 6, pp. 1514–1541, 06 2018.
[Online]. Available: https://doi.org/10.1162/neco_a_01086
[74] A. Tavanaei and A. Maida, “Bp-stdp: Approximating backpropagation using spike
timing dependent plasticity,” Neurocomputing, vol. 330, pp. 39–47, 2019. [Online].
Available: https://www.sciencedirect.com/science/article/pii/S0925231218313420
[75] J. H. Lee, T. Delbruck, and M. Pfeiffer, “Training deep spiking neural networks
using backpropagation,” Frontiers in Neuroscience, vol. 10, 2016. [Online]. Available:
https://www.frontiersin.org/articles/10.3389/fnins.2016.00508
[76] Y. Jin, W. Zhang, and P. Li, “Hybrid macro/micro level backpropagation
for training deep spiking neural networks,” in Advances in Neural Information
Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman,
N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc.,
2018. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2018/file/
3fb04953d95a94367bb133f862402bce-Paper.pdf
[77] C. Lee, S. S. Sarwar, P. Panda, G. Srinivasan, and K. Roy, “Enabling spikebased
backpropagation for training deep neural network architectures,” Frontiers in
Neuroscience, vol. 14, 2020. [Online]. Available: https://www.frontiersin.org/articles/
10.3389/fnins.2020.00119
[78] Y. Wu, L. Deng, G. Li, J. Zhu, and L. Shi, “Spatio-temporal backpropagation for training
high-performance spiking neural networks,” Frontiers in Neuroscience, vol. 12, 2018.
[Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.2018.00331
[79] W. Severa, C. M. Vineyard, R. Dellana, S. J. Verzi, and J. B. Aimone, “Training deep
neural networks for binary communication with the whetstone method,” Nature Machine
Intelligence, vol. 1, no. 2, pp. 86–94, 2019.
[80] W. Fang, Z. Yu, Y. Chen, T. Masquelier, T. Huang, and Y. Tian, “Incorporating learnable
membrane time constant to enhance learning of spiking neural networks,” in Proceedings
of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2021,
pp. 2661–2671.
[81] H. Zheng, Y. Wu, L. Deng, Y. Hu, and G. Li, “Going deeper with directly-trained
larger spiking neural networks,” Proceedings of the AAAI Conference on Artificial
Intelligence, vol. 35, no. 12, pp. 11 062–11 070, May 2021. [Online]. Available:
https://ojs.aaai.org/index.php/AAAI/article/view/17320
[82] Q. Meng, M. Xiao, S. Yan, Y. Wang, Z. Lin, and Z.-Q. Luo, “Training high-performance
low-latency spiking neural networks by differentiation on spike representation,” in Proceedings
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), June 2022, pp. 12 444–12 453.
[83] W. Fang, Z. Yu, Y. Chen, T. Huang, T. Masquelier, and Y. Tian, “Deep
residual learning in spiking neural networks,” in Advances in Neural Information
Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and
J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 21 056–
21 069. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2021/file/
afe434653a898da20044041262b3ac74-Paper.pdf
[84] Y. Hu, L. Deng, Y. Wu, M. Yao, and G. Li, “Advancing spiking neural networks towards
deep residual learning,” 2023.
[85] M. Yao, G. Zhao, H. Zhang, Y. Hu, L. Deng, Y. Tian, B. Xu, and G. Li, “Attention spiking
neural networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 45, no. 8, pp. 9393–9410, 2023.
[86] Z. Zhou, Y. Zhu, C. He, Y. Wang, S. YAN, Y. Tian, and L. Yuan, “Spikformer: When
spiking neural network meets transformer,” in The Eleventh International Conference
on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?
id=frE4fUwz_h
[87] M. Yao, H. Gao, G. Zhao, D. Wang, Y. Lin, Z. Yang, and G. Li, “Temporal-wise attention
spiking neural networks for event streams classification,” in Proceedings of the
IEEE/CVF International Conference on Computer Vision (ICCV), October 2021, pp.
10 221–10 230.
[88] W. He, Y. Wu, L. Deng, G. Li, H. Wang, Y. Tian, W. Ding, W. Wang, and
Y. Xie, “Comparing snns and rnns on neuromorphic vision datasets: Similarities
and differences,” Neural Networks, vol. 132, pp. 108–120, 2020. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S0893608020302902
[89] S. B. Shrestha and G. Orchard, “Slayer: Spike layer error reassignment in time,”
in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach,
H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran
Associates, Inc., 2018. [Online]. Available: https://proceedings.neurips.cc/paper_files/
paper/2018/file/82f2b308c3b01637c607ce05f52a2fed-Paper.pdf
[90] N. Rathi and K. Roy, “Diet-snn: A low-latency spiking neural network with direct input
encoding and leakage and threshold optimization,” IEEE Transactions on Neural Networks
and Learning Systems, vol. 34, no. 6, pp. 3174–3182, 2023.
[91] N. Rathi, G. Srinivasan, P. Panda, and K. Roy, “Enabling deep spiking neural
networks with hybrid conversion and spike timing dependent backpropagation,” in
International Conference on Learning Representations, 2020. [Online]. Available:
https://openreview.net/forum?id=B1xSperKvH
[92] C. Lee, P. Panda, G. Srinivasan, and K. Roy, “Training deep spiking convolutional
neural networks with stdp-based unsupervised pre-training followed by supervised
fine-tuning,” Frontiers in Neuroscience, vol. 12, 2018. [Online]. Available: https:
//www.frontiersin.org/articles/10.3389/fnins.2018.00435
[93] S. Narayanan, K. Taht, R. Balasubramonian, E. Giacomin, and P.-E. Gaillardon,
“Spinalflow: An architecture and dataflow tailored for spiking neural networks,” in 2020
ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA),
2020, pp. 349–362.
[94] L. Liang, Z. Qu, Z. Chen, F. Tu, Y. Wu, L. Deng, G. Li, P. Li, and Y. Xie, “H2learn:
High-efficiency learning accelerator for high-accuracy spiking neural networks,” IEEE
Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41,
no. 11, pp. 4782–4796, 2022.
[95] R. Yin, A. Moitra, A. Bhattacharjee, Y. Kim, and P. Panda, “Sata: Sparsity-aware training
accelerator for spiking neural networks,” IEEE Transactions on Computer-Aided Design
of Integrated Circuits and Systems, vol. 42, no. 6, pp. 1926–1938, 2023.
[96] N. Qiao, H. Mostafa, F. Corradi, M. Osswald, F. Stefanini, D. Sumislawska,
and G. Indiveri, “A reconfigurable on-line learning spiking neuromorphic processor
comprising 256 neurons and 128k synapses,” Frontiers in Neuroscience, vol. 9, 2015.
[Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.2015.00141
[97] J. Stuijt, M. Sifalakis, A. Yousefzadeh, and F. Corradi, “μbrain: An event-driven and
fully synthesizable architecture for spiking neural networks,” Frontiers in Neuroscience,
vol. 15, 2021. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.
2021.664208
[98] S. Kim, S. Kim, S. Um, S. Kim, K. Kim, and H.-J. Yoo, “Neuro-cim: A 310.4 tops/w neuromorphic
computing-in-memory processor with low wl/bl activity and digital-analog
mixed-mode neuron firing,” in 2022 IEEE Symposium on VLSI Technology and Circuits
(VLSI Technology and Circuits), 2022, pp. 38–39.
[99] C. Shi, J. Zhang, T. Wang, Z. Zhong, J. He, H. Gao, J. Yu, P. Li, and M. Tian, “An edge
neuromorphic hardware with fast on-chip error-triggered learning on compressive sensed
spikes,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 70, no. 7, pp.
2665–2669, 2023.
[100] E. Painkras, L. A. Plana, J. Garside, S. Temple, F. Galluppi, C. Patterson, D. R. Lester,
A. D. Brown, and S. B. Furber, “Spinnaker: A 1-w 18-core system-on-chip for massivelyparallel
neural network simulation,” IEEE Journal of Solid-State Circuits, vol. 48, no. 8,
pp. 1943–1953, 2013.
[101] B. V. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A. R. Chandrasekaran, J.-M. Bussat,
R. Alvarez-Icaza, J. V. Arthur, P. A. Merolla, and K. Boahen, “Neurogrid: A mixedanalog-
digital multichip system for large-scale neural simulations,” Proceedings of the
IEEE, vol. 102, no. 5, pp. 699–716, 2014.
[102] J. Schemmel, D. Brüderle, A. Grübl, M. Hock, K. Meier, and S. Millner, “A waferscale
neuromorphic hardware system for large-scale neural modeling,” in 2010 IEEE
International Symposium on Circuits and Systems (ISCAS), 2010, pp. 1947–1950.
[103] E. Baek, H. Lee, Y. Kim, and J. Kim, “Flexlearn: Fast and highly efficient brain
simulations using flexible on-chip learning,” in Proceedings of the 52nd Annual
IEEE/ACM International Symposium on Microarchitecture, ser. MICRO ’52. New
York, NY, USA: Association for Computing Machinery, 2019, p. 304–318. [Online].
Available: https://doi.org/10.1145/3352460.3358268
[104] Y. Kuang, X. Cui, Y. Zhong, K. Liu, C. Zou, Z. Dai, Y. Wang, D. Yu, and R. Huang,
“A 64k-neuron 64m-1b-synapse 2.64pj/sop neuromorphic chip with all memory on chip
for spike-based models in 65nm cmos,” IEEE Transactions on Circuits and Systems II:
Express Briefs, vol. 68, no. 7, pp. 2655–2659, 2021.
[105] Y.-H. Chen, T. Krishna, J. S. Emer, and V. Sze, “Eyeriss: An energy-efficient reconfigurable
accelerator for deep convolutional neural networks,” IEEE Journal of Solid-State
Circuits, vol. 52, no. 1, pp. 127–138, 2017.
[106] S. R. Kheradpisheh, M. Ganjtabesh, and T. Masquelier, “Bio-inspired unsupervised
learning of visual features leads to robust invariant object recognition,” Neurocomputing,
vol. 205, pp. 382–392, 2016. [Online]. Available: https://www.sciencedirect.com/
science/article/pii/S0925231216302880
[107] M. Mozafari, M. Ganjtabesh, A. Nowzari-Dalini, S. J. Thorpe, and T. Masquelier, “Combining
stdp and reward-modulated stdp in deep convolutional spiking neural networks for
digit recognition,” arXiv preprint arXiv:1804.00227, vol. 1, 2018.
[108] M. Mozafari, S. R. Kheradpisheh, T. Masquelier, A. Nowzari-Dalini, and M. Ganjtabesh,
“First-spike-based visual categorization using reward-modulated stdp,” IEEE Transactions
on Neural Networks and Learning Systems, vol. 29, no. 12, pp. 6178–6190, 2018.
[109] Y. Cao, Y. Chen, and D. Khosla, “Spiking deep convolutional neural networks for energyefficient
object recognition,” International Journal of Computer Vision, vol. 113, pp.
54–66, 2015.
[110] E. Hunsberger and C. Eliasmith, “Spiking deep networks with lif neurons,” 2015.
[111] P. U. Diehl, G. Zarrella, A. Cassidy, B. U. Pedroni, and E. Neftci, “Conversion of artificial
recurrent neural networks to spiking neural networks for low-power neuromorphic
hardware,” in 2016 IEEE International Conference on Rebooting Computing (ICRC),
2016, pp. 1–8.
[112] A. Balaji, F. Corradi, A. Das, S. Pande, S. Schaafsma, and F. Catthoor, “Power-accuracy
trade-offs for heartbeat classification on neural networks hardware,” Journal of low
power electronics, vol. 14, no. 4, pp. 508–519, 2018.
[113] S. K. Esser, R. Appuswamy, P. Merolla, J. V. Arthur, and D. S. Modha, “Backpropagation
for energy-efficient neuromorphic computing,” in Advances in Neural Information
Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett,
Eds., vol. 28. Curran Associates, Inc., 2015. [Online]. Available: https://proceedings.
neurips.cc/paper_files/paper/2015/file/10a5ab2db37feedfdeaab192ead4ac0e-Paper.pdf
[114] D. Huh and T. J. Sejnowski, “Gradient descent for spiking neural networks,” in Advances
in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle,
K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates,
Inc., 2018. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2018/
file/185e65bc40581880c4f2c82958de8cfe-Paper.pdf
[115] Y. Jin, W. Zhang, and P. Li, “Hybrid macro/micro level backpropagation
for training deep spiking neural networks,” in Advances in Neural Information
Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman,
N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc.,
2018. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2018/file/
3fb04953d95a94367bb133f862402bce-Paper.pdf
[116] J. C. Thiele, O. Bichler, and A. Dupret, “Spikegrad: An ann-equivalent computation
model for implementing backpropagation with spikes,” arXiv preprint
arXiv:1906.00851, 2019.
[117] S. R. Kheradpisheh and T. Masquelier, “Temporal backpropagation for spiking neural
networks with one spike per neuron,” International Journal of Neural Systems, vol. 30,
no. 06, p. 2050027, 2020.
[118] A. Krizhevsky, “Learning multiple layers of features from tiny images,” Toronto, ON,
Canada, Tech. Rep., 2009.
[119] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward
neural networks,” in Proceedings of the Thirteenth International Conference on
Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research,
Y. W. Teh and M. Titterington, Eds., vol. 9. Chia Laguna Resort, Sardinia, Italy:
PMLR, 13–15 May 2010, pp. 249–256. [Online]. Available: https://proceedings.mlr.
press/v9/glorot10a.html
[120] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, realtime
object detection,” in Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), June 2016.
[121] V. P. Nambiar, J. Pu, Y. K. Lee, A. Mani, T. Luo, L. Yang, E. K. Koh, M. M. Wong, F. Li,
W. L. Goh, and A. T. Do, “0.5v 4.8 pj/sop 0.93uw leakage/core neuromorphic processor
with asynchronous noc and reconfigurable lif neuron,” in 2020 IEEE Asian Solid-State
Circuits Conference (A-SSCC), 2020, pp. 1–4.
[122] R. Yin, Y. Li, A. Moitra, and P. Panda, “Mint: Multiplier-less integer quantization for
spiking neural networks,” 2023.
[123] M. Xiao, Q. Meng, Z. Zhang, Y. Wang, and Z. Lin, “Spide: A purely spike-based
method for training feedback spiking neural networks,” Neural Networks, vol. 161,
pp. 9–24, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/
S0893608023000266
[124] J. Göltz, L. Kriener, A. Baumbach, S. Billaudelle, O. Breitwieser, B. Cramer, D. Dold,
A. F. Kungl, W. Senn, J. Schemmel et al., “Fast and energy-efficient neuromorphic deep
learning with first-spike times,” Nature machine intelligence, vol. 3, no. 9, pp. 823–835,
2021.
[125] S. Kim, S. Kim, S. Hong, S. Kim, D. Han, and H.-J. Yoo, “C-dnn: A 24.5-85.8tops/w
complementary-deep-neural-network processor with heterogeneous cnn/snn core architecture
and forward-gradient-based sparsity generation,” in 2023 IEEE International
Solid- State Circuits Conference (ISSCC), 2023, pp. 334–336.
[126] S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint
arXiv:1605.07146, 2016.
[127] Y. LeCun, C. Cortes, and C. Burges, “Mnist handwritten digit database,” ATT Labs [Online].
Available: http://yann.lecun.com/exdb/mnist, vol. 2, 2010.
[128] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading
digits in natural images with unsupervised feature learning,” in NIPS Workshop on
Deep Learning and Unsupervised Feature Learning 2011, 2011. [Online]. Available:
http://ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf
[129] TensorFlow. (2016) Explanation for why tf.gradients() no longer propagates through
integer tensors. [Online]. Available: https://github.com/tensorflow/tensorflow/issues/
20524
[130] M. Harris. (2016) Mixed-precision programming with cuda 8. [Online]. Available:
https://developer.nvidia.com/blog/mixed-precision-programming-cuda-8/