神經迴路的靈活性使生物體能夠動態地適應環境條件，而不需要對其網路進行結構性改變。這種特性從螃蟹和海蛞蝓的小型神經迴路至哺乳動物的大規模皮質網路等系統中都有體現。雖然先前的研究已經探索了透過調節突觸權重或上游刺激來實現靈活性，但實現循環神經網路中快速功能切換的潛在機制仍然未解，特別是當網路架構和突觸權重保持不變時。

在本文中，我們研究了分叉(bifurcations)在賦予循環神經網路靈活性方面的作用。我們介紹了 CRIREL 系統，這是一個耦合興奮型和抑制型迴路組成的 4 神經元電路。透過對參數空間的廣泛探索，我們發現 CRIREL 網路經歷了雙尖分叉(double-cusp bifurcation)，其中兩個尖分叉獨立發生。利用這種與雙尖分叉的接近性，我們證明了由刺激控制電流可以誘導網路在多個共存功能之間快速切換，而不會改變突觸權重。我們的研究結果強調了遞迴動力學 (recurrent dynamics) 和分叉接近度在實現無與倫比的靈活性方面的重要性，為生物和人工神經系統提供了見解。

Flexibility in neural circuits enables organisms to adapt their behavior to dynamic environmental conditions without requiring structural changes to their networks. This property is exemplified in systems ranging from small neural circuits in crabs and sea slugs to large-scale cortical networks in mammals. While prior studies have explored flexibility through modulation of synaptic weights or contextual inputs, the underlying mechanisms that enable rapid functional switching in recurrent neural networks remain poorly understood, especially when the network archeticture and synamptic weights remained fixed.

In this thesis, we investigate the role of bifurcations in endowing recurrent neural networks with flexibility. Specifically, we introduce the CRIREL system, a 4-neuron circuit composed of recurrently coupled excitatory and inhibitory loops. Through an extensive exploration of parameter space, we reveal that the CRIREL network undergoes a double-cusp bifurcation, where two cusp bifurcations occur independently. Leveraging this proximity to a double-cusp bifurcation, we demonstrate that contextual control currents can induce the network to rapidly switch between multiple coexisting functions without altering synaptic weights. Our results highlight the significance of recurrent dynamics and bifurcation proximity in achieving unparalleled flexibility, offering insights into both biological and artificial neural systems.

[1] B. Liu, A. J. White, and C.-C. Lo, “Augmenting flexibility: Mutual inhibition between inhibitory neurons expands functional diversity,” bioRxiv, 2022.

[2] A. J. White, B. Liu, M.-J. Hsieh, M.-F. Chang, K.-A. Wu, and C. C. Lo, “Project title.” https://github.com/ajw131/CRIRELLogicgates, 2024.

[3] V. A. Webster-Wood, J. P. Gill, P. J. Thomas, and H. J. Chiel, “Control for multifunctionality: Bioinspired control based on feeding in Aplysia californica,” Biological Cybernetics, vol. 114, pp. 557–588, Dec. 2020.

[4] G. R. Yang, M. R. Joglekar, H. F. Song, W. T. Newsome, and X.-J. Wang, “Task representations in neural networks trained to perform many cognitive tasks,” Nature Neuroscience, vol. 22, pp. 297–306, Feb. 2019.

[5] D. W. Morton and H. J. Chiel, “Neural architectures for adaptive behavior,” Trends in Neurosciences, vol. 17, pp. 413–420, Jan. 1994.

[6] C. K. Machens, R. Romo, and C. D. Brody, “Flexible Control of Mutual Inhibition: A Neural Model of Two-Interval Discrimination,” Science, vol. 307, pp. 1121–1124, Feb. 2005.

[7] D. N. Lyttle, J. P. Gill, K. M. Shaw, P. J. Thomas, and H. J. Chiel, “Robustness, flexibility, and sensitivity in a multifunctional motor control model,” Biological Cybernetics, vol. 111, no. 1, pp. 25–47, 2017.

[8] K. V. Shenoy, M. T. Kaufman, M. Sahani, and M. M. Churchland, “A dynamical systems view of motor preparation: Implications for neural prosthetic system design,” Progress in Brain Research, vol. 192, pp. 33–58, 2011.

[9] X.-J. Wang, “Decision Making in Recurrent Neuronal Circuits,” Neuron, vol. 60, pp. 215–234, Oct. 2008.

[10] A. Destexhe and E. Marder, “Plasticity in single neuron and circuit computations,” Nature, vol. 431, pp. 789–795, Oct. 2004.

[11] J. L. Leonard, “Network Architectures and Circuit Function: Testing Alternative Hypotheses in Multifunctional Networks,” Brain, Behavior and Evolution, vol. 55, no. 5, pp. 248–255, 2000.

[12] E. Marder and V. Thirumalai, “Cellular, synaptic and network effects of neuromodulation,” Neural Networks, vol. 15, pp. 479–493, July 2002.

[13] R. M. Harris-Warrick and E. Marder, “Modulation of neural networks for behavior,” Annual Review of Neuroscience, vol. 14, pp. 39–57, 1991.

[14] C. I. Bargmann, “Beyond the connectome: how neuromodulators shape neural circuits,” BioEssays, vol. 34, pp. 458–465, June 2012.

[15] E. Marder, M. L. Goeritz, and A. G. Otopalik, “Robust circuit rhythms in small circuits arise from variable circuit components and mechanisms,” Current Opinion in Neurobiology, vol. 31, pp. 156–163, Apr. 2015.

[16] E. Marder and A. L. Taylor, “Multiple models to capture the variability in biological neurons and networks,” Nature Neuroscience, vol. 14, pp. 133–138, Feb. 2011.

[17] D. N. Lyttle, J. P. Gill, K. M. Shaw, P. J. Thomas, and H. J. Chiel, “Neuromechanical bistability contributes to robust and flexible behavior in a model of motor pattern generation,” BMC Neuroscience, vol. 16, p. P33, Dec. 2015.

[18] J. Yu, H. Hu, A. Agmon, and K. Svoboda, “Recruitment of GABAergic interneurons in the barrel cortex during active tactile behavior,” Neuron, vol. 104, no. 2, pp. 412–427.e4, 2019.

[19] F. Najafi, G. F. Elsayed, R. Cao, E. Pnevmatikakis, P. E. Latham, J. P. Cunningham, and A. K. Churchland, “Excitatory and Inhibitory Subnetworks Are Equally Selective during Decision-Making and Emerge Simultaneously during Learning,” Neuron, vol. 105, pp. 165–179.e8, Jan. 2020.

[20] J. Fritz, S. Shamma, M. Elhilali, and D. Klein, “Rapid task-related plasticity of spectrotemporal receptive fields in primary auditory cortex,” Nature Neuroscience, vol. 6, pp. 1216–1223, Nov. 2003.

[21] K. V. Kuchibhotla, J. V. Gill, G. W. Lindsay, E. S. Papadoyannis, R. E. Field, T. A. H. Sten, K. D. Miller, and R. C. Froemke, “Parallel processing by cortical inhibition enables context-dependent behavior,” Nature Neuroscience, vol. 20, pp. 62–71, Jan. 2017.

[22] C. C. Rodgers and M. R. DeWeese, “Neural correlates of task switching in prefrontal cortex and primary auditory cortex in a novel stimulus selection task for rodents,” Neuron, vol. 82, no. 5, pp. 1157–1170, 2014.

[23] C. M. Niell and M. P. Stryker, “Modulation of visual responses by behavioral state in mouse visual cortex,” Neuron, vol. 65, no. 4, pp. 472–479, 2010.

[24] R. S. Erzurumlu and P. Gaspar, “How the barrel cortex became a working model for developmental plasticity: a historical perspective,” Journal of Neuroscience, vol. 40, no. 34, pp. 6460–6473, 2020.

[25] R. S. Williamson, K. E. Hancock, B. G. Shinn-Cunningham, and D. B. Polley, “Locomotion and task demands differentially modulate thalamic audiovisual processing during active search,” Current Biology, vol. 25, no. 14, pp. 1885–1891, 2015.

[26] S. Jaramillo, K. Borges, and A. M. Zador, “Auditory thalamus and auditory cortex are equally modulated by context during flexible categorization of sounds,” Journal of Neuroscience, vol. 34, no. 15, pp. 5291–5301, 2014.

[27] H. K. Kato, S. N. Gillet, and J. S. Isaacson, “Flexible sensory representations in auditory cortex driven by behavioral relevance,” Neuron, vol. 88, no. 5, pp. 1027–1039, 2015.

[28] C.-T. Wang, C.-T. Lee, X.-J. Wang, and C.-C. Lo, “Top-down modulation on perceptual decision with balanced inhibition through feedforward and feedback inhibitory neurons,” PLoS ONE, vol. 8, no. 4, p. e62379, 2013.

[29] C.-C. Lo and X.-J. Wang, “Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks,” Nature Neuroscience, vol. 9, pp. 956–963, July 2006.

[30] K.-F. Wong and X.-J. Wang, “A Recurrent Network Mechanism of Time Integration in Perceptual Decisions,” Journal of Neuroscience, vol. 26, pp. 1314–1328, Jan. 2006.

[31] G. Hennequin, T. Vogels, and W. Gerstner, “Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements,” Neuron, vol. 82, pp. 1394–1406, June 2014.

[32] F. C. Hoppensteadt and E. M. Izhikevich, “Local Analysis of Weakly Connected Maps,” in Weakly Connected Neural Networks (F. C. Hoppensteadt and E. M. Izhikevich, eds.), Applied Mathematical Sciences, pp. 209–218, New York, NY: Springer New York, 1997.

[33] E. M. Izhikevich, Dynamical Systems in Neuroscience. MIT Press, 2007.

[34] R. D. Beer, “On the Dynamics of Small Continuous-Time Recurrent Neural Networks,” Adaptive Behavior, 1995.

[35] N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami, “The limitations of deep learning in adversarial settings,” in 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387, IEEE, 2016.

[36] E. Galván and P. Mooney, “Neuroevolution in deep neural networks: Current trends and future challenges,” IEEE Transactions on Artificial Intelligence, vol. 2, no. 6, pp. 476–493, 2021.

[37] P. Helfer, C. Teeter, A. Hill, C. M. Vineyard, J. B. Aimone, and D. Kudithipudi, “Context Modulation Enables Multi-tasking and Resource Efficiency in Liquid State Machines,” in Proceedings of the 2023 International Conference on Neuromorphic Systems (ICONS ’23), (New York, NY, USA), pp. 1–9, Association for Computing Machinery, Aug. 2023.

[38] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.

[39] L. Itti and C. Koch, “Computational modelling of visual attention,” Nature Reviews Neuroscience, vol. 2, no. 3, pp. 194–203, 2001.

[40] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proceedings of the National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, 1982.

[41] F. C. Hoppensteadt and E. M. Izhikevich, Weakly Connected Neural Networks. Applied Mathematical Sciences, New York: Springer-Verlag, 1997.

[42] H. R. Wilson and J. D. Cowan, “A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue,” Kybernetik, vol. 13, pp. 55–80, Sept. 1973.

[43] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Transactions on Neural Networks, vol. 14, no. 6, pp. 1569–1572, 2003.

[44] D. Durstewitz, J. K. Seamans, and T. J. Sejnowski, “Neurocomputational models of working memory,” Nature Neuroscience, vol. 3, no. 11, pp. 1184–1191, 2000.

[45] K. A. Ferguson and J. A. Cardin, “Mechanisms underlying gain modulation in the cortex,” Nature Reviews Neuroscience, vol. 21, no. 2, pp. 80–92, 2020.

[46] C. Espinoza, S. J. Guzman, X. Zhang, and P. Jonas, “Parvalbumin+ interneurons obey unique connectivity rules and establish a powerful lateral-inhibition microcircuit in dentate gyrus,” Nature Communications, vol. 9, no. 1, p. 4605, 2018.

[47] N. Haroush and S. Marom, “Inhibition increases response variability and reduces stimulus discrimination in random networks of cortical neurons,” Scientific Reports, vol. 9, no. 1, p. 4969, 2019.

[48] X.-J. Wang, “Neurophysiological and computational principles of cortical rhythms in cognition,” Physiological Reviews, vol. 90, no. 3, pp. 1195–1268, 2010.

[49] M. A. Whittington, R. D. Traub, N. Kopell, B. Ermentrout, and E. H. Buhl, “Inhibition-based rhythms: experimental and mathematical observations on network dynamics,” International Journal of Psychophysiology, vol. 38, no. 3, pp. 315–336, 2000.

[50] C. Van Vreeswijk, L. Abbott, and G. Bard Ermentrout, “When inhibition not excitation synchronizes neural firing,” Journal of Computational Neuroscience, vol. 1, pp. 313–321, 1994.

[51] Y.-C. Huang, C.-T. Wang, T.-S. Su, K.-W. Kao, Y.-J. Lin, C.-C. Chuang, A.-S. Chiang, and C.-C. Lo, “A Single-Cell Level and Connectome-Derived Computational Model of the Drosophila Brain,” Frontiers, 2018.

[52] C.-T. Shih, O. Sporns, S.-L. Yuan, T.-S. Su, Y.-J. Lin, C.-C. Chuang, T.-Y. Wang, C.-C. Lo, R. J. Greenspan, and A.-S. Chiang, “Connectomics-based analysis of information flow in the Drosophila brain,” Current Biology, vol. 25, pp. 1249–1258, May 2015.

[53] C.-T. Shih, Y.-J. Lin, C.-T. Wang, T.-Y. Wang, C.-C. Chen, T.-S. Su, C.-C. Lo, and A.-S. Chiang, “Diverse Community Structures in the Neuronal-Level Connectome of the Drosophila Brain,” Neuroinformatics, vol. 18, pp. 267–281, Apr. 2020.

[54] A.-S. Chiang, C.-Y. Lin, C.-C. Chuang, H.-M. Chang, C.-H. Hsieh, C.-W. Yeh, C.-T. Shih, J.-J. Wu, G.-T. Wang, Y.-C. Chen, C.-C. Wu, G.-Y. Chen, Y.-T. Ching, P.-C. Lee, C.-Y. Lin, H.-H. Lin, C.-C. Wu, H.-W. Hsu, Y.-A. Huang, J.-Y. Chen, H.-J. Chiang, C.-F. Lu, R.-F. Ni, C.-Y. Yeh, and J.-K. Hwang, “Three-Dimensional Reconstruction of Brain-wide Wiring Networks in Drosophila at Single-Cell Resolution,” Current Biology, vol. 21, pp. 1–11, Jan. 2011.

[55] R. Kim and T. J. Sejnowski, “Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks,” Nature Neuroscience, vol. 24, pp. 129–139, Jan. 2021.

[56] L. J. Herstel and C. J. Wierenga, “Network control through coordinated inhibition,” Current Opinion in Neurobiology, vol. 67, pp. 34–41, Apr. 2021.

[57] S. Sadeh and C. Clopath, “Inhibitory stabilization and cortical computation,” Nature Reviews Neuroscience, vol. 22, pp. 21–37, Jan. 2021.

[58] G. Mongillo, S. Rumpel, and Y. Loewenstein, “Inhibitory connectivity defines the realm of excitatory plasticity,” Nature Neuroscience, vol. 21, pp. 1463–1470, Oct. 2018.

[59] C.-C. Lo, C.-T. Wang, and X.-J. Wang, “Speed-accuracy tradeoff by a control signal with balanced excitation and inhibition,” Journal of Neurophysiology, vol. 114, no. 1, pp. 650–661, 2015.

[60] C.-T. Wang, C.-T. Lee, X.-J. Wang, and C.-C. Lo, “Top-down modulation on perceptual decision with balanced inhibition through feedforward and feedback inhibitory neurons,” PLoS One, vol. 8, no. 4, p. e62379, 2013.

[61] 薛又齊, “腦神經系統如何建立身體動作相關的內在模型,” Master’s thesis, 國立清華大學, Jan. 2020.

[62] A. J. White, B. Liu, M.-J. Hsieh, M.-F. Chang, K.-A. Wu, and C.-C. Lo, “Hyper flexible neural networks rapidly switch between logic operations in a compact four neuron circuit,” bioRxiv, pp. 2024–01, 2024.

[63] T. Lau, G. J. Gage, J. D. Berke, and M. Zochowski, “Local dynamics of gap-junction-coupled interneuron networks,” Physical Biology, vol. 7, p. 16015, Mar. 2010.

[64] V. Buendía, S. di Santo, P. Villegas, R. Burioni, and M. A. Muñoz, “Self-organized bistability and its possible relevance for brain dynamics,” Physical Review Research, vol. 2, p. 013318, Mar. 2020.

[65] J. V. Toups, J.-M. Fellous, P. J. Thomas, T. J. Sejnowski, and P. H. Tiesinga, “Multiple Spike Time Patterns Occur at Bifurcation Points of Membrane Potential Dynamics,” PLoS Computational Biology, vol. 8, p. e1002615, Oct. 2012.

[66] S. Kumar, X. Wang, J. P. Strachan, Y. Yang, and W. D. Lu, “Dynamical memristors for higher-complexity neuromorphic computing,” Nature Reviews Materials, vol. 7, no. 7, pp. 575–591, 2022.

[67] T. Wang, J. Meng, X. Zhou, Y. Liu, Z. He, Q. Han, Q. Li, J. Yu, Z. Li, Y. Liu, et al., “Reconfigurable neuromorphic memristor network for ultralow-power smart textile electronics,” Nature Communications, vol. 13, no. 1, p. 7432, 2022.

[68] W. Zhang, B. Gao, J. Tang, P. Yao, S. Yu, M.-F. Chang, H.-J. Yoo, H. Qian, and H. Wu, “Neuro-inspired computing chips,” Nature Electronics, vol. 3, no. 7, pp. 371–382, 2020.

[69] C. Pan, C.-Y. Wang, S.-J. Liang, Y. Wang, T. Cao, P. Wang, C. Wang, S. Wang, B. Cheng, A. Gao, et al., “Reconfigurable logic and neuromorphic circuits based on electrically tunable two-dimensional homojunctions,” Nature Electronics, vol. 3, no. 7, pp. 383–390, 2020.

[70] M. Lukoševičius and H. Jaeger, “Reservoir computing approaches to recurrent neural network training,” Computer Science Review, vol. 3, no. 3, pp. 127–149, 2009.

[71] B. J. Kagan, A. C. Kitchen, N. T. Tran, F. Habibollahi, M. Khajehnejad, B. J. Parker, A. Bhat, B. Rollo, A. Razi, and K. J. Friston, “In vitro neurons learn and exhibit sentience when embodied in a simulated game-world,” Neuron, vol. 110, no. 23, pp. 3952–3969, 2022.

[72] H. Cai, Z. Ao, C. Tian, Z. Wu, H. Liu, J. Tchieu, M. Gu, K. Mackie, and F. Guo, “Brain organoid reservoir computing for artificial intelligence,” Nature Electronics, vol. 6, no. 12, pp. 1032–1039, 2023.

[73] T. Fukuda and T. Kosaka, “Ultrastructural study of gap junctions between dendrites of parvalbumin-containing GABAergic neurons in various neocortical areas of the adult rat,” Neuroscience, vol. 120, pp. 5–20, Aug. 2003.

[74] T. Fukuda, T. Kosaka, W. Singer, and R. A. W. Galuske, “Gap Junctions among Dendrites of Cortical GABAergic Neurons Establish a Dense and Widespread Intercolumnar Network,” Journal of Neuroscience, vol. 26, pp. 3434–3443, Mar. 2006.

[75] G. Gallego et al., “Event-based vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 1, pp. 154–180, 2020.

[76] N. Kogo, F. B. Kern, T. Nowotny, R. v. Ee, R. v. Wezel, and T. Aihara, “Dynamics of a mutual inhibition between pyramidal neurons compared to human perceptual competition,” bioRxiv, p. 2020.05.26.113324, Sept. 2020.

[77] Y.-C. Huang, C.-T. Wang, T.-S. Su, K.-W. Kao, Y.-J. Lin, C.-C. Chuang, A.-S. Chiang, and C.-C. Lo, “A single-cell level and connectome-derived computational model of the drosophila brain,” Frontiers in Neuroinformatics, vol. 12, p. 99, 2019.