物理資訊神經網路（PINNs）是一種現代計算偏微分方程的新興方法，近年來在電腦科學
和涉及物理現象的領域引起了廣泛關注。這種方法的優點在於能夠在無網格的情況下進行計
算，並能針對多樣化的目標進行訓練。然而，在處理高頻率和高剛性方程式時，PINNs常面
臨精確度和收斂性問題，這可能導致計算時間增加甚至結果發散。部分文獻為這些問題提供
了解決方案，常見方式有透過調整神經網路結構和損失函數權重來解決這些問題，但這要求
使用者俱備一定數學方面的知識。
因此，本文提出了一種結合資料集載入和遷移學習的創新方法，以提高對高頻率方程式
預測的準確性，並使其更加使用者友好。本文透過比較預測高頻率常微分方程的準確性來展
示此方法的有效性，並使用多篇文獻中用來演示模型有效性的二維穩態頂蓋驅動空穴流場來
比較本文所提出的各訓練序列模式的準確度提升效果。

Physics-Informed Neural Networks (PINNs) represent a modern approach to solving partial differ-
ential equations (PDEs), gaining traction in computer science and physics-related disciplines. Despite
their advantages in calculating with meshless collocations and task flexibility, PINNs often struggle
with predicting high-frequency and numerically stiff equations, leading to challenges in accuracy and
convergence rates. These issues can escalate computational costs or even result in model divergence.
While common solutions involve modifying neural network structures and adjusting loss weights,
these methods require background knowledge of numerical perspective to implement effectively.
To address these challenges, this thesis proposes an user-friendly yet innovative approach com-
bining dataset loading and transfer learning techniques. This method aims to improve the accuracy
of learning higher-frequency PDEs. We demonstrate its effectiveness through experiments on a chal-
lenging high-frequency ordinary differential equation. Additionally, we compare various training
sequence modes with 2D Lid-driven cavity, which is a common flow field scenario that often used to
validate model efficacy in existing literature.

[1] Hyuk Lee and In Seok Kang. “Neural algorithm for solving differential equations”. In: Journal
of Computational Physics 91.1 (1990), pp. 110–131. ISSN: 0021-9991. DOI: https://doi.
org / 10 . 1016 / 0021 - 9991(90 ) 90007 - N. URL: https : / / www . sciencedirect . com /
science/article/pii/002199919090007N.
[2] A.J. Meade and A.A. Fernandez. “Solution of nonlinear ordinary differential equations by feed-
forward neural networks”. In: Mathematical and Computer Modelling 20.9 (1994), pp. 19–44.
ISSN: 0895-7177. DOI: https : / / doi . org / 10 . 1016 / 0895 - 7177(94 ) 00160 - X. URL:
https://www.sciencedirect.com/science/article/pii/089571779400160X.
[3] M. Perdikaris P. Karniadakis G. E. Raissi. “Physics-informed neural networks: A deep learning
framework for solving forward and inverse problems involving nonlinear partial differential
equations.” In: Journal of Computational physics. 378 (2019), pp. 686–707.
[4] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. “Physics Informed Deep Learn-
ing (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations”. In: CoRR
abs/1711.10561 (2017). arXiv: 1711.10561. URL: http://arxiv.org/abs/1711.10561.
[5] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. “Physics Informed Deep Learn-
ing (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations”. In: CoRR
abs/1711.10566 (2017). arXiv: 1711.10566. URL: http://arxiv.org/abs/1711.10566.

[6] Atilim Gunes Baydin et al. “Automatic Differentiation in Machine Learning: a Survey”. In:
Journal of Machine Learning Research 18.153 (2018), pp. 1–43. URL: http://jmlr.org/
papers/v18/17-468.html.
[7] Stephen J. Wright Suvrit Sra Sebastian Nowozin. Optimization for Machine Learning. The
MIT Press, 2011. ISBN: 9780262016469.
[8] Yoshua Bengio Ian Goodfellow and Aaron Courville. Deep Learning. The MIT Press, 2016.
ISBN: 9780262035613.
[9] Hyogu Jeong et al. “A Physics-Informed Neural Network-based Topology Optimization (PIN-
NTO) framework for structural optimization”. In: Engineering Structures 278 (2023), p. 115484.
ISSN: 0141-0296. DOI: https://doi.org/10.1016/j.engstruct.2022.115484. URL:
https://www.sciencedirect.com/science/article/pii/S0141029622015607.
[10] Hoel Kervadec et al. “Boundary loss for highly unbalanced segmentation”. In: Medical Image
Analysis 67 (2021), p. 101851. ISSN: 1361-8415. DOI: https : / / doi . org / 10 . 1016 / j .
media.2020.101851. URL: https://www.sciencedirect.com/science/article/pii/
S1361841520302152.
[11] Pushan Sharma et al. “A Review of Physics-Informed Machine Learning in Fluid Mechanics”.
In: Energies 16.5 (2023). ISSN: 1996-1073. DOI: 10.3390/en16052343. URL: https://www.
mdpi.com/1996-1073/16/5/2343.
[12] Guofei Pang, Lu Lu, and George Em Karniadakis. “fPINNs: Fractional Physics-Informed Neu-
ral Networks”. In: SIAM Journal on Scientific Computing 41.4 (2019), A2603–A2626. DOI:
10.1137/18M1229845. eprint: https://doi.org/10.1137/18M1229845. URL: https:
//doi.org/10.1137/18M1229845.
[13] Chao-An Lin Yuan-Ting Liao. “Accelerating Fractional step simulation with AutoEncoder”.
In: Master thesis (2021).

[14] Ameya D. Jagtap, Ehsan Kharazmi, and George Em Karniadakis. “Conservative physics-informed
neural networks on discrete domains for conservation laws: Applications to forward and in-
verse problems”. In: Computer Methods in Applied Mechanics and Engineering 365 (2020),
p. 113028. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2020.113028. URL:
https://www.sciencedirect.com/science/article/pii/S0045782520302127.
[15] Ameya D Jagtap and George Em Karniadakis. “Extended physics-informed neural networks
(XPINNs): A generalized space-time domain decomposition based deep learning framework
for nonlinear partial differential equations”. In: Communications in Computational Physics
28.5 (2020).
[16] Justin Sirignano, Jonathan F. MacArt, and Jonathan B. Freund. “DPM: A deep learning PDE
augmentation method with application to large-eddy simulation”. In: Journal of Computational
Physics 423 (2020), p. 109811. ISSN: 0021-9991. DOI: https : / / doi . org / 10 . 1016 / j .
jcp.2020.109811. URL: https://www.sciencedirect.com/science/article/pii/
S0021999120305854.
[17] Dmitrii Kochkov et al. “Machine learning–accelerated computational fluid dynamics”. In: Pro-
ceedings of the National Academy of Sciences 118.21 (2021), e2101784118. DOI: 10.1073/
pnas.2101784118. eprint: https://www.pnas.org/doi/pdf/10.1073/pnas.2101784118.
URL: https://www.pnas.org/doi/abs/10.1073/pnas.2101784118.
[18] R. Maulik et al. “Subgrid modelling for two-dimensional turbulence using neural networks”.
In: Journal of Fluid Mechanics 858 (2019), pp. 122–144. DOI: 10.1017/jfm.2018.770.
[19] Hamidreza Eivazi et al. “Physics-informed neural networks for solving Reynolds-averaged
Navier–Stokes equations”. In: Physics of Fluids 34.7 (2022).
[20] Xuepeng Fu et al. “Data-driven approach for modeling Reynolds stress tensor with invariance
preservation”. In: Computers and Fluids 274 (2024), p. 106215. ISSN: 0045-7930. DOI: https:

//doi.org/10.1016/j.compfluid.2024.106215. URL: https://www.sciencedirect.
com/science/article/pii/S0045793024000471.
[21] Zheng Shi et al. “Finite Difference Neural Networks: Fast Prediction of Partial Differential
Equations”. In: (2020). arXiv: 2006.01892.
[22] N. Sukumar and Ankit Srivastava. “Exact imposition of boundary conditions with distance
functions in physics-informed deep neural networks”. In: Computer Methods in Applied Me-
chanics and Engineering 389 (2022), p. 114333. ISSN: 0045-7825. DOI: https://doi.org/
10 . 1016 / j . cma . 2021 . 114333. URL: https : / / www . sciencedirect . com / science /
article/pii/S0045782521006186.
[23] Jiaji Wang et al. “Exact Dirichlet boundary Physics-informed Neural Network EPINN for
solid mechanics”. In: Computer Methods in Applied Mechanics and Engineering 414 (2023),
p. 116184. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2023.116184. URL:
https://www.sciencedirect.com/science/article/pii/S0045782523003080.
[24] Jian Cheng Wong et al. “LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on
Complex Geometry”. In: (June 2023). DOI: 10.1109/ijcnn54540.2023.10191236. URL:
http://dx.doi.org/10.1109/IJCNN54540.2023.10191236.
[25] Somdatta Goswami et al. “Transfer learning enhanced physics informed neural network for
phase-field modeling of fracture”. In: Theoretical and Applied Fracture Mechanics 106 (2020),
p. 102447. ISSN: 0167-8442. DOI: https://doi.org/10.1016/j.tafmec.2019.102447.
URL: https://www.sciencedirect.com/science/article/pii/S016784421930357X.
[26] Souvik Chakraborty. “Transfer learning based multi-fidelity physics informed deep neural net-
work”. In: Journal of Computational Physics 426 (2021), p. 109942. ISSN: 0021-9991. DOI:
https://doi.org/10.1016/j.jcp.2020.109942. URL: https://www.sciencedirect.
com/science/article/pii/S0021999120307166.

[27] Michael Penwarden et al. “A metalearning approach for Physics-Informed Neural Networks
(PINNs): Application to parameterized PDEs”. In: Journal of Computational Physics 477
(2023), p. 111912. ISSN: 0021-9991. DOI: https : / / doi . org / 10 . 1016 / j . jcp . 2023 .
111912. URL: https://www.sciencedirect.com/science/article/pii/S0021999123000074.
[28] Aditi Krishnapriyan et al. “Characterizing possible failure modes in physics-informed neural
networks”. In: Advances in Neural Information Processing Systems 34 (2021), pp. 26548–
26560.
[29] Sifan Wang et al. “An Expert’s Guide to Training Physics-informed Neural Networks”. In:
(2023). arXiv: 2308.08468.
[30] Sifan Wang, Yujun Teng, and Paris Perdikaris. “Understanding and Mitigating Gradient Flow
Pathologies in Physics-Informed Neural Networks”. In: SIAM Journal on Scientific Computing
43.5 (2021), A3055–A3081. DOI: 10.1137/20M1318043. eprint: https://doi.org/10.
1137/20M1318043. URL: https://doi.org/10.1137/20M1318043.
[31] Nasim Rahaman et al. “On the Spectral Bias of Neural Networks”. In: Proceedings of the 36th
International Conference on Machine Learning (2019).
[32] Sifan Wang, Xinling Yu, and Paris Perdikaris. “When and why PINNs fail to train: A neural
tangent kernel perspective”. In: Journal of Computational Physics 449 (2022), p. 110768. ISSN:
0021-9991. DOI: https://doi.org/10.1016/j.jcp.2021.110768. URL: https://www.
sciencedirect.com/science/article/pii/S002199912100663X.
[33] Alex Kendall, Yarin Gal, and Roberto Cipolla. “Multi-Task Learning Using Uncertainty to
Weigh Losses for Scene Geometry and Semantics”. In: (2018). arXiv: 1705.07115 [cs.CV].
[34] Zixue Xiang et al. “Self-adaptive loss balanced Physics-informed neural networks”. In: Neu-
rocomputing 496 (2022), pp. 11–34. ISSN: 0925-2312. DOI: https://doi.org/10.1016/

j.neucom.2022.05.015. URL: https://www.sciencedirect.com/science/article/
pii/S092523122200546X.
[35] Chao-An Lin Yi-En Chou. “Impact of Loss Weight and Model Complexity on Physics-Informed
Neural Networks for Computational Fluid Dynamics”. In: Master thesis (2023).
[36] Matthew Tancik et al. “Fourier features let networks learn high frequency functions in low
dimensional domains”. In: Advances in neural information processing systems 33 (2020),
pp. 7537–7547.
[37] Sifan Wang, Hanwen Wang, and Paris Perdikaris. “On the eigenvector bias of Fourier fea-
ture networks: From regression to solving multi-scale PDEs with physics-informed neural net-
works”. In: Computer Methods in Applied Mechanics and Engineering 384 (2021), p. 113938.
ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.cma.2021.113938. URL: https:
//www.sciencedirect.com/science/article/pii/S0045782521002759.
[38] Arthur Jacot, Franck Gabriel, and Cl ́ement Hongler. “Neural tangent kernel: Convergence and
generalization in neural networks”. In: Advances in neural information processing systems 31
(2018).
[39] Rambod Mojgani, Maciej Balajewicz, and Pedram Hassanzadeh. “Kolmogorov n–width and
Lagrangian physics-informed neural networks: A causality-conforming manifold for convection-
dominated PDEs”. In: Computer Methods in Applied Mechanics and Engineering 404 (Feb.
2023), p. 115810. ISSN: 0045-7825. DOI: 10 . 1016 / j . cma . 2022 . 115810. URL: http :
//dx.doi.org/10.1016/j.cma.2022.115810.
[40] Ziqi Liu. “Multi-Scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann
Equation in Complex Domains”. In: Communications in Computational Physics 28 (June 2020),
pp. 1970–2001. DOI: 10.4208/cicp.OA-2020-0179.

[41] R. Wienands and W. Joppich. Practical Fourier Analysis for Multigrid Methods. Numerical
Insights. Taylor & Francis, 2005. ISBN: 9781584884927. URL: https : / / books . google .
com.tw/books?id=IOSux5GxacsC.
[42] Nils Margenberg et al. “A neural network multigrid solver for the Navier-Stokes equations”.
In: Journal of Computational Physics 460 (2022), p. 110983. ISSN: 0021-9991. DOI: https:
//doi.org/10.1016/j.jcp.2022.110983. URL: https://www.sciencedirect.com/
science/article/pii/S0021999122000456.
[43] Nils Margenberg et al. “DNN-MG: A hybrid neural network/finite element method with appli-
cations to 3D simulations of the Navier–Stokes equations”. In: Computer Methods in Applied
Mechanics and Engineering 420 (2024), p. 116692. ISSN: 0045-7825. DOI: https://doi.
org/10.1016/j.cma.2023.116692. URL: https://www.sciencedirect.com/science/
article/pii/S0045782523008150.
[44] Mohammad Amin Nabian, Rini Jasmine Gladstone, and Hadi Meidani. “Efficient training of
physics-informed neural networks via importance sampling”. In: Computer-Aided Civil and
Infrastructure Engineering 36.8 (2021), pp. 962–977. DOI: https://doi.org/10.1111/
mice.12685. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/mice.
12685. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/mice.12685.
[45] Chenxi Wu et al. “A comprehensive study of non-adaptive and residual-based adaptive sam-
pling for physics-informed neural networks”. In: Computer Methods in Applied Mechanics and
Engineering 403 (2023), p. 115671. ISSN: 0045-7825. DOI: https://doi.org/10.1016/j.
cma.2022.115671. URL: https://www.sciencedirect.com/science/article/pii/
S0045782522006260.
[46] Elisa Riccietti et al. Multilevel physics informed neural networks (MPINNs). 2022. URL: https:
//openreview.net/forum?id=g5odb-gVVZY.

[47] Sinno Jialin Pan and Qiang Yang. “A Survey on Transfer Learning”. In: IEEE Transactions on
Knowledge and Data Engineering 22.10 (2010), pp. 1345–1359. DOI: 10.1109/TKDE.2009.
191.
[48] Lisa Torrey and Jude Shavlik. “Transfer learning”. In: (2010), pp. 242–264.
[49] Pao-Hsiung Chiu et al. “CAN-PINN: A fast physics-informed neural network based on coupled-
automatic–numerical differentiation method”. In: Computer Methods in Applied Mechanics
and Engineering 395 (May 2022), p. 114909. ISSN: 0045-7825. DOI: 10.1016/j.cma.2022.
114909. URL: http://dx.doi.org/10.1016/j.cma.2022.114909.
[50] Jian Cheng Wong et al. “Learning in Sinusoidal Spaces with Physics-Informed Neural Net-
works”. In: IEEE Transactions on Artificial Intelligence (2022), pp. 1–15. DOI: 10 . 1109 /
TAI.2022.3192362.
[51] Prajit Ramachandran, Barret Zoph, and Quoc V Le. “Searching for activation functions”. In:
arXiv preprint arXiv:1710.05941 (2017).
[52] Kaiming He et al. “Delving Deep into Rectifiers: Surpassing Human-Level Performance on
ImageNet Classification”. In: (2015), pp. 1026–1034. DOI: 10.1109/ICCV.2015.123.
[53] Vincent Sitzmann et al. “Implicit Neural Representations with Periodic Activation Functions”.
In: 33 (2020). Ed. by H. Larochelle et al., pp. 7462–7473. URL: https : / / proceedings .
neurips.cc/paper_files/paper/2020/file/53c04118df112c13a8c34b38343b9c10-
Paper.pdf.
[54] Adam Paszke et al. “Automatic differentiation in PyTorch.” In: In NIPS 2017 Autodi Workshop:
The Future of Gradient-based Machine Learning Software and Techniques (2017).
[55] Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: (2017).
arXiv: 1412.6980 [cs.LG].
[56] U Ghia, K.N Ghia, and C.T Shin. “High-Re solutions for incompressible flow using the Navier-
Stokes equations and a multigrid method”. In: Journal of Computational Physics 48.3 (1982),
pp. 387–411. ISSN: 0021-9991. DOI: https://doi.org/10.1016/0021-9991(82)90058-4.
URL: https://www.sciencedirect.com/science/article/pii/0021999182900584.
[57] Xiaowei Jin et al. “NSFnets (Navier-Stokes flow nets): Physics-informed neural networks
for the incompressible Navier-Stokes equations”. In: Journal of Computational Physics 426
(2021), p. 109951. ISSN: 0021-9991. DOI: https : / / doi . org / 10 . 1016 / j . jcp . 2020 .
109951. URL: https://www.sciencedirect.com/science/article/pii/S0021999120307257.
[58] Shirong Li and Xinlong Feng. “Dynamic weight strategy of physics-informed neural networks
for the 2d navier–stokes equations”. In: Entropy 24.9 (2022), p. 1254.
[59] George Karniadakis, Zhicheng Wang, and Xuhui Meng. “Solution multiplicity and effects of
data and eddy viscosity on Navier-Stokes solutions inferred by physics-informed neural net-
works”. In: Bulletin of the American Physical Society (2023).
[60] Qinghua Jiang et al. “Applications of finite difference-based physics-informed neural networks
to steady incompressible isothermal and thermal flows”. In: International Journal for Numeri-
cal Methods in Fluids 95.10 (2023), pp. 1565–1597.
[61] Xinhai Chen et al. “An improved data-free surrogate model for solving partial differential equa-
tions using deep neural networks”. In: Scientific reports 11.1 (2021), p. 19507.