隨著科技進步，自動化是現代人們追求的目標，自駕車的發展越來越多樣化，我們根據這個主題進行研究。我們使用comma.ai在Github開源的自駕車系統openpilot以及他們的硬體設備。根據他們的模型與資料集comma2k19進行研究，建構出與openpilot相似的模型，並尋找合適的訓練方式。在未來，我們能慢慢建構自己的自駕車模型，並在這成熟的自駕車設備上運行。
ar
我們將一個完整的自駕車模型，細分成兩個模型：深度學習神經網路、路徑預測模型。前者為利用神經網路預測結果，後者則是將結果分析、預測成可視化的圖形。在深度學習神經網路中，我們利用YUV儲存方式的特性進行資料前處理，再組合不同的神經網路模型，並且比較backbone EfficientNet與EfficientNetV2在我們的更動與使用下兩者的差異。將訓練結果交予路徑預測模型進行分析，我們分析預測出的種種結果，著重於道路與道路線的預測，將預測出的世界座標轉換成二維座標，使其在圖形可視化。最後，藉由整套自駕車模型結構，我們便能順利預測出道路路徑與道路線。

With the advancement of technology, automation is a goal pursued by modern people, so the development of self-driving cars is becoming more and more diverse. We study this theme, using openpilot, comma.ai's open-source of the self-driving car system on Github, and their hardware devices. Our research is based on their software and hardware systems. We build a deep neural network (DNN), name it as OPNet similar to their DNN supercombo in openpilot, and train it on their dataset comma2k19. OPNet consists of three sub-networks, namely, an encoding NN, a recurrent NN, and a pose NN having regression outputs for path planning and self-driving maneuvers. In the future, we will create our own self-driving model and run on their self-driving devices.
ar

We also develop a path prediction (PP) model from OPNet's output and a simulator for visualizing, analyzing, and verifying OPNet-PP's performance in an end-to-end and real-time manner. Input images to OPNet are encoded in YUV (luminance, blue projection, red projection) instead of RGB (red, green, blue) format. We use and compare two encoding NNs, namely, EfficientNet and its variant V2. Our results show that OPNet-PP can successfully predict driving path and lane lines on the simulator in real time.

[1] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural net-
works,” International conference on machine learning, pp. 6015–6114, 2019.
[2] M. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” International
conference on machine learning., pp. 10096–10106, 2021.
[3] P. Cunningham, M. Cord, and S. J. Delany, “Supervised learning,” Machine learning tech-
niques for multimedia, pp. 21–49, 2018.
[4] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent
neural networks on sequence modeling,” arXiv preprint, p. 1412.3555, 2014.
[5] Y. LeCun and et al., “Backpropagation applied to handwritten zip code recognition,” Neu-
ral Computation, pp. 541 – 551, 1989.
[6] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep con-
volutional neural networks,” Advances in neural information processing systems, p. 25,
2012.
[7] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image
recognition,” arXiv preprint, p. 1409.1556, 2014.
[8] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”
Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–
778, 2016.
[9] A. G. Howard and et al, “Mobilenets: Efficient convolutional neural networks for mobile
vision applications,” arXiv preprint, p. 1704.04861, 2017.
[10] Elman and J. L, “Finding structure in time,” Cognitive science, pp. 179–211, 1990.
[11] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,
pp. 1735–1780, 1997.
[12] Comma.ai, “Openpilot git repo.” https://github.com/commaai/openpilot.
[13] Comma.ai, “Comma2k19.” https://github.com/commaai/comma2k19.
[14] Y.-P. Liu, “Trajectory prediction and radar simulation on the highway with deep learning
methods,” Master’s Thesis, National Tsing Hua University, 202