擴增實境技術近來發展迅速，已成功應用於外科手術等醫療領域，然而在較偏向民生的整型美容上，仍然較缺少深入探討。本研究針對顏面相關的醫美療程，分別就手術規劃、執行與追蹤階段，發展創新性之擴增實境輔助技術，目的在於提高術前醫病溝通的效率，給予手術進行中的輔助資訊，減少潛在的醫療錯誤。於手術前的規劃階段，將手術細節以虛擬資訊投影的方式呈現，以三維臉部資料輔助受術者作手術選擇。在手術施行方面，則模擬臉部投影整合在頭戴式顯示裝置中的畫面，即時呈現手術施行與限制範圍，引導醫師正確操作；此外即時顯示受術者生理訊號和臉部溫度，減少醫師的心智負荷，避免醫療疏失。而手術操作過程將自動記錄於系統中，作為術後追蹤的依據。

Augmented reality technology has recently developed rapidly and been successfully applied in surgical specialty. However, there is still a lack of in-depth discussion on the aesthetic medicine. To improve the efficiency of preoperative evaluation and reduce potential medical errors for micro cosmetic surgery on face, this study aims at using augmented reality as assistive technology for functional development and further provides auxiliary information during the stage of aesthetic surgery like surgical planning, operation and following up of status post. In the surgical planning stage, the surgical details are presented in the form of monitor based, and the three-dimensional facial data is used to assist the patient in surgical selection. In terms of surgical operation stage, our proposed methods leverage video see-through device to simulate the screen of the head-mounted display, and present surgical limitation range to orient the operation correctly in time. In addition, by displaying the patient’s physiological signal and facial temperature for doctor, the proposed methods demonstrate the ability to lessening medical negligence. Finally, the surgical procedure will be automatically recorded in the system as a basis for following up status post.

[1] 衛生福利部新聞 (2013), 衛生福利部重申「美容醫學」屬「醫療行為」範疇, Available: https://www.mohw.gov.tw/cp-3204-21697-1.html
[2] Aesthetic Medicine Market Size and Share Industry Report (2019), Aesthetic Medicine Market Size, Share & Trends Analysis Report, By Product (Energy-based Devices, Anti-wrinkle Products, By End Use (Clinics & Beauty Centers, Home Use), By Region, And Segment Forecasts, 2019 – 2026, Available: https://www.grandviewresearch.com/industry-analysis/medical-aesthetics-market/toc
[3] Industry ARC (2017), Augmented Reality & Virtual Reality Component Market, Available: https://www.industryarc.com/Report/16344/augmented-reality-virtual-reality-component-market.html
[4] Industry ARC (2016), Wearable Medical Devices Market, Available: https://www.industryarc.com/Report/205/Wearable-medical-devices-Market-Analysis-Report.html
[5] Khor, W. S., Baker, B., Amin, K., Chan, A., Patel, K., & Wong, J. (2016). Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. Annals of translational medicine, 4(23).
[6] Shuhaiber, J. H. (2004). Augmented reality in surgery. Archives of surgery, 139(2), 170-174.
[7] Van Krevelen, D., & Poelman, R. (2007). Augmented reality: Technologies, applications, and limitations. Vrije Univ. Amsterdam, Dep. Comput. Sci.
[8] Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321-1329.
[9] Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995, December). Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and telepresence technologies (Vol. 2351, pp. 282-292). International Society for Optics and Photonics.
[10] Basoglu, N. A., Goken, M., Dabic, M., Ozdemir Gungor, D., & Daim, T. U. (2018). Exploring adoption of augmented reality smart glasses: Applications in the medical industry.
[11] Chang, J. Y. C., Tsui, L. Y., Yeung, K. S. K., Yip, S. W. Y., & Leung, G. K. K. (2016). Surgical vision: Google Glass and surgery. Surgical innovation, 23(4), 422-426.
[12] Wu, T. S., Dameff, C. J., & Tully, J. L. (2014). Ultrasound-guided central venous access using google glass. The Journal of emergency medicine, 47(6), 668-675.
[13] Jeroudi, O. M., Christakopoulos, G., Christopoulos, G., Kotsia, A., Kypreos, M. A., Rangan, B. V., ... & Brilakis, E. S. (2015). Accuracy of remote electrocardiogram interpretation with the use of Google Glass technology. The American journal of cardiology, 115(3), 374-377.
[14] Yoon, J. W., Chen, R. E., Kim, E. J., Akinduro, O. O., Kerezoudis, P., Han, P. K., ... & Pirris, S. M. (2018). Augmented reality for the surgeon: Systematic review. The International Journal of Medical Robotics and Computer Assisted Surgery, 14(4), e1914.
[15] Kim, Y., Kim, H., & Kim, Y. O. (2017). Virtual reality and augmented reality in plastic surgery: a review. Archives of plastic surgery, 44(3), 179.
[16] Carl, B., Bopp, M., Saß, B., Voellger, B., & Nimsky, C. (2019). Implementation of augmented reality support in spine surgery. European Spine Journal, 1-15.
[17] Badiali, G., Ferrari, V., Cutolo, F., Freschi, C., Caramella, D., Bianchi, A., & Marchetti, C. (2014). Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. Journal of cranio-Maxillofacial surgery, 42(8), 1970-1976.
[18] Miyake, R. K., Zeman, H. D., Duarte, F. H., Kikuchi, R., Ramacciotti, E., Lovhoiden, G., & Vrancken, C. (2006). Vein imaging: a new method of near infrared imaging, where a processed image is projected onto the skin for the enhancement of vein treatment. Dermatologic surgery, 32(8), 1031-1038.
[19] Cui, N., Kharel, P., & Gruev, V. (2017). Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery. In Molecular-Guided Surgery: Molecules, Devices, and Applications III (Vol. 10049, p. 100490I). International Society for Optics and Photonics.
[20] Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016). Systematic review on the effectiveness of augmented reality applications in medical training. Surgical endoscopy, 30(10), 4174-4183.
[21] Lin, Y., Wang, X., Wu, F., Chen, X., Wang, C., & Shen, G. (2014). Development and validation of a surgical training simulator with haptic feedback for learning bone-sawing skill. Journal of biomedical informatics, 48, 22-129.
[22] Tepper, O. M., Rudy, H. L., Lefkowitz, A., Weimer, K. A., Marks, S. M., Stern, C. S., & Garfein, E. S. (2017). Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room. Plastic and reconstructive surgery, 140(5), 1066-1070.
[23] Nair, L., & Patel, A. (2018). Mixed Reality in Plastic Surgery: A Primer. Plastic and reconstructive surgery, 142(4), 612e-613e.
[24] Mischkowski, R. A., Zinser, M. J., Kübler, A. C., Krug, B., Seifert, U., & Zöller, J. E. (2006). Application of an augmented reality tool for maxillary positioning in orthognathic surgery–a feasibility study. Journal of Cranio-Maxillofacial Surgery, 34(8), 478-483.
[25] Zuo, K. J., Saun, T. J., & Forrest, C. R. (2019). Facial Recognition Technology: A Primer for Plastic Surgeons. Plastic and reconstructive surgery, 143(6), 1298e-1306e.
[26] Wang, J., Suenaga, H., Yang, L., Kobayashi, E., & Sakuma, I. (2017). Video see‐through augmented reality for oral and maxillofacial surgery. The International Journal of Medical Robotics and Computer Assisted Surgery, 13(2), e1754.
[27] Ulrich, M., Wiedemann, C., & Steger, C. (2011). Combining scale-space and similarity-based aspect graphs for fast 3D object recognition. IEEE transactions on pattern analysis and machine intelligence, 34(10), 1902-1914.
[28] Kalal, Z., Mikolajczyk, K., & Matas, J. (2011). Tracking-learning-detection. IEEE transactions on pattern analysis and machine intelligence, 34(7), 1409-1422.
[29] Murugesan, Y. P., Alsadoon, A., Manoranjan, P., & Prasad, P. W. C. (2018). A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries. The International Journal of Medical Robotics and Computer Assisted Surgery, 14(3), e1889.
[30] Transition Technologies PS. (2018). The application of Augmented Reality in esthetic medicine and plastic surgery. Retrieved from:
https://ttpsc.com/en/casestudy/the-application-of-augmented-reality-in-esthetic-medicine-and-plastic-surgery/
[31] Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International journal of computer vision, 57(2), 137-154.
[32] Freund, Y., Schapire, R., & Abe, N. (1999). A short introduction to boosting. Journal-Japanese Society For Artificial Intelligence, 14(771-780), 1612.
[33] Kazemi, V., & Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1867-1874).
[34] Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2016). 300 faces in-the-wild challenge: Database and results. Image and vision computing, 47, 3-18.
[35] Heikkila, J., & Silven, O. (1997). A four-step camera calibration procedure with implicit image correction. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on (pp. 1106-1112). IEEE. Society Conference on (pp. 1106-1112). IEEE.
[36] Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009). Epnp: An accurate o (n) solution to the pnp problem. International journal of computer vision, 81(2), 155.
[37] 常春新聞 (2019), 詳實記錄手術過程減少醫療糾紛, Available:
https://www.ttv.com.tw/lohas/view/14775