A Joint Framework of GFP-GAN and Real-ESRGAN for Real-World Image Restoration
Article Sidebar
Main Article Content
Abstract
In the current era of digitalization, the restoration of old photos holds profound significance as it allows us to preserve and revive cherished memories. However, the limitations imposed by various websites offering photo restoration services prompted our research endeavor in the field of image restoration. Our motive originated from the personal desire to restore old photos, which often face constraints and restrictions on existing platforms. As individuals, we often encounter old and faded photographs that require restoration to revive the emotions and moments captured within them. The limits of existing photo restoration services prompted us to conduct this research, with the ultimate goal of contributing to the field of image restoration. To address this issue, we propose a joint framework that combines the Real-ESRGAN and GFP-GAN methods. Our recommended joint structure has been thoroughly tested on a broad range of severely degraded image datasets, and it has shown its efficiency in preserving fine details, recovering colors, and reducing artifacts. The research not only addresses the personal motive for restoring old photos but also has wider applications in preserving memories, cultural artifacts, and historical records through an effective and adaptable solution. Our deep learning-based approach, which leverages the synergistic capabilities of Real-ESRGAN and GFP-GAN, holds immense potential for revitalizing images that have suffered from severe degradation. This proposed framework opens up new avenues for restoring the visual integrity of invaluable historical images, thereby preserving precious memories for generations to come.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
X. Wang, L. Xie, C. Dong, and Y. Shan, “Real-esrgan: Training real-world blind super-resolution with pure synthetic data,” in International Conference on Computer Vision Workshops (ICCVW).
X. Wang, Y. Li, H. Zhang, and Y. Shan, “Towards real-world blind face restoration with generative facial prior,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. https://doi.org/10.1109/CVPR46437.2021.00905
T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4401–4410. https://doi.org/10.1109/CVPR.2019.00453
T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 8110–8119. https://doi.org/10.1109/CVPR42600.2020.00813
J. Gu, Y. Shen, and B. Zhou, “Image processing using multi-code gan prior,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 3012–3021.
X. Pan, X. Zhan, B. Dai, D. Lin, C. C. Loy, and P. Luo, “Exploiting deep generative prior for versatile image restoration and manipulation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 7474–7489, 2021. https://doi.org/10.1109/TPAMI.2021.3115428
S. Menon, A. Damian, S. Hu, N. Ravi, and C. Rudin, “Pulse: Self-supervised photo upsampling via latent space exploration of generative models,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2020, pp. 2437–2445. https://doi.org/10.1109/CVPR42600.2020.00251
Y. Guo, J. Chen, J. Wang, Q. Chen, J. Cao, Z. Deng, Y. Xu, and M. Tan, “Closedloop matters: Dual regression networks for single image super-resolution,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5407–5416. https://doi.org/10.1109/CVPR42600.2020.00545
D. Liu, B. Wen, Y. Fan, C. C. Loy, and T. S. Huang, “Non-local recurrent network for image restoration,” Advances in neural information processing systems, vol. 31, 2018S. https://doi.org/10.1007/978-3-030-01234-2_18
Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, “Image super-resolution using very deep residual channel attention networks,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 286–301. https://doi.org/10.1109/CVPR42600.2020.00243
J. Liu, W. Zhang, Y. Tang, J. Tang, and G. Wu, “Residual feature aggregation network for image super-resolution,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2359–2368. https://doi.org/10.1109/TIP.2017.2662206
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE transactions on image processing, vol. 26, no. 7, pp. 3142–3155, 2017.
M. El Helou, R. Zhou, and S. Susstrunk, “Stochastic frequency masking to im- ¨ prove super-resolution and denoising networks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. Springer, 2020, pp. 749–766. https://doi.org/10.1007/978-3-030-58517-4_44
O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, “Deblurgan: Blind motion deblurring using conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8183–8192. https://doi.org/10.1109/CVPR.2018.00854
J. Guo and H. Chao, “Building dual-domain representations for compression artifacts reduction,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 2016, pp. 628–644. https://doi.org/10.1007/978-3-319-46448-0_38
C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, ´ A. Tejani, J. Totz, Z. Wang et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681–4690. https://doi.org/10.1109/CVPR.2017.19
X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy, “Esrgan: Enhanced super-resolution generative adversarial networks,” in Proceedings of the European conference on computer vision (ECCV) workshops, 2018, pp. 0–0. https://doi.org/10.1007/978-3-030-11021-5_5
J. Chen, J. Chen, H. Chao, and M. Yang, “Image blind denoising with generative adversarial network based noise modeling,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3155–3164. https://doi.org/10.1109/CVPR.2018.00333
Q. Cao, L. Lin, Y. Shi, X. Liang, and G. Li, “Attention-aware face hallucination via deep reinforcement learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 690–698. https://doi.org/10.1109/CVPR.2017.180
X. Yu, B. Fernando, R. Hartley, and F. Porikli, “Super-resolving very lowresolution face images with supplementary attributes,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 908–917.
Y. Chen, Y. Tai, X. Liu, C. Shen, and J. Yang, “Fsrnet: End-to-end learning face super-resolution with facial priors,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2492–2501. https://doi.org/10.1109/CVPR.2018.00264
D. Kim, M. Kim, G. Kwon, and D.-S. Kim, “Progressive face super-resolution via attention to facial landmark,” arXiv preprint arXiv:1908.08239, 2019.
C. Chen, X. Li, L. Yang, X. Lin, L. Zhang, and K.-Y. K. Wong, “Progressive semantic-aware style transformation for blind face restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 896–11 905. https://doi.org/10.1109/CVPR46437.2021.01172
X. Yu, B. Fernando, B. Ghanem, F. Porikli, and R. Hartley, “Face super-resolution guided by facial component heatmaps,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 217–233.
X. Li, M. Liu, Y. Ye, W. Zuo, L. Lin, and R. Yang, “Learning warped guidance for blind face restoration,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 272–289.
X. Li, W. Li, D. Ren, H. Zhang, M. Wang, and W. Zuo, “Enhanced blind face restoration with multi-exemplar images and adaptive spatial feature fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2706–2715.
X. Li, C. Chen, S. Zhou, X. Lin, W. Zuo, and L. Zhang, “Blind face restoration via deep multi-scale component dictionaries,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16. Springer, 2020, pp. 399–415. https://doi.org/10.1007/978-3-030-58545-7_23
R. Abdal, Y. Qin, and P. Wonka, “Image2stylegan: How to embed images into the stylegan latent space?” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4432–4441. https://doi.org/10.1109/ICCV.2019.00453
] J. Zhu, Y. Shen, D. Zhao, and B. Zhou, “In-domain gan inversion for real image editing,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16. Springer, 2020, pp. 592– 608. https://doi.org/10.1007/978-3-030-58520-4_35
K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, “Ghostnet: More features from cheap operations,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 1580–1589. https://doi.org/10.1109/CVPR42600.2020.00165
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
Y. Chen, J. Li, H. Xiao, X. Jin, S. Yan, and J. Feng, “Dual path networks,” Advances in neural information processing systems, vol. 30, 2017.
X. Zhao, Y. Zhang, T. Zhang, and X. Zou, “Channel splitting network for single mr image super-resolution,” IEEE transactions on image processing, vol. 28, no. 11, pp. 5649–5662, 2019. https://doi.org/10.1109/TIP.2019.2921882
] X. Wang, K. Yu, C. Dong, and C. C. Loy, “Recovering realistic texture in image super-resolution by deep spatial feature transform,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 606–615. https://doi.org/10.1109/CVPR.2018.00070
T. Park, M.-Y. Liu, T.-C. Wang, and J.-Y. Zhu, “Semantic image synthesis with spatially-adaptive normalization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 2337–2346. https://doi.org/10.1109/CVPR.2019.00244
J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699. https://doi.org/10.1109/CVPR.2019.00482
H. Li, “Image super-resolution algorithm based on rrdb model,” IEEE Access, vol. 9, pp. 156 260–156 273, 2021.
Z. Liu, P. Luo, X. Wang, and X. Tang, “Large-scale celebfaces attributes (celeba) dataset,” Retrieved August, vol. 15, no. 2018, p. 11, 2018.
S. Wu, S. Zhong, and Y. Liu, “Deep residual learning for image steganalysis,” Multimedia tools and applications, vol. 77, pp. 10 437–10 453, 2018.
K. He, “Georgia gkioxari, et al. mask r-cnn,” arXiv preprint arXiv:1703.06870, 2017.
Akila, Mrs. P. G., Batri, K., Sasi, G., & Ambika, R. (2019). Denoising of MRI Brain Images using Adaptive Clahe Filtering Method. In International Journal of Engineering and Advanced Technology (Vol. 9, Issue 1s, pp. 91–95). https://doi.org/10.35940/ijeat.a1018.1091s19
Sharma, Dr. K., & Garg, N. (2021). An Extensive Review on Image Segmentation Techniques. In Indian Journal of Image Processing and Recognition (Vol. 1, Issue 2, pp. 1–5). https://doi.org/10.54105/ijipr.b1002.061221
Image Synthesis M/2D/HWT in VLSI Technology. (2019). In International Journal of Innovative Technology and Exploring Engineering (Vol. 9, Issue 1, pp. 2976–2982). https://doi.org/10.35940/ijitee.a9119.119119