NEWS

STUDY OF RECENT IMAGE RESTORATION TECHNIQUES: A COMPREHENSIVE SURVEY


(Received: 24-Dec.-2024, Revised: 22-Feb.-2025 , Accepted: 26-Mar.-2025)
The rapid advancements in digital imaging technologies, including image restoration (IR), have created a growing demand for effective image-restoration techniques. Various kinds of degradation, including noise, blur and low resolution, should be handled with these techniques. Restoration is important in many applications, including medical imaging, surveillance, photography and remote sensing, where image quality will be critical to the correctness of analysis and decision. This article provides an all-inclusive review of state-of-the-art (SOTA) methods in image restoration, covering traditional methods as well as modern techniques like deep learning (DL) and transformer-based models. Traditional image-restoration techniques include deblurring, denoising and super-resolution based on mathematical models and handcrafted algorithms. These methods were indeed effective for certain types of noise or blur, but generalized poorly to various real-world scenarios. Recent advances in machine learning (ML), especially DL using convolutional neural networks (CNNs), have made data-driven approaches that learn directly from large datasets much more effective. Recently, transformer-based models, such as Vision Transformers and Swin Transformers, have shown the ability to capture global dependencies in images, leading to superior performance on complex restoration tasks. It is also to mention the challenge of generalization across the type of degradation, say mixed noise or blur, and across different datasets. The proposed survey indicates the limitations of existing approaches, including computational cost and generalization challenges and offers insights into possible directions for future research. Considering these challenges and achievements, this article attempts to provide helpful guidance on methods for future research on restoring images.

[1] Z. Liang et al., "GIFM: An Image Restoration Method with Generalized Image Formation Model for Poor Visible Conditions," IEEE Trans. on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022.

[2] S. Jiang et al., "Local Adaptive Prior-based Image Restoration Method for Space Diffraction Imaging Systems," IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–10, 2023.

[3] Z. Yang, J. Huang, M. Zhou, N. Zheng and F. Zhao, "IRVR: A General Image Restoration Framework for Visual Recognition," IEEE Transactions on Multimedia, vol. 26, pp. 7012-7026, 2024.

[4] R. Chen, T. Guo, Y. Mu and L. Shen, "Learning Compact Hyperbolic Representations of Latent Space for Old Photo Restoration," IEEE Transactions on Image Processing, vol. 33, pp. 3578–3589, 2024.

[5] T. Kim, C. Shin, S. Lee and S. Lee, "Block-attentive Subpixel Prediction Networks for Computationally Efficient Image Restoration," IEEE Access, vol. 9, pp. 90881–90895, 2021.

[6] S. Kong, W. Wang, X. Feng and X. Jia, "Deep RED Unfolding Network for Image Restoration," IEEE Transactions on Image Processing, vol. 31, pp. 852–867, 2021.

[7] D. Perdios et al., "CNN-based Image Reconstruction Method for Ultrafast Ultrasound Imaging," IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, vol. 69, no. 4, pp. 1154–1168, 2021.

[8] Z. Pan et al., "VCRNet: Visual Compensation Restoration Network for No-reference Image Quality Assessment," IEEE Transactions on Image Processing, vol. 31, pp. 1613–1627, 2022.

[9] Q. Zhang et al., "Combined Deep Priors with Low-rank Tensor Factorization for Hyperspectral Image Restoration," IEEE Geoscience and Remote Sensing Letters, vol. 20, pp. 1–5, 2023.

[10] J. Ke et al., "Artifact Detection and Restoration in Histology Images with Stain-style and Structural Preservation," IEEE Transactions on Medical Imaging, vol. 42, no. 12, pp. 3487–3500, 2023.

[11] Y. Pang, J. Mao, L. He, H. Lin and Z. Qiang, "An Improved Face Image Restoration Method Based on Denoising Diffusion Probabilistic Models," IEEE Access, vol. 12, pp. 3581-3596, 2024.

[12] X. Zhang and J. Feng, "A Novel Blind Restoration Method for Miner Face Images Based on Improved GFP-GAN Model," IEEE Access, vol. 12, pp. 104676–104687, 2024.

[13] M. Yao, R. Xu, Y. Guan, J. Huang and Z. Xiong, "Neural Degradation Representation Learning for All-in-one Image Restoration," IEEE Transactions on Image Processing, vol. 33, pp. 5408–5423, 2024.

[14] W. Zhang, W. Zhao, J. Li, P. Zhuang, H. Sun, Y. Xu and C. Li, "Cvanet: Cascaded Visual Attention Network for Single Image Super-resolution," Neural Networks, vol. 170, pp. 622–634, 2024.

[15] J. Liang et al., "SwinIR: Image Restoration Using Swin Transformer," Proc. of the IEEE/CVF Int. Conf. on Computer Vision, pp. 1833–1844, Montreal, Canada, 2021.

[16] Z. Deng et al., "RFormer: Transformer-based Generative Adversarial Network for Real Fundus Image Restoration on a New Clinical Benchmark," IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 9, pp. 4645–4655, 2022.

[17] Z. Wang et al., "Uformer: A General U-shaped Transformer for Image Restoration," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recog., pp. 17683–17693, New Orleans, USA, 2022.

[18] J. Tan et al., "Blind Face Restoration for Under-display Camera via Dictionary Guided Transformer," IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 6, pp. 4914–4927, 2023.

[19] Y. Zhang, Q. Yang, D. M. Chandler and X. Mou, "Reference-based Multi-stage Progressive Restoration for Multi-degraded Images," IEEE Transactions on Image Processing, vol. 33, pp. 4982–4997, 2024.

[20] B. Zhou et al., "DuDoUFNet: Dual-domain Under-to-fully-complete Progressive Restoration Network for Simultaneous Metal Artifact Reduction and Low-dosect Reconstruction," IEEE Transactions on Medical Imaging, vol. 41, no. 12, pp. 3587–3599, 2022.

[21] I. Marivani et al., "Designing CNNS for Multimodal Image Restoration and Fusion via Unfolding the Method of Multipliers," IEEE (TCSVT) Journal, vol. 32, no. 9, pp. 5830–5845, 2022.

[22] X. Feng et al., "Deep-masking Generative Network: A Unified Framework for Background Restoration from Superimposed Images," IEEE  Transactions on Image Processing, vol. 30, pp. 4867–4882, 2021.

[23] H. Yan et al., "UW-CycleGAN: Model-driven CycleGAN for Underwater Image Restoration," IEEE Transactions on Geoscience and Remote Sensing, vol. 61, DOI: 10.1109/TGRS.2023.3315772, 2023.

[24] Y. Zhu et al., "Denoising Diffusion Models for Plug-and-play Image Restoration," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recogn., pp. 1219–1229, Vancouver, Canada, 2023.

[25] S. Welker, H. N. Chapman and T. Gerkmann, "DriftRec: Adapting Diffusion Models to Blind JPEG Restoration," IEEE Transactions on Image Processing, vol. 33, pp. 2795-2807, 2024.

[26] H. Cho, H.-K. Shin et al., "PD-CR: Patch-based Diffusion Using Constrained Refinement for Image Restoration," IEEE Signal Processing Letters, vol. 31, pp. 949–953, 2024.

[27] Z. Luo et al., "Refusion: Enabling Large-size Realistic Image Restoration with Latent-space Diffusion Models," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 1680–1691, Vancouver, Canada, 2023.

[28] J. V. Ortega, M. Haas and A. Effland, "Learning Diffusion Functions for Image Restoration," Proc. of the 2024 IEEE Int. Symposium on Biomedical Imaging (ISBI), Athens, Greece, 2024.

[29] Z. Yue, J. Wang and C. C. Loy, "Efficient Diffusion Model for Image Restoration by Residual Shifting," arXiv preprint, arXiv: 2403.07319, 2024.

[30] S. Yan et al., "HybrUR: A Hybrid Physical-neural Solution for Unsupervised Underwater Image Restoration," IEEE Trans. on Image Processing, vol. 32, pp. 5004–5016, 2023.

[31] Y. Chang et al., "Hyperspectral Image Restoration: Where Does the Low-rank Property Exist?" IEEE Trans. on Geoscience and Remote Sensing, vol. 59, no. 8, pp. 6869–6884, 2020.

[32] W. He et al., "Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 4, pp. 2089–2107, 2020.

[33] D. Berman, D. Levy, S. Avidan and T. Treibitz, "Underwater Single Image Color Restoration Using Haze-lines and a New Quantitative Dataset," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 8, pp. 2822–2837, 2020.

[34] M. Li et al., "Imaging Simulation and Learning-based Image Restoration for Remote Sensing Time Delay and Integration Cameras," IEEE Transactions on Geoscience and Remote Sensing, vol. 61, 2023.

[35] C. Li et al., "Efficient Dehazing Method for Outdoor and Remote Sensing Images," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 16, pp. 4516–4528, 2023.

[36] S. Zhong et al., "RPIR: A Semi-blind Unsupervised Learning Image Restoration Method for Optical Synthetic Aperture Imaging Systems with Co-phase Errors," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 17, pp. 15344-15358 2024.

[37] W. Zhang et al., "Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement," IEEE Transactions on Image Processing, vol. 31, pp. 3997–4010, 2022.

[38] W. Zhang et al., "Underwater Image Enhancement via Weighted Wavelet Visual Perception Fusion," IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 4, pp. 2469–2483, 2023.

[39] L. Denis et al., "A Review of Deep-learning Techniques for SAR Image Restoration," Proc. of the 2021 IEEE Int. Geoscience and Remote Sensing Symposium GARSS, pp. 411–414, 2021.

[40] R. Kumar et al., "A Review on Generative Adversarial Networks Used for Image Reconstruction in Medical Imaging," Proc. of the 2021 9th Int. Conf. on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), pp. 1–5, Noida, India, 2021.

[41] W.-C. Sui, X. Cheng and H. A. Chan, "Critical Review on Deep Learning and Smart Technologies for Image Super-resolution," Proc. of the IEEE TENCON 2022-2022 IEEE Region 10 Conf. (TENCON), pp. 1–8, Hong Kong, Hong Kong, 2022.

[42] S. Hou, Y. Wang, K. Li, Y. Zhao, B. Lu and L. Fan, "Deep Learning for Screen-shot Image Demoiréing: A Survey," IEEE Access, vol. 10, pp. 108453–108468, 2022.

[43] M. Pandey, G. Rawat and P. Kanti, "Image Restoration Application and Methods for Different Images: A Review," Proc. of the 2022 IEEE Int. Conf. on Advances in Computing, Communication and Materials (ICACCM), pp. 1–4, Dehradun, India, 2022.

[44] X. Li, Y. Ren, X. Jin, C. Lan, X. Wang, W. Zeng, X. Wang and Z. Chen, "Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey," arXiv preprint, arXiv:2308.09388, 2023.

[45] G. P. Kumar et al., "A Comprehensive Review on Image Restoration Methods Due to Salt and Pepper Noise," Proc. of the 2023 2nd IEEE Int. Conf. on Automation, Computing and Renewable Systems (ICACRS), pp. 562– 567, Pudukkottai, India, 2023.

[46] Q. Feng et al., "GAN-based Image Deblurring: A Comparison," Proc. of the 2023 IEEE 2nd Int. Conf. on Electrical Engineering, Big Data and Algorithms (EEBDA), pp. 318–324, 2023.

[47] L. Zhai, Y. Wang, S. Cui and Y. Zhou, "A Comprehensive Review of Deep Learning-based Real-world Image Restoration," IEEE Access, vol. 11, pp. 21049–21067, 2023.

[48] N. Deluxni et al., "A Scrutiny on Image Enhancement and Restoration Techniques for Underwater Optical Imaging Applications," IEEE Access, DOI:10.1109/ACCESS.2023.3322153, 2023.

[49] S. Yu et al., "Review of Quality Assessment Algorithms on the Realistic Blurred Image Database (BID2011)," Proc. of the 2023 8th IEEE Int. Conf. on Signal and Image Process., pp. 450–454, 2023.

[50] K. Rajput et al., "An Enhanced Analysis of Machine Learning Techniques for Image Restoration and Enhancement," Proc. of the 2024 15th IEEE Int. Conf. on Computing Communication and Networking Technologies (ICCCNT), pp. 1–6, 2024.

[51] T. Manjunath et al., "Development of an Image Restoration Algorithm Utilizing Generative Adversarial Networks (GAN’s) for Enhanced Performance in Engineering Applications: A Comprehensive Approach to Improving Image Quality and Clarity through Advanced Machine Learning Techniques," Proc. of the 2024 IEEE Int. Conf. on Innovation and Novelty in Eng. and Tech., vol. 1, pp. 1–6, 2024.

[52] R. S. Jebur et al., "A Comprehensive Review of Image Denoising in Deep Learning," Multimedia Tools and Applications, vol. 83, no. 20, pp. 58181–58199, 2024.

[53] Z. Zhang et al., "NTIRE 2024 Challenge on Bracketing Image Restoration and Enhancement: Datasets, Methods and Results," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 6153–6166, Seattle, USA, 2024.

[54] T. Karras et al., "A Style-based Generator Architecture for Generative Adversarial Networks," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 4401–4410, 2019.

[55] Z. Liu, P. Luo, X. Wang and X. Tang, "Deep Learning Face Attributes in the Wild," Proc. of the IEEE International Conference on Computer Vision, pp. 3730–3738, 2015.

[56] R. Timofte et al., "NTIRE 2017 Challenge on Single Image Super-resolution: Methods and Results," Proc. of the Conf. on Comp. Vision and Pattern Recog. Workshops, pp.114–125, Honolulu, USA, 2017.

[57] E. Agustsson and R. Timofte, "NTIRE 2017 Challenge on Single Image Super-resolution: Dataset and Study," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, USA, July 2017.

[58] A. Foi et al., "Pointwise Shape-adaptive DCT for High-quality Denoising and Deblocking of Grayscale and Color Images," IEEE Transactions on Image Processing, vol. 16, no. 5, pp. 1395–1411, 2007.

[59] H. Sheikh, "Live Image Quality Assessment Database Release 2," [Online], Available: http://live.ece.utexas.edu/research/quality, 2005.

[60] A. Abdelhamed et al., "A High-quality Denoising Dataset for Smartphone Cameras," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1692– 1700, Salt Lake City, USA, 2018.

[61] X. Zhang et al., "Single Image Reflection Separation with Perceptual Losses," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 4786–4794, Salt Lake City, USA, 2018.

[62] N. Silberman et al., "Indoor Segmentation and Support Inference from RGBD Images," Proc. of the Computer Vision–ECCV 2012: 12th European Conf. on Computer Vision, Part V12, pp. 746–760, Florence, Italy, 2012.

[63] D. Martin, C. Fowlkes, D. Tal and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. of the 8th Int. Conf. Computer Vision, vol. 2, pp. 416–423, Vancouver, Canada, July 2001.

[64] R. Qian, R. T. Tan, W. Yang, J. Su and J. Liu, "Attentive Generative Adversarial Network for Raindrop Removal from a Single Image," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2482–2491, Salt Lake City, USA, 2018.

[65] C. Li, J. Guo and C. Guo, "Emerging from Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer," IEEE Signal Processing Letters, vol. 25, no. 3, pp. 323–327, 2018.

[66] S. Nah, T. H. Kim and K. M. Lee, "Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring," Proc. of the 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 257-265, July 2017.

[67] B. A. Research, "BSD68: Part of Berkeley Segmentation Dataset and Benchmark," [Online], Available:  https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/, 2018.

[68] A. Blau, K. Michaeli, et al., "The PIRM Challenge on Perceptual Image Enhancement on Smartphones," Proc. of the European Conf. Computer Vision Workshops, pp. 391–411, 2018.

[69] J. Zhang et al., "HAC Dataset: Benchmarking Adverse Condition Image Restoration," [Online], Available: https://github.com/jzbjyb/HAC-dataset, 2019.

[70] T. Karras et al., "FFHQ: Flickr-faces-HQ Dataset," [Online], Available: https://github.com/NVlabs/ffhq-dataset, 2019.

[71] H. Yue et al., "SCISR: Synthetic and Camera-based Image Super-resolution Dataset," [Online],
Available: https://github.com/SCISR/dataset, 2019.

[72] S. Shen et al., "Hide: Human-aware Image Deblurring Dataset," [Online], Available: https://github.com/joeylitalien/hide-dataset, 2019.

[73] W. Qian et al., "Raindrop Dataset: Image Pairs with Raindrop Artifacts," [Online], Available: https://github.com/riddhishb/raindrop-removal, 2020.

[74] J. Wei et al., "UHDS: Ultra High-definition Synthetic Dataset for Rainy Image Restoration," [Online], Available: https://github.com/uhds/uhds-dataset, 2022.

[75] C. Program, "Sentinel-2 Satellite Images Dataset," [Online], Available: https://scihub.copernicus.eu/, 2022.

[76] Y. Yu et al., "TinyPerson Dataset for Tiny Object Detection," [Online], Available: http://github.com/TinyPerson/dataset, 2022.

[77] Z. Liu et al., "LSDIR: Large-scale Dataset for Image Restoration," [Online], Available: https://github.com/LSDIR/dataset, 2023.

[78] N. Corporation, "HQ-50K: High-quality Dataset for Image Restoration," [Online], Available: https://github.com/NVIDIA/ HQ-50K, 2023.