DUAL-LEVEL PROTOTYPE ALIGNMENT VIA CROSS- ATTENTION FOR FEW-SHOT REMOTE SENSING SEMANTIC SEGMENTATION

(Received: 28-Jun.-2026, Revised: 8-Aug.-2026 , Accepted: 10-Aug.-2026)

Authors Mustafa Alawadi, Mansoor Fateh,

Keywords #Few shot #Remote sensing #Segmentation #Prototype alignment #Cross-attention

Abstract Analyzing remote sensing imagery relies heavily on accurate pixel-level classification to interpret complex geographical scenes. However, training robust deep-learning models demands densely annotated datasets, which are expensive and time-consuming to create. Few-shot semantic segmentation (FSS) provides a practical alternative by adapting to new categories from limited labeled examples. Despite success in natural images, direct application to remote sensing faces unique challenges, such as massive scale variations, strong visual similarities among densely packed objects and severe background interference. Existing FSS methods typically rely on masked average pooling for prototype extraction, which discards critical spatial geometries and fine-grained details. While some approaches attempt multi-scale or global-local modeling, they lack effective, decoupled alignment between support prototypes and query features, limiting their ability to simultaneously capture broad scene context and precise object boundaries in cluttered aerial views. To address these limitations, we propose DLPANet, a novel dual-level prototype alignment network centered on Prototype-Guided Spatial Attention (PGSA). Our framework constructs global and semantic prototypes from hierarchical backbone features, enabling simultaneous modeling of scene context and fine-grained details. Dual cross-attention modules then align these prototypes with decoupled global and semantic query feature maps, facilitating dynamic and precise feature enhancement that better distinguishes adjacent objects and suppresses background ambiguity. A learnable decoder integrates the enriched features, optimized via Dice loss to handle extreme class imbalance. Evaluated on the iSAID-5i benchmark, DLPANet achieves 36.12% and 41.05% mIoU in 1-shot and 5-shot settings, respectively, consistently outperforming state-of-the-art methods. These results demonstrate that our decoupled dual cross-attention mechanism provides superior prototype-query alignment compared to prior global-local strategies. https://github.com/mansoorfateh83-lab/dlpanet.

References

[1] B.-Y. Li et al., "Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing," Proc. of the AAAI Conf. on Artificial Intelligence, volume 40, pp. 5982-5991, 2026.

[2] K. Cheikh, R. Aitahcene, A. Toumi and Z. Hammoudi, "Fusion of Deep Learning Architectures for Enhanced Target Recognition on Sar Images," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 9, no. 4, pp. 347-359, 2023.

[3] H. Bi et al., "ViRefSAM: Visual Reference-guided Segment Anything Model for Remote Sensing Segmentation," IEEE Trans. on Geoscience and Remote Sensing, vol. 64, pp. 5606720- 5606720, 2026.

[4] M. Gholami, M. Fateh and A. Fateh, "Text-enhanced Semantic Segmentation via Contrastive Language-image Pre-training Guided Multi-modal Feature Fusion with Feature Refinement Approach," Int. Journal of Engineering, vol. 39, no. 6, pp.1422-1437, 2026.

[5] A. Saber et al., "A Lightweight Multiscale Refinement Network for Gastrointestinal Disease Classification," Expert Systems with Applications, vol. 308, Article no. 131029, 2026.

[6] K. Amoafo et al., "Enhancing Few-shot Semantic Segmentation in Remote Sensing through Magnitude-based Pruning," Signal Processing: Image Communication, vol. 143, Article no. 117492, 2026.

[7] G. Hcini and I. Jdey, "Privacy Aware Malaria Detection: U-net Model with K-anonymity for Confidential Image Analysis," Jordan. J. of Computers and Information Tech. (JJCIT), vol. 11, no. 1, pp.85-99, 2025.

[8] A. Fateh, M. R. Mohammadi and M. R. Jahed-Motlagh, "Adapting SAM with a triple-prompt strategy for one-shot Semantic Segmentation," Neurocomputing, vol. 681, p. 133301, 2026.

[9] N. Singhal et al., "Study of Recent Image Restoration Techniques: A Comprehensive Survey," Jordanian J. of Computers and Information Technology (JJCIT), vol. 11, no. 2, pp. 211-237, 2025.

[10] Y. Shi et al., "Twostage Fine-tuning of Large Vision-language Models with Hierarchical Prompting for Few-shot Object Detection in Remote Sensing Images," Remote Sensing, vol. 18, no. 2, p. 266, 2026.

[11] S. Zhang, X. Zhang, Y. Xu and K. Wang, "Few-shot Object Detection on Remote Sensing Images Based on Decoupled Training, Contrastive Learning and Self-training," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 19, pp. 3983-3997, 2026.

[12] X. Yao et al., "Scale-aware Detailed Matching for Few-shot Aerial Image Semantic Segmentation," IEEE Trans. on Geoscience and Remote Sensing, vol. 60, Article no. 5611711, pp. 1-11, 2021.

[13] C. Lang et al., "Global Rectification and Decoupled Registration for Few-shot Segmentation in Remote Sensing Imagery," IEEE Trans. on Geosci. and Remote Sens., vol. 61, Art. no. 5617211, pp. 1-11, 2023.

[14] W. Ao, S. Zheng and Y. Meng, "SEMPNet: Enhancing Few-shot Remote Sensing Image Semantic Segmentation through the Integration of the Segment Anything Model," GIScience & Remote Sensing, vol. 61, no. 1, Article no. 2426589, 2024.

[15] J. Long, E. Shelhamer and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proc. of the IEEE Conf. on Computer Vision and Patt. Recog. (CVPR), pp. 3431-3440, Boston, USA, 2015.

[16] O. Ronneberger, P. Fischer and T. Brox, "U-net: Convolutional Networks for Biomedical Image Segmentation," Proc. of the 18th Int. Conf. on Medical Image Computing and Computer-assisted Intervention (MICCAI 2015), pp. 234-241, Munich, Germany, October 5-9, 2015.

[17] V. Badrinarayanan et al., "SegNet: A Deep Convolutional Encoder-decoder Architecture for Image Segmentation," IEEE Trans. on Pattern Analy. and Machine Intelli., vol. 39, no. 12, pp. 2481-2495, 2017.

[18] L.-C. Chen, G. Papandreou, F. Schroff and H. Adam, "Rethinking Atrous Convolution for Semantic Image Segmentation," arXiv preprint, arXiv: 1706.05587, 2017.

[19] L.-C. Chen et al., "Encoder-decoder with Atrous Separable Convolution for Semantic Image Segmentation," Proc. of the Europ. Conf. on Comp. Vision (ECCV2018), vol. 11211, pp. 801-818, 2018.

[20] S. Zheng et al., "Rethinking Semantic Segmentation from a Sequence-to-sequence Perspective with Transformers," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 6881-6890, Nashville, USA, 2021.

[21] M. Fu et al., "LSKSANet: A Novel Architecture for Remote Sensing Image Semantic Segmentation Leveraging Large Selective Kernel and Sparse Attention Mechanism," Proc. of 2024 IEEE Int. Geoscience and Remote Sensing Symposium (the IGARSS 2024), pp. 8438-8441, Athens, Greece, 2024.

[22] H. Feng et al., "FTransDeepLab: Multimodal Fusion Transformer-based Deeplabv3+ for Remote Sensing Semantic Segmentation," IEEE Trans. on Geosci. and Remote Sens., vol. 63, Article no. 4406618, 2025.

[23] K. Wang et al., "PANet: Few-shot Image Semantic Segmentation with Prototype Alignment," Proc. of the 2019 IEEE/CVF Int. Conf. on Computer Vision, pp. 9197-9206, Seoul, S. Korea, 2019.

[24] Z. Tian et al., "Prior Guided Feature Enrichment Network for Few-shot Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 2, pp. 1050-1065, 2020.

[25] G. Li et al., "Adaptive Prototype Learning and Allocation for Few-shot Segmentation," Proc. of the 2021 IEEE/CVF Conf. on Comp. Vision and Pattern Recog. (CVPR), pp. 8334-8343, Nashville, USA, 2021.

[26] J. Min, D. Kang and M. Cho, "Hyper-correlation Squeeze for Few-shot Segmentation," Proc. of the 2021 IEEE/CVF Int. Conf. on Computer Vision, pp. 6941-6952, Montreal, Canada, 2021.

[27] F. Yang, J. Chen, D. Luo and H. Lv, "Few-shot Segmentation of Remote Sensing Imagery with Textual Prompts," Int. Journal of Remote Sensing, vol. 47, no. 3, pp. 1040-1060, 2026.

[28] S. A. Immanuel, W. Cho, J. Heo and D. Kwon, "Tackling Few-shot Segmentation in Remote Sensing via Inpainting Diffusion Model," arXiv preprint, arXiv: 2503.03785, 2025.

[29] J. Wang et al., "Few-shot Semantic Segmentation on Remote Sensing Images with Learnable Prototype," IEEE Trans. on Geoscience and Remote Sensing, vol. 63, Article no. 5104411, 2025.

[30] C. Broni-Bediako et al., "Generalized Few-shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark," IEEE Geoscience and Remote Sensing Letters, vol. 21, pp. 1-5, 2024.

[31] H.-L. Jiang et al., "A Few-shot High-resolution Remote Sensing Image Semantic Segmentation Method," Scientific Reports, vol. 16, Article no. 15262, 2026.

[32] A. Vaswani et al., "Attention Is All You Need," Advances in Neural Inf. Process. Syst., vol. 30, 2017.

[33] C. Lang, G. Cheng, B. Tu and J. Han, "Learning What Not to Segment: A New Perspective on Few-shot Segmentation," Proc. of the 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 8057-8067, New Orleans, USA, 2022.

[34] G. Zhang, G. Kang, Y. Yang and Y. Wei, "Few-shot Segmentation via Cycle-consistent Transformer," Advances in Neural Information Processing Systems, vol. 34, pp. 21984-21996, 2021.

[35] S. W. Zamir et al., "iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images," Proc. of the 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops, pp. 28-37, 2019.

[36] M. Everingham, L. V. Gool, C. K. Williams, J. Winn and A. Zisserman, "The Pascal Visual Object Classes (VOC) Challenge," Int. J. of Computer Vision, vol. 88, pp. 303-338, 2010.

[37] B. Hariharan, P. Arbeláez, L. Bourdev, S. Maji and J. Malik, "Semantic Contours from Inverse Detectors," Proc. of the IEEE 2011 Int. Conf. on Computer Vision, pp. 991-998, Barcelona, Spain, 2011.

[38] Y. Liu et al., "Learning Non-target Knowledge for Few-shot Semantic Segmentation," Proc. of the 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 11573-11582, New Orleans, 2022.

[39] X. Jiang, N. Zhou and X. Li, "Few-shot Segmentation of Remote Sensing Images Using Deep Metric Learning," IEEE Geoscience and Remote Sensing Letters, vol. 19, Article no. 6507405, pp. 1-5, 2022.

[40] G. Puthumanaillam and U. Verma, "Texture Based Prototypical Network for Few-shot Semantic Segmentation of Forest Cover: Generalizing for Different Geographical Regions," Neurocomputing, vol. 538, Article no. 126201, 2023.

[41] G. Liang, F. Xie and Y.-R. Chien, "Class-aware Self-and Cross-attention Network for Few-shot Semantic Segmentation of Remote Sensing Images," Mathematics, vol. 12, no. 17, pp. 2761, 2024.

[42] X. Qi et al., "DSMF-Net: Dual Semantic Metric Learning Fusion Network for Few-shot Aerial Image Semantic Segmentation," IEEE J. of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 18, pp. 853-864, 2024.

[43] S. Wang, M. Meng, M. Chang and Z. Yang, "Learning Spatial-frequency Adaptive prototype for Remote Sensing Few-shot Segmentation," Knowledge-based Systems, vol. 349, Article no. 116464, 2025.

[44] H. Tang, Y. Jia, J. Cheng, Y. Mu, Y. Wu and X. Yao, "Glocaldualnet: Disentangling Scale and Representation for Few-shot Remote Sensing Segmentation," IEEE J. of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 19, pp. 8018-8030, 2026.

[45] Q. Wang, H. Jiang, J. Feng, G. Zhang and J. Yin, "Prototype-guided Structural Learning from Visual Foundation Model for Few-shot Aerial Image Semantic Segmentation," Proc. of the 2024 IEEE Int. Geoscience and Remote Sensing Symposium (IGARSS 2024), pp. 8433-8437, Athens, Greece, 2024.

[46] H. Ni, Q. Liu, X. Niu, D. Hong, L. Zhao and H. Guan, "Cross-domain Few-shot Segmentation via Ordinary Differential Equations over Time Intervals," arXiv preprint, arXiv: 2509.01299, 2025.

[47] A. Fateh et al., "MSDNet: Multi-scale Decoder for Few-shot Semantic Segmentation via Transformer-guided Prototyping," Image and Vision Computing, vol. 162, p. 105672, 2025.

[48] C. Zhang, G. Lin, F. Liu, J. Guo, Q. Wu and R. Yao, "Pyramid Graph Networks with Connection Attentions for Region-based One-shot Semantic Segmentation," Proc. of the 2019 IEEE/CVF Int. Conf. on Computer Vision, pp. 9587-9595, Seoul, S. Korea, 2019.

[49] Z. Lu et al., "Simpler Is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer," Proc. of the 2021 IEEE/CVF Int. Conf. on Computer Vision, pp. 8741-8750, 2021.

[50] Y. Wang, R. Sun and T. Zhang, "Rethinking the Correlation in Few-shot Segmentation: A Buoys View," Proc. of the 2023 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 7183-7192, Vancouver, Canada, 2023.

[51] H. Ding, H. Zhang and X. Jiang, "Self-regularized Prototypical Network for Few-shot Semantic Segmentation," Pattern Recognition, vol. 133, Article no. 109018, 2023.

[52] Y. Tang and Y. Yu, "Query-guided Prototype Learning with Decoder Alignment and Dynamic Fusion in Few-shot Segmentation," ACM Trans. on Multimedia Computing, Communications and Applications, vol. 19, no. 2s, Article no. 84, pp. 1-20, 2023.

[53] Z. Chang et al., "DRNet: Disentanglement and Recombination Network for Few-shot Semantic Segmentation," IEEE Trans. on Circuits and Sys. for Video Tech., vol. 34, no. 7, pp. 5560-5574, 2024.

[54] W. Liu, Z. Wu, H. Ding, F. Liu, J. Lin, G. Lin and W. Zhou, "Exploiting Independent Query Information for Few-shot Image Segmentation," Displays, vol. 91, Article no. 103179, 2026.

Abstract: Analyzing remote sensing imagery relies heavily on accurate pixel-level classification to interpret complex geographical scenes. However, training robust deep-learning models demands densely annotated datasets, which are expensive and time-consuming to create. Few-shot semantic segmentation (FSS) provides a practical alternative by adapting to new categories from limited labeled examples. Despite success in natural images, direct application to remote sensing faces unique challenges, such as massive scale variations, strong visual similarities among densely packed objects and severe background interference. Existing FSS methods typically rely on masked average pooling for prototype extraction, which discards critical spatial geometries and fine-grained details. While some approaches attempt multi-scale or global-local modeling, they lack effective, decoupled alignment between support prototypes and query features, limiting their ability to simultaneously capture broad scene context and precise object boundaries in cluttered aerial views. To address these limitations, we propose DLPANet, a novel dual-level prototype alignment network centered on Prototype-Guided Spatial Attention (PGSA). Our framework constructs global and semantic prototypes from hierarchical backbone features, enabling simultaneous modeling of scene context and fine-grained details. Dual cross-attention modules then align these prototypes with decoupled global and semantic query feature maps, facilitating dynamic and precise feature enhancement that better distinguishes adjacent objects and suppresses background ambiguity. A learnable decoder integrates the enriched features, optimized via Dice loss to handle extreme class imbalance. Evaluated on the iSAID-5i benchmark, DLPANet achieves 36.12% and 41.05% mIoU in 1-shot and 5-shot settings, respectively, consistently outperforming state-of-the-art methods. These results demonstrate that our decoupled dual cross-attention mechanism provides superior prototype-query alignment compared to prior global-local strategies. https://github.com/mansoorfateh83-lab/dlpanet.
URL: https://jjcit.org/paper/302

,abstract={Analyzing remote sensing imagery relies heavily on accurate pixel-level classification to interpret complex geographical scenes. However, training robust deep-learning models demands densely annotated datasets, which are expensive and time-consuming to create. Few-shot semantic segmentation (FSS) provides a practical alternative by adapting to new categories from limited labeled examples. Despite success in natural images, direct application to remote sensing faces unique challenges, such as massive scale variations, strong visual similarities among densely packed objects and severe background interference. Existing FSS methods typically rely on masked average pooling for prototype extraction, which discards critical spatial geometries and fine-grained details. While some approaches attempt multi-scale or global-local modeling, they lack effective, decoupled alignment between support prototypes and query features, limiting their ability to simultaneously capture broad scene context and precise object boundaries in cluttered aerial views. To address these limitations, we propose DLPANet, a novel dual-level prototype alignment network centered on Prototype-Guided Spatial Attention (PGSA). Our framework constructs global and semantic prototypes from hierarchical backbone features, enabling simultaneous modeling of scene context and fine-grained details. Dual cross-attention modules then align these prototypes with decoupled global and semantic query feature maps, facilitating dynamic and precise feature enhancement that better distinguishes adjacent objects and suppresses background ambiguity. A learnable decoder integrates the enriched features, optimized via Dice loss to handle extreme class imbalance. Evaluated on the iSAID-5i benchmark, DLPANet achieves 36.12% and 41.05% mIoU in 1-shot and 5-shot settings, respectively, consistently outperforming state-of-the-art methods. These results demonstrate that our decoupled dual cross-attention mechanism provides superior prototype-query alignment compared to prior global-local strategies. https://github.com/mansoorfateh83-lab/dlpanet.},
keywords={Few shot,Remote sensing,Segmentation,Prototype alignment,Cross-attention},
ISSN={2413-9351},
month={}}

AB - Analyzing remote sensing imagery relies heavily on accurate pixel-level classification to interpret complex geographical scenes. However, training robust deep-learning models demands densely annotated datasets, which are expensive and time-consuming to create. Few-shot semantic segmentation (FSS) provides a practical alternative by adapting to new categories from limited labeled examples. Despite success in natural images, direct application to remote sensing faces unique challenges, such as massive scale variations, strong visual similarities among densely packed objects and severe background interference. Existing FSS methods typically rely on masked average pooling for prototype extraction, which discards critical spatial geometries and fine-grained details. While some approaches attempt multi-scale or global-local modeling, they lack effective, decoupled alignment between support prototypes and query features, limiting their ability to simultaneously capture broad scene context and precise object boundaries in cluttered aerial views. To address these limitations, we propose DLPANet, a novel dual-level prototype alignment network centered on Prototype-Guided Spatial Attention (PGSA). Our framework constructs global and semantic prototypes from hierarchical backbone features, enabling simultaneous modeling of scene context and fine-grained details. Dual cross-attention modules then align these prototypes with decoupled global and semantic query feature maps, facilitating dynamic and precise feature enhancement that better distinguishes adjacent objects and suppresses background ambiguity. A learnable decoder integrates the enriched features, optimized via Dice loss to handle extreme class imbalance. Evaluated on the iSAID-5i benchmark, DLPANet achieves 36.12% and 41.05% mIoU in 1-shot and 5-shot settings, respectively, consistently outperforming state-of-the-art methods. These results demonstrate that our decoupled dual cross-attention mechanism provides superior prototype-query alignment compared to prior global-local strategies. https://github.com/mansoorfateh83-lab/dlpanet.

Download Full Text