publications

2024

1. 

   PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition

   Chenhongyi Yang, Zehui Chen, Miguel Espinosa, and 4 more authors

   Mar 2024

   Abs Bib PDF Code

   We present PlainMamba: a simple non-hierarchical state space model (SSM) designed for general visual recognition. The recent Mamba model has shown how SSMs can be highly competitive with other architectures on sequential data and initial attempts have been made to apply it to images. In this paper, we further adapt the selective scanning process of Mamba to the visual domain, enhancing its ability to learn features from two-dimensional images by (i) a continuous 2D scanning process that improves spatial continuity by ensuring adjacency of tokens in the scanning sequence, and (ii) direction-aware updating which enables the model to discern the spatial relations of tokens by encoding directional information. Our architecture is designed to be easy to use and easy to scale, formed by stacking identical PlainMamba blocks, resulting in a model with constant width throughout all layers. The architecture is further simplified by removing the need for special tokens. We evaluate PlainMamba on a variety of visual recognition tasks including image classification, semantic segmentation, object detection, and instance segmentation. Our method achieves performance gains over previous non-hierarchical models and is competitive with hierarchical alternatives. For tasks requiring high-resolution inputs, in particular, PlainMamba requires much less computing while maintaining high performance. Code and models are available at https://github.com/ChenhongyiYang/PlainMamba.

   @article{yang_2024_3_plainmamba,
     author = {Yang, Chenhongyi and Chen, Zehui and Espinosa, Miguel and Ericsson, Linus and Wang, Zhenyu and Liu, Jiaming and Crowley, Elliot J.},
     title = {PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition},
     year = {2024},
     month = mar,
     booktitle = {arXiv preprint arXiv:2403.17695},
     institution = {University of Edinburgh},
     url = {https://arxiv.org/abs/2403.17695},
   }

2023

1. 

   Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps

   Miguel Espinosa, and Elliot J. Crowley

   Aug 2023

   Abs Bib PDF Blog Code Poster

   Despite recent advancements in image generation, diffusion models still remain largely underexplored in Earth Observation. In this paper we show that state-of-theart pretrained diffusion models can be conditioned on cartographic data to generate realistic satellite images. For this purpose, we provide two large datasets of paired maps and satellite views over the region of Mainland Scotland and the Central Belt. We train a ControlNet model and qualitatively evaluate the results, demonstrating that both image quality and map fidelity are possible. Additionally, we explore its use for the reconstruction of historical maps. Finally, we provide some insights on the opportunities and challenges of applying these models for remote sensing.

   @article{espinosa_2023_8_mapsat,
     author = {Espinosa, Miguel and Crowley, Elliot J.},
     title = {Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps},
     year = {2023},
     month = aug,
     booktitle = {NeurIPS 2023 Workshop on Diffusion Models},
     institution = {University of Edinburgh},
     url = {https://arxiv.org/abs/2308.16648},
   }