publications

2024

1. 

   einspace: Searching for Neural Architectures from Fundamental Operations

   Linus Ericsson, Miguel Espinosa, Chenhongyi Yang, and 5 more authors

   2024

   Abs Bib PDF Code

   Neural architecture search (NAS) finds high performing networks for a given task. Yet the results of NAS are fairly prosaic; they did not e.g. create a shift from convolutional structures to transformers. This is not least because the search spaces in NAS often aren’t diverse enough to include such transformations a priori. Instead, for NAS to provide greater potential for fundamental design shifts, we need a novel expressive search space design which is built from more fundamental operations. To this end, we introduce einspace, a search space based on a parameterised probabilistic context-free grammar. Our space is versatile, supporting architectures of various sizes and complexities, while also containing diverse network operations which allow it to model convolutions, attention components and more. It contains many existing competitive architectures, and provides flexibility for discovering new ones. Using this search space, we perform experiments to find novel architectures as well as improvements on existing ones on the diverse Unseen NAS datasets. We show that competitive architectures can be obtained by searching from scratch, and we consistently find large improvements when initialising the search with strong baselines. We believe that this work is an important advancement towards a transformative NAS paradigm where search space expressivity and strategic search initialisation play key roles.

   @article{ericsson2024einspace,
     author = {Ericsson, Linus and Espinosa, Miguel and Yang, Chenhongyi and Antoniou, Antreas and Storkey, Amos and Cohen, Shay B. and McDonagh, Steven and Crowley, Elliot J.},
     title = {einspace: Searching for Neural Architectures from Fundamental Operations},
     year = {2024},
     institution = {University of Edinburgh},
     url = {https://arxiv.org/abs/2405.20838},
   }

2. 

   PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition

   Chenhongyi Yang, Zehui Chen, Miguel Espinosa, and 4 more authors

   BMVC, 2024

   Abs Bib PDF Code

   We present PlainMamba: a simple non-hierarchical state space model (SSM) designed for general visual recognition. The recent Mamba model has shown how SSMs can be highly competitive with other architectures on sequential data and initial attempts have been made to apply it to images. In this paper, we further adapt the selective scanning process of Mamba to the visual domain, enhancing its ability to learn features from two-dimensional images by (i) a continuous 2D scanning process that improves spatial continuity by ensuring adjacency of tokens in the scanning sequence, and (ii) direction-aware updating which enables the model to discern the spatial relations of tokens by encoding directional information. Our architecture is designed to be easy to use and easy to scale, formed by stacking identical PlainMamba blocks, resulting in a model with constant width throughout all layers. The architecture is further simplified by removing the need for special tokens. We evaluate PlainMamba on a variety of visual recognition tasks including image classification, semantic segmentation, object detection, and instance segmentation. Our method achieves performance gains over previous non-hierarchical models and is competitive with hierarchical alternatives. For tasks requiring high-resolution inputs, in particular, PlainMamba requires much less computing while maintaining high performance. Code and models are available at https://github.com/ChenhongyiYang/PlainMamba.

   @article{yang_2024_3_plainmamba,
     author = {Yang, Chenhongyi and Chen, Zehui and Espinosa, Miguel and Ericsson, Linus and Wang, Zhenyu and Liu, Jiaming and Crowley, Elliot J.},
     title = {PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition},
     year = {2024},
     journal = {BMVC},
     institution = {University of Edinburgh},
     url = {https://arxiv.org/abs/2403.17695},
   }

2023

1. 

   Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps

   Miguel Espinosa, and Elliot J. Crowley

   NeurIPS Workshop on Diffusion Models, 2023

   Abs Bib PDF Blog Code Poster

   Despite recent advancements in image generation, diffusion models still remain largely underexplored in Earth Observation. In this paper we show that state-of-theart pretrained diffusion models can be conditioned on cartographic data to generate realistic satellite images. For this purpose, we provide two large datasets of paired maps and satellite views over the region of Mainland Scotland and the Central Belt. We train a ControlNet model and qualitatively evaluate the results, demonstrating that both image quality and map fidelity are possible. Additionally, we explore its use for the reconstruction of historical maps. Finally, we provide some insights on the opportunities and challenges of applying these models for remote sensing.

   @article{espinosa_2023_8_mapsat,
     author = {Espinosa, Miguel and Crowley, Elliot J.},
     title = {Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps},
     year = {2023},
     journal = {NeurIPS Workshop on Diffusion Models},
     institution = {University of Edinburgh},
     url = {https://arxiv.org/abs/2308.16648},
   }