DESPINA: Synthesizing Lunar Horizon Imagery
2024–2026, Group · ISPRS Congress 2026 (Toronto)
DESPINA is a research project on generating synthetic, geometry-accurate lunar horizon imagery — realistic ground-level views of the Moon's surface for places where no real photography exists. It began as my undergraduate research in the DAIS lab at the University of Houston (advised by Dr. Christoph Eick, mentored by Raunak Sarbajna) and evolved through two very different generations: an early GAN-based pipeline, and the final diffusion-based system that was published in the ISPRS Annals.
The motivation is coverage. The Apollo missions captured only about 4,000 surface photographs between 1969–1972, and that coverage is sparse, dated, and full of film artifacts. A model that can synthesize believable horizon views for arbitrary locations could, for instance, generate large training datasets for tasks like visual place recognition.
The first approach: a GAN pipeline (DESPINA 1)
The first version framed the problem as image-to-image translation with GANs (Pix2PixHD): given a semantic label map of a scene — ground, hills, sky, rock, crater, and so on — generate a photorealistic lunar view of it.
Getting there took a substantial data pipeline:
- Curating Apollo imagery. Starting from ~50,000 candidate photos, I trained a classifier to sort "lunar surface" from "not lunar."
- Removing fiducial markers. The Apollo film frames are covered in cross-shaped fiducial markers that wreck segmentation models — removing them cleanly became its own sub-project (write-up here).
- Semantic segmentation. I fine-tuned a DINOv2 segmentation head to break each image into ground, hills, sky, rock, crater, astronaut, rover, and sun — hand-labeling roughly 400 images to bootstrap the training set.
- GAN training. With paired label-maps and photos, a Pix2PixHD GAN learned to render each label in the context of its surroundings, over 150 epochs on the university's Sabine HPC cluster. It took nine model iterations to get the hyperparameters and data preparation working well.
Where the GAN approach fell short
The results were generally realistic, but the approach had real limitations:
- Evaluation was subjective. The quality of synthetic imagery is hard to measure quantitatively, so I leaned on human judgment — which is noisy and hard to defend.
- Quality was capped by the label maps. The worst outputs traced directly back to low-quality label maps the GAN had been trained on.
- It wasn't pushing the field forward. The DEM pipeline was still only future work, and the approach hadn't been measured against the state of the art — diffusion models — which is where the real opportunity turned out to be.
That feedback shaped the next version directly.
The final approach: DEM-guided diffusion (DESPINA 2)
The published version keeps the goal but changes the engine. Diffusion models produce far higher-quality images than GANs — but out of the box they can't be constrained to match a specific piece of terrain. ControlNet solves exactly that, letting me impose geometric structure on the generation process.
The final system combines:
- A diffusion backbone (SDXL) with a fine-tuned ControlNet conditioned on three signals derived from elevation data: depth maps, soft edges, and segmentation label maps.
- A custom LoRA trained on 500 lunar images to lock in the correct style and lighting — which, among other things, fixed an early failure mode where the model kept painting an Earth-like white sky.
- A DEM-to-Depth pipeline that ray-casts artificial horizon views from a surface observer's position. Those views become the ControlNet conditioning, so the output is photorealistic and geometrically faithful to the real terrain.
This version compares against and improves over the state of the art, and is the one published in the ISPRS Annals (2026).
Where it's going
- Applying DESPINA to the ocean floor on Earth — bringing the method somewhere with obvious commercial appeal.
- A working Mars proof of concept, likely a future paper.
- Packaging the DEM-to-Depth pipeline as a reusable Python library — it works today, but needs cleanup before it's broadly useful.
This project also grew out of the Apollo image preprocessing covered in the fiducial markers write-up.