SparseSat-NeRF: Dense Depth Supervised Neural Radiance Fields for Sparse Satellite Images

Digital surface model generation using traditional multi-view stereo matching (MVS) performs poorly over non-Lambertian surfaces, with asynchronous acquisitions, or at discontinuities. Neural radiance fields (NeRF) offer a new paradigm for reconstructing surface geometries using continuous volumetric representation. NeRF is self-supervised, does not require ground truth geometry for training, and provides an elegant way to include in its representation physical parameters about the scene, thus potentially remedying the challenging scenarios where MVS fails. However, NeRF and its variants require many views to produce convincing scene's geometries which in earth observation satellite imaging is rare. In this paper we present SparseSat-NeRF (SpS-NeRF) - an extension of Sat-NeRF adapted to sparse satellite views. SpS-NeRF employs dense depth supervision guided by crosscorrelation similarity metric provided by traditional semi-global MVS matching. We demonstrate the effectiveness of our approach on stereo and tri-stereo Pleiades 1B/WorldView-3 images, and compare against NeRF and Sat-NeRF. The code is available at https://github.com/LulinZhang/SpS-NeRFComment: ISPRS Annals 202

Pointless Global Bundle Adjustment With Relative Motions Hessians

Bundle adjustment (BA) is the standard way to optimise camera poses and to produce sparse representations of a scene. However, as the number of camera poses and features grows, refinement through bundle adjustment becomes inefficient. Inspired by global motion averaging methods, we propose a new bundle adjustment objective which does not rely on image features' reprojection errors yet maintains precision on par with classical BA. Our method averages over relative motions while implicitly incorporating the contribution of the structure in the adjustment. To that end, we weight the objective function by local hessian matrices - a by-product of local bundle adjustments performed on relative motions (e.g., pairs or triplets) during the pose initialisation step. Such hessians are extremely rich as they encapsulate both the features' random errors and the geometric configuration between the cameras. These pieces of information propagated to the global frame help to guide the final optimisation in a more rigorous way. We argue that this approach is an upgraded version of the motion averaging approach and demonstrate its effectiveness on both photogrammetric datasets and computer vision benchmarks

A pipeline for automated processing of declassified Corona KH-4 (1962-1972) stereo imagery

This study was supported by the Strategic Priority Research Program of Chinese Academy of Sciences (XDA20100300) and the Swiss National Science Foundation (200021E 177652/1) within the framework of the DFG Research Unit GlobalCDA (FOR2630).The Corona KH-4 reconnaissance satellite missions acquired panoramic stereo imagery with high spatial resolution of 1.8–7.5m from 1962-1972. The potential of 800,000+ declassified Corona images has not been leveraged due to the complexities arising from handling of panoramic imaging geometry, film distortions and limited availability of the metadata required for georeferencing of the Corona imagery. This paper presents the Corona Stereo Pipeline (CoSP): A pipeline for processing of Corona KH-4 stereo panoramic imagery. CoSP utilizes deep learning based feature matcher SuperGlue to automatically match features point between Corona KH-4 images and recent satellite imagery to generate Ground Control Points (GCPs). To model the imaging geometry and the scanning motion of the panoramic KH-4 cameras, a rigorous camera model consisting of modified collinearity equations with time-dependent exterior orientation parameters is employed. Using the entire frame of the Corona image, bundle adjustment with well-distributed GCPs results in an average standard deviation or σ0 of less than two pixels. We evaluate fiducial marks on the Corona films and show that pre-processing the Corona images to compensate for film bending improves the 3D reconstruction accuracy. The distortion pattern of image residuals of GCPs and y-parallax in epipolar resampled images suggest that film distortions due to long-term storage likely cause systematic deviations of up to six pixels. Compared to the SRTM DEM, the Corona DEM computed using CoSP achieved a Normalized Median Absolute Deviation of elevation differences of ≈ 4m over an area of approx. 4000km2 after a tile-based fine coregistration of the DEMs. We further assess CoSP on complex scenes involving high relief and glacierized terrain and show that the resulting DEMs can be used to compute long-term glacier elevation changes over large areas.PostprintPeer reviewe

Preface: the 2020 edition of the XXIVth ISPRS congress

International audienc

Photogrammetric shape reconstruction of diffuse and specular objects in time

Zsfassung in dt. Sprache12

Sinusoidal Wave Estimation Using Photogrammetry and Short Video Sequences

The objective of the work is to model the shape of the sinusoidal shape of regular water waves generated in a laboratory flume. The waves are traveling in time and render a smooth surface, with no white caps or foam. Two methods are proposed, treating the water as a diffuse and specular surface, respectively. In either case, the water is presumed to take the shape of a traveling sine wave, reducing the task of the 3D reconstruction to resolve the wave parameters. The first conceived method performs the modeling part purely in 3D space. Having triangulated the points in a separate phase via bundle adjustment, a sine wave is fitted into the data in a least squares manner. The second method presents a more complete approach for the entire calculation workflow beginning in the image space. The water is perceived as a specular surface, and the traveling specularities are the only observations visible to the cameras, observations that are notably single image. The depth ambiguity is removed given additional constraints encoded within the law of reflection and the modeled parametric surface. The observation and constraint equations compose a single system of equations that is solved with the method of least squares adjustment. The devised approaches are validated against the data coming from a capacitive level sensor and on physical targets floating on the surface. The outcomes agree to a high degree

SparseSat-NeRF: Dense Depth Supervised Neural Radiance Fields for Sparse Satellite Images

International audienceDigital surface model generation using traditional multi-view stereo matching (MVS) performs poorly over non-Lambertian surfaces , with asynchronous acquisitions, or at discontinuities. Neural radiance fields (NeRF) offer a new paradigm for reconstructing surface geometries using continuous volumetric representation. NeRF is self-supervised, does not require ground truth geometry for training, and provides an elegant way to include in its representation physical parameters about the scene, thus potentially remedying the challenging scenarios where MVS fails. However, NeRF and its variants require many views to produce convincing scene's geometries which in earth observation satellite imaging is rare. In this paper we present SparseSat-NeRF (SpS-NeRF)-an extension of Sat-NeRF adapted to sparse satellite views. SpS-NeRF employs dense depth supervision guided by cross-correlation similarity metric provided by traditional semi-global MVS matching. We demonstrate the effectiveness of our approach on stereo and tri-stereo Pléiades 1B/WorldView-3 images, and compare against NeRF and Sat-NeRF. The code is available at https://github.com/LulinZhang/SpS-NeR

More surface detail with One-Two-Pixel Matching

Photogrammetrically derived Digital Surface Models have been widely adopted in geoscientific applications such as mapping and change detection across volcanic surfaces, glaciers, areas of seismic activity, forests, river landforms etc. Resolution of the reconstructed surface is crucial as more accurate information enables more profound understanding of the phenomena. With this objective in mind, the research presented here proposes a new matching cost function that produces surfaces of enhanced resolution with respect to the gold standard: the window-based semi-global matching technique. We evaluate the algorithm on different image datasets spanning various acquisition geometries, radiometric qualities and ground sample distance sizes. In particular , results on Earth satellites (SPOT-7, Pléiades), extraterres-trial (Chang'E3 moon landing), aerial and terrestrial acquisitions are shown. The implementation of the method is available in MicMac-the free open-source software for photogrammetry. I. INTRODUCTION Digital surface model (DSM or photogrammetric DSM) generation using dense image matching is an accepted technique across the geoscience communities. Next to other competitive techniques such as LiDAR or radar, image-based reconstruction produces denser 3D information, it is cost-effective and richer as it includes photometric observations that allow, for example, 3D change detection or classification. Photogrammetric DSMs in geoscience applications can be generated from terrestrial images, unmanned aerial vehicle (UAV) acquisitions or high-resolution optical satellite imaging. a) Terrestrial and UAV applications: Modelling of surface roughness parameters [1]; mapping volcanic surfaces [2]; and measuring glaciers' microrelief progression [3] are some of many examples of terrestrial applications carried out with consumer grade cameras and little expert knowledge. UAV-based surveys are increasingly presented as an alternative to terrestrial surveys due to their larger reach, their ease of deployment and reduced operational cost. With respect to resolution, UAV surveys are a compromise between high-resolution close-range and moderate-resolution satellite imaging. The success of the UAV technology is reflected in numerous publications which show that UAV-collected imagery can: enable modelling of forest canopy height [4]; determine the rate and extent of landslide movements [5]; quantify coastal erosion [6], [7] and deposition processes [8] in aeolian research; map ultrafine (i.e. centimetric) tectonic faults in tectonic research [9]; or be employed in repeated surveys of the ice-sheet masses in glaciology [10]. b) Earth satellite and extraterrestrial applications: With the available optical satellite data provided by modern (e.g. Pléiades 1A/B, SPOT-satellites, QuickBird, WorldView 2/3/4, CubeSat) or older satellites (e.g. CartoSat, ASTER), we ca

Sinusoidal Wave Estimation Using Photogrammetry and Short Video Sequences

International audienc

MicMac – a free, open-source solution for photogrammetry

Abstract The publication familiarizes the reader with MicMac - a free, open-source photogrammetric software for 3D reconstruction. A brief history of the tool, its organisation and unique features vis-à-vis other software tools are in the highlight. The essential algorithmic aspects of the structure from motion and image dense matching problems are discussed from the implementation and the user’s viewpoints