On Small Satellites for Oceanography: A Survey

The recent explosive growth of small satellite operations driven primarily from an academic or pedagogical need, has demonstrated the viability of commercial-off-the-shelf technologies in space. They have also leveraged and shown the need for development of compatible sensors primarily aimed for Earth observation tasks including monitoring terrestrial domains, communications and engineering tests. However, one domain that these platforms have not yet made substantial inroads into, is in the ocean sciences. Remote sensing has long been within the repertoire of tools for oceanographers to study dynamic large scale physical phenomena, such as gyres and fronts, bio-geochemical process transport, primary productivity and process studies in the coastal ocean. We argue that the time has come for micro and nano satellites (with mass smaller than 100 kg and 2 to 3 year development times) designed, built, tested and flown by academic departments, for coordinated observations with robotic assets in situ. We do so primarily by surveying SmallSat missions oriented towards ocean observations in the recent past, and in doing so, we update the current knowledge about what is feasible in the rapidly evolving field of platforms and sensors for this domain. We conclude by proposing a set of candidate ocean observing missions with an emphasis on radar-based observations, with a focus on Synthetic Aperture Radar.Comment: 63 pages, 4 figures, 8 table

훈련 자료 자동 추출 알고리즘과 기계 학습을 통한 SAR 영상 기반의 선박 탐지

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 지구환경과학부, 2021. 2. 김덕진.Detection and surveillance of vessels are regarded as a crucial application of SAR for their contribution to the preservation of marine resources and the assurance on maritime safety. Introduction of machine learning to vessel detection significantly enhanced the performance and efficiency of the detection, but a substantial majority of studies focused on modifying the object detector algorithm. As the fundamental enhancement of the detection performance would be nearly impossible without accurate training data of vessels, this study implemented AIS information containing real-time information of vessel’s movement in order to propose a robust algorithm which acquires the training data of vessels in an automated manner. As AIS information was irregularly and discretely obtained, the exact target interpolation time for each vessel was precisely determined, followed by the implementation of Kalman filter, which mitigates the measurement error of AIS sensor. In addition, as the velocity of each vessel renders an imprint inside the SAR image named as Doppler frequency shift, it was calibrated by restoring the elliptic satellite orbit from the satellite state vector and estimating the distance between the satellite and the target vessel. From the calibrated position of the AIS sensor inside the corresponding SAR image, training data was directly obtained via internal allocation of the AIS sensor in each vessel. For fishing boats, separate information system named as VPASS was applied for the identical procedure of training data retrieval. Training data of vessels obtained via the automated training data procurement algorithm was evaluated by a conventional object detector, for three detection evaluating parameters: precision, recall and F1 score. All three evaluation parameters from the proposed training data acquisition significantly exceeded that from the manual acquisition. The major difference between two training datasets was demonstrated in the inshore regions and in the vicinity of strong scattering vessels in which land artifacts, ships and the ghost signals derived from them were indiscernible by visual inspection. This study additionally introduced a possibility of resolving the unclassified usage of each vessel by comparing AIS information with the accurate vessel detection results.전천후 지구 관측 위성인 SAR를 통한 선박 탐지는 해양 자원의 확보와 해상 안전 보장에 매우 중요한 역할을 한다. 기계 학습 기법의 도입으로 인해 선박을 비롯한 사물 탐지의 정확도 및 효율성이 향상되었으나, 이와 관련된 다수의 연구는 탐지 알고리즘의 개량에 집중되었다. 그러나, 탐지 정확도의 근본적인 향상은 정밀하게 취득된 대량의 훈련자료 없이는 불가능하기에, 본 연구에서는 선박의 실시간 위치, 속도 정보인 AIS 자료를 이용하여 인공 지능 기반의 선박 탐지 알고리즘에 사용될 훈련자료를 자동적으로 취득하는 알고리즘을 제안하였다. 이를 위해 이산적인 AIS 자료를 SAR 영상의 취득시각에 맞추어 정확하게 보간하고, AIS 센서 자체가 가지는 오차를 최소화하였다. 또한, 이동하는 산란체의 시선 속도로 인해 발생하는 도플러 편이 효과를 보정하기 위해 SAR 위성의 상태 벡터를 이용하여 위성과 산란체 사이의 거리를 정밀하게 계산하였다. 이렇게 계산된 AIS 센서의 영상 내의 위치로부터 선박 내 AIS 센서의 배치를 고려하여 선박 탐지 알고리즘의 훈련자료 형식에 맞추어 훈련자료를 취득하고, 어선에 대한 위치, 속도 정보인 VPASS 자료 역시 유사한 방법으로 가공하여 훈련자료를 취득하였다. AIS 자료로부터 취득한 훈련자료는 기존 방법대로 수동 취득한 훈련자료와 함께 인공 지능 기반 사물 탐지 알고리즘을 통해 정확도를 평가하였다. 그 결과, 제시된 알고리즘으로 취득한 훈련 자료는 수동 취득한 훈련 자료 대비 더 높은 탐지 정확도를 보였으며, 이는 기존의 사물 탐지 알고리즘의 평가 지표인 정밀도, 재현율과 F1 score를 통해 진행되었다. 본 연구에서 제안한 훈련자료 자동 취득 기법으로 얻은 선박에 대한 훈련자료는 특히 기존의 선박 탐지 기법으로는 분별이 어려웠던 항만에 인접한 선박과 산란체 주변의 신호에 대한 정확한 분별 결과를 보였다. 본 연구에서는 이와 함께, 선박 탐지 결과와 해당 지역에 대한 AIS 및 VPASS 자료를 이용하여 선박의 미식별성을 판정할 수 있는 가능성 또한 제시하였다.Chapter 1. Introduction - 1 - 1.1 Research Background - 1 - 1.2 Research Objective - 8 - Chapter 2. Data Acquisition - 10 - 2.1 Acquisition of SAR Image Data - 10 - 2.2 Acquisition of AIS and VPASS Information - 20 - Chapter 3. Methodology on Training Data Procurement - 26 - 3.1 Interpolation of Discrete AIS Data - 29 - 3.1.1 Estimation of Target Interpolation Time for Vessels - 29 - 3.1.2 Application of Kalman Filter to AIS Data - 34 - 3.2 Doppler Frequency Shift Correction - 40 - 3.2.1 Theoretical Basis of Doppler Frequency Shift - 40 - 3.2.2 Mitigation of Doppler Frequency Shift - 48 - 3.3 Retrieval of Training Data of Vessels - 53 - 3.4 Algorithm on Vessel Training Data Acquisition from VPASS Information - 61 - Chapter 4. Methodology on Object Detection Architecture - 66 - Chapter 5. Results - 74 - 5.1 Assessment on Training Data - 74 - 5.2 Assessment on AIS-based Ship Detection - 79 - 5.3 Assessment on VPASS-based Fishing Boat Detection - 91 - Chapter 6. Discussions - 110 - 6.1 Discussion on AIS-Based Ship Detection - 110 - 6.2 Application on Determining Unclassified Vessels - 116 - Chapter 7. Conclusion - 125 - 국문 요약문 - 128 - Bibliography - 130 -Maste

Ship detection on open sea and coastal environment

Synthetic Aperture Radar (SAR) is a high-resolution ground-mapping technique with the ability to effectively synthesize a large radar antenna by processing the phase of a smaller radar antenna on a moving platform like an airplane or a satellite. SAR images, due to its properties, have been the focus of many applications such as land and sea monitoring, remote sensing, mapping of surfaces, weather forecasting, among many others. Their relevance is increasing on a daily basis, thus it’s crucial to apply the best suitable method or technique to each type of data collected. Several techniques have been published in the literature so far to enhance automatic ship detection using Synthetic Aperture Radar (SAR) images, like multilook imaging techniques, polarization techniques, Constant False Alarm Rate (CFAR) techniques, Amplitude Change Detection (ACD) techniques among many others. Depending on how the information is gathered and processed, each technique presents different performance and results. Nowadays there are several ongoing SAR missions, and the need to improve ship detection, oil-spills or any kind of sea activity is fundamental to preserve and promote navigation safety as well as constant and accurate monitoring of the surroundings, for example, detection of illegal fishing activities, pollution or drug trafficking. The main objective of this MSc dissertation is to study and implement a set of algorithms for automatic ship detection using SAR images from Sentinel-1 due to its characteristics as well as its ease access. The dissertation organization is as follows: Chapter 1 presents a brief introduction to the theme of this dissertation and its aim, as well as its structure; Chapter 2 summarizes a variety of fundamental key points from historical events and developments to the SAR theory, finishing with a summary of some well-known ship detection methods; Chapter 3 presents a basic guideline to choose the best ship detection technique depending on the data type and operational scenario; Chapter 4 focus on the CFAR technique detailing the implemented algorithms. This technique was selected, given the data set available for testing in this work; Chapter 5 presents the results obtained using the implemented algorithms; Chapter 6 presents the conclusions, final remarks and future work

Sea Ice SAR Imagery Classification and Regression Based On Convolutional Neural Networks

Due to the global warming, there have been signficant reductions in the ice extent and ice thickness in the Arctic and marginal seas. Monitoring these changes in sea ice is very important for human activities including weather forecasting, natural-resource extraction, and ship navigation. Of the various sea ice monitoring activities, and sea ice and open water classification, sea ice concentration estimation has attracted significant attention due to the importance of this type of information. Satellite imagery is widely used for monitoring the ice cover. In this regard, images from synthetic aperture radar (SAR) are of interest due to their high spatial resolution. However, automated SAR imagery interpretation is a complex recognition task that requires algorithms with strong ability to learn complex features. Convolutional neural networks (CNNs) are the state-of-the-art in the image recognition field and CNNs have demonstrated an excellent ability to learn complicated image features. In this thesis, we first used a CNN-based transfer learning method to address sea ice and water classification challenge, which achieves an impressive classification accuracy (92.36%). Then sea ice concentration estimation from SAR image using CNNs is developed. The CNN models are trained from scratch using image analysis charts as ground truth. Based on the designed CNN, several studies are conducted. We first demonstrate the importance of including samples of intermediate ice concentration in our training data. Then experiments are carried out to increase the number of these samples in our dataset. The results from experiments indicate that model performance can be improved by adding more intermediate ice concentration samples from new datasets, regardless of the location, time, and sea ice features of new datasets. Another benefit of balancing the dataset is that the estimation results of intermediate ice concentrations from the CNN become more accurate. In addition, the CNN model we adopted is found to outperform other algorithms on distinguishing the marginal ice zone

Remote sensing satellite image processing techniques for image classification: a comprehensive survey

This paper is a brief survey of advance technological aspects of Digital Image Processing which are applied to remote sensing images obtained from various satellite sensors. In remote sensing, the image processing techniques can be categories in to four main processing stages: Image preprocessing, Enhancement, Transformation and Classification. Image pre-processing is the initial processing which deals with correcting radiometric distortions, atmospheric distortion and geometric distortions present in the raw image data. Enhancement techniques are applied to preprocessed data in order to effectively display the image for visual interpretation. It includes techniques to effectively distinguish surface features for visual interpretation. Transformation aims to identify particular feature of earth’s surface and classification is a process of grouping the pixels, that produces effective thematic map of particular land use and land cover

Study of groundwater properties and behaviour using geospatial techniques

Groundwater contributes a significant proportion of the earth’s freshwater and is essential to sustain life on earth, but its availability in spatial and temporal dimensions is not uniform. With the advent of efficient pumps and rural electrification, global groundwater extraction increased from 312 km3/year in the 1960s to 800 km3/year in 2000s; approximately 70% of this extraction is used for agriculture. About half of domestic human water consumption in urban areas is from groundwater. The ever-increasing dependence on groundwater has led to its depletion across various parts of the world. This trend must be reversed to sustain the critical role of groundwater. Groundwater monitoring based on validated data can provide information that can guide decision making to decrease groundwater stress on local and global scales. This thesis aims to monitor spatio-temporal changes in groundwater and related phenomena (like land subsidence) using geospatial techniques like InSAR, GRACE, GIS, data analysis and data visualisation. The over-extraction or rebound of groundwater can lead to land deformation because of the change in effective stress of underground sediments. Groundwater-induced land movement can cause damage to property and resources, and hence it must be monitored for the safety and economics of a city. This thesis explores the suitability of Persistent Scatterer Interferometric Synthetic Aperture Radar (PSInSAR) to measure land deformation and different senor-software for InSAR processing. The groundwater quantity variation and resulting land deformation for London using InSAR and Gravity Recovery and Climate Experiment (GRACE) between 2002-2010 were analysed. Long-term, decreasing, complex, non-linear patterns in the spatial and temporal domains from both InSAR and GRACE datasets were observed. The land movement velocities varied from -6 to +6 mm/year, and their reliability was validated with observed GNSS data by conducting a two-sample t-test. The average groundwater loss estimated from GRACE was found to be 9.003 MCM/year. The results demonstrate that InSAR and GRACE complement each other and can be an excellent source of monitoring groundwater for hydrologists. Then groundwater induced subsidence for London and the National Capital Territory of Delhi (NCT-Delhi) between 2016 and 2020 were studied. The land movement velocities were found to vary between -24 mm/year to +24 mm/year for London and between -18 mm/year to +30 mm/year for NCT-Delhi. This land movement was compared with observed groundwater levels and spatio-temporal variation of groundwater. A 1-D mathematical model was used to quantify land deformation for a given change in groundwater level. It was broadly observed that when large volumes of groundwater are extracted, it leads to land subsidence, and when groundwater is recharged, surface uplift is witnessed. However the local geology, did play an important role in the extent of subsidence, which was considered in the mathematical model. The increased pressure on groundwater can cause spatio-temporal changes in its quality because of various atmospheric stimulations, varied geology, variation in subsurface mineralogy and factors controlling residence times. Moreover, the variation of groundwater quality is vital for the sustainable management and safety of groundwater. Thus, the variation in groundwater quality is analysed from observed data for London between 2000 and 2020. The data samples were used from 500 wells in the London basin, and the data is provided in the free open access domain by Environment Agency. The overall groundwater in London was found to be dominant magnesium bicarbonate type which typically represents shallow fresh groundwater, and spatio-temporal variations of hardness, sodium, and dissolved oxygen (DO) were also studied. Significant variations in the range of each constituent were found, which was attributed to variation in the geology of the London Palaeogene aquifers and anthropogenic activities. All the case studies help better understand the phenomenon of spatio-temporal variation in groundwater behaviour and associated land deformation for urban cities. The research presented in this thesis can be used to determine whether groundwater is available and suitable for its intended purpose, discover pollutants, examine any spatio-temporal variations, and monitor land subsidence

Elevation and Deformation Extraction from TomoSAR

3D SAR tomography (TomoSAR) and 4D SAR differential tomography (Diff-TomoSAR) exploit multi-baseline SAR data stacks to provide an essential innovation of SAR Interferometry for many applications, sensing complex scenes with multiple scatterers mapped into the same SAR pixel cell. However, these are still influenced by DEM uncertainty, temporal decorrelation, orbital, tropospheric and ionospheric phase distortion and height blurring. In this thesis, these techniques are explored. As part of this exploration, the systematic procedures for DEM generation, DEM quality assessment, DEM quality improvement and DEM applications are first studied. Besides, this thesis focuses on the whole cycle of systematic methods for 3D & 4D TomoSAR imaging for height and deformation retrieval, from the problem formation phase, through the development of methods to testing on real SAR data. After DEM generation introduction from spaceborne bistatic InSAR (TanDEM-X) and airborne photogrammetry (Bluesky), a new DEM co-registration method with line feature validation (river network line, ridgeline, valley line, crater boundary feature and so on) is developed and demonstrated to assist the study of a wide area DEM data quality. This DEM co-registration method aligns two DEMs irrespective of the linear distortion model, which improves the quality of DEM vertical comparison accuracy significantly and is suitable and helpful for DEM quality assessment. A systematic TomoSAR algorithm and method have been established, tested, analysed and demonstrated for various applications (urban buildings, bridges, dams) to achieve better 3D & 4D tomographic SAR imaging results. These include applying Cosmo-Skymed X band single-polarisation data over the Zipingpu dam, Dujiangyan, Sichuan, China, to map topography; and using ALOS L band data in the San Francisco Bay region to map urban building and bridge. A new ionospheric correction method based on the tile method employing IGS TEC data, a split-spectrum and an ionospheric model via least squares are developed to correct ionospheric distortion to improve the accuracy of 3D & 4D tomographic SAR imaging. Meanwhile, a pixel by pixel orbit baseline estimation method is developed to address the research gaps of baseline estimation for 3D & 4D spaceborne SAR tomography imaging. Moreover, a SAR tomography imaging algorithm and a differential tomography four-dimensional SAR imaging algorithm based on compressive sensing, SAR interferometry phase (InSAR) calibration reference to DEM with DEM error correction, a new phase error calibration and compensation algorithm, based on PS, SVD, PGA, weighted least squares and minimum entropy, are developed to obtain accurate 3D & 4D tomographic SAR imaging results. The new baseline estimation method and consequent TomoSAR processing results showed that an accurate baseline estimation is essential to build up the TomoSAR model. After baseline estimation, phase calibration experiments (via FFT and Capon method) indicate that a phase calibration step is indispensable for TomoSAR imaging, which eventually influences the inversion results. A super-resolution reconstruction CS based study demonstrates X band data with the CS method does not fit for forest reconstruction but works for reconstruction of large civil engineering structures such as dams and urban buildings. Meanwhile, the L band data with FFT, Capon and the CS method are shown to work for the reconstruction of large manmade structures (such as bridges) and urban buildings

Remote sensing of sea ice properties and dynamics using SAR interferometry

Landfast ice is attached to the coastline and islands and stays immobile over most of the ice season. It is an important element of polar ecosystems and plays a vital role as a marine habitat and in life of local people and economy through offshore technology. Landfast ice is routinely used for on-ice traffic, tourism, and industry, and it protects coasts from storms in winter from erosion. However, landfast ice can break or experience deformation in order of centimeters to meters, which can be dangerous for the coastline and man-made structures, beacons, on-ice traffic, and represents a safety risk for working on the ice and local people. Therefore, landfast ice deformation and stability are important topics in coastal engineering and sea ice modeling. In the framework of this dissertation, InSAR (SAR Interferometry) technology has been applied for deriving landfast ice displacements (publication I), and mapping sea ice morphology, topography and its temporal change (publication III). Also, advantages of InSAR remote sensing in sea ice classification compared to backscatter intensity were demonstrated (publications II and IV). In publication I, for the first time, Sentinel-1 repeat-pass InSAR data acquired over the landfast ice areas were used to study the landfast ice displacements in the Gulf of Bothnia. An InSAR pair with a temporal baseline of 12 days acquired in February 2015 was used. In the study, the surface of landfast ice was stable enough to preserve coherence over the 12-day period, enabling analysis of the interferogram. The advantage of this long temporal baseline is in separating the landfast ice from drift ice and detecting long-term trends in deformation maps. The interferogram showed displacements of landfast ice on the order of 40 cm. The main factor seemed to be compression by drift ice, which was driven against the landfast ice boundary by strong winds from southwest. Landfast ice ridges can hinder ship navigation, but grounded ridges help to stabilize the ice cover. In publication III, ridge formation and displacements in the landfast ice near Utqiaġvik, Alaska were examined. The phase signatures of two single-pass bistatic X-band SAR (Synthetic Aperture Radar) image pairs acquired by TanDEM-X (TerraSAR-X add-on for Digital Elevation Measurements) satellite on 13 and 24 January 2012 were analyzed. Altogether six cases were identified with ridge displacement in four and formation in two cases under onshore compression. The ridges moved approximately 0.6 and 3.7 km over the study area and ridge formation reached up to 1 meter in upward. The results well corresponded with the locations identified as convergence zones retrieved from the drift algorithm generated by a SAR-based sea ice-tracking algorithm, backscatter intensity images and coastal radar imagery. This method could potentially be used in future to evaluate sea ice stability and ridge formation. A bistatic InSAR pair acquired by the TanDEM-X mission in March 2012 over the Bothnian Bay was used in two further studies (publications II and IV). The potential of X-band InSAR imagery for automated sea ice classification was evaluated. The first results were presented in publication II and the data were further elaborated in publication IV. The backscatter intensity, coherence magnitude and InSAR-phase features, as well as their different combinations, were used as the informative features in classification experiments. In publication II, the purpose was to assess ice properties on the scale used in ice charting, with ice types based on ice concentration and sea ice morphology, while in publication IV, a detailed small-scale analysis was performed. In addition, the sampling design was different in these publications. In publication II, to achieve the best discrimination between open water and several sea-ice types, RF (Random Forests) and ML (Maximum likelihood) classifiers were employed. The best overall accuracies were achieved by combining backscatter intensity & InSAR-phase using RF approach and backscatter intensity & coherence-magnitude using ML approach. The results showed the advantage of adding InSAR features to backscatter intensity for sea ice classification. In the further study (publication IV), a set of state-of-the-art classification approaches including ML, RF and SVM (Support Vector Machine) classifiers were used to achieve the best discrimination between open water and several sea-ice types. Adding InSAR-phase and coherence magnitude to backscatter intensity improved the OA (Overall Accuracy) compared to using only backscatter intensity. The RF and SVM algorithms gave somewhat larger OA compared to ML at the expense of a somewhat longer processing time. Results of publications II and IV demonstrate InSAR features have potential to improve sea ice classification. InSAR could be used by operational ice services to improve mapping accuracy of automated sea ice charting with statistical and machine learning classification approaches.Viime vuosikymmeninä satelliittivälitteisestä SAR-tutkasta on tullut erittäin tärkeä työkalu merijään kaukokartoituksessa. Tämän tutka perustuu sähkömagneettisten aaltojen sirontaan kiinnostavasta kohteesta takaisin tutkaan, mitä seuraa signaalin voimakkuuden mittaaminen. SAR-tutkat käyttävät synteettistä antennia, joka perustuu satelliitin liikkeeseen, mikä mahdollistaa tarkkojen, korkean erotuskyvyn kuvien tuottamisen. SAR-anturit mittaavat myös signaalin vaihetta, jota käytetään interferometria tekniikassa pinnan topografian ja siirtymien laskemiseen eri sovelluksissa, kuten maan muodonmuutoksissa, tarkassa kartoituksessa, maanjäristyksen arvioinnissa ja tulivuorenpurkauksien tarkkailussa. Interferometri tekniikkaa käytettiin tässä opinnäytetyössä pienten jäänsiirtymien analysointiin kiintojäävyöhykkeellä, joka on kiinni rantaviivassa ja saarissa eikä juuri liiku tuulien tai virtausten mukana. Kiintojääalueilla on pohjaan tarttuneita jäävalleja, jotka edistävät kiintojääpeitteen vakautumista. Kiintojäällä on tärkeä rooli merellisenä elinympäristönä, maankäytön kysymyksissä sekä paikallisten ihmisten elämässä ja meritekniikassa. Kiintojää voi murtua liikahdella useita metrejä, mikä voi olla vaarallista rakenteille, majakoille ja jäällä liikkujille. Tässä väitöskirjassa Sentinel-1A ja TanDEM-X satelliitteja ja interferometri tekniikkaa on käytetty arktisilla alueilla ja Itämerellä mittaamaan kiintojään muodonmuutoksia ja siirtymiä sekä niihin liittyviä mekanismeja. Lisäksi on tutkittu automaattista merijääluokitusta interferometrian apuohjelmiston avulla, mikä laajentaa operatiivisten merijääpalvelujen tutkahavaintojen käyttöä. Sentinel-1A:n avulla voitiin tarkastella 12 päivän pituisia muutoksia Pohjanlahden kiintojäävyöhykkeellä, kun interferometria tekniikka mittasi voimakkaan tuulen aiheuttaman 40 cm:n siirtymiä. Pohjoisella jäämerellä voitiin tunnistaa jäävallien siirtymiä ja muodostumia. Vallit siirtyivät noin 0,6 ja 3,7 km matkoja ja muodostuessaan ne kasvoivat metrin korkeuteen. Interferometri tekniikan lisääminen tutkakuvauksen analyysiin osoitti potentiaalin parantaa automaattisen merijääkartoituksen kartoituksen tarkkuutta tilastollisilla ja koneoppimiseen perustuvan luokittelun menetelmillä. Tulevaisuuden työnä merijään luokituksessa ja vallitutkimuksissa olisi suositeltavaa käyttää erilaisia ja useampia tutkakuvauksen geometrioita sekä erilaisia jääolosuhteita eri sääolosuhteiden vallitessa

Interferometric Synthetic Aperture RADAR and Radargrammetry towards the Categorization of Building Changes

The purpose of this work is the investigation of SAR techniques relying on multi image acquisition for fully automatic and rapid change detection analysis at building level. In particular, the benefits and limitations of a complementary use of two specific SAR techniques, InSAR and radargrammetry, in an emergency context are examined in term of quickness, globality and accuracy. The analysis is performed using spaceborne SAR data