ARIADNE3D seeks to develop a scalable, high performance and high functionality data storage and exploration system for integrated 3D urban remote sensing data. I have been working on ARIADNE3D since 2017. We use Hadoop, MapReduce, HBase, Spark as the core technologies to develop solutions for management, integration, and analysis of 3D urban data on distributed computing infrastructures.
Research outcome to date:
This presentation to the OGC Point Cloud DWG highlights the requirement for development of point cloud data access services to off-load heavy data management and data access tasks from point cloud application development. Most of the existing point cloud services are developed for data warehousing and data visualization purposes, which consider only raw point clouds. We envision that point cloud data can also be enriched with computation/simulation data in an incremental manner to be used as a final urban model or a new type of spatial resource for further enrichment. As enriched points clouds become more popular, data service development needs to take into account this more complex type of spatial data.
A scalable solution for handling large full waveform LiDAR datasets is introduced in this paper. The work involves a full waveform database built atop HBase. By combining a 6-dimensional Hilbert spatial code and a temporal index into a compound indexing key, the database system is capable of supporting multiple spatial, temporal, and spatio-temporal queries.
This paper presents our investigation of 4 different data models for storage of point clouds in HBase - the distributed, key-value data store within the Hadoop ecosystem. The paper shares the lessons learned and reassesses several well-known point cloud management techniques in the context of the key-value database.
RETURN (Rethinking Tunnelling in Urban Neighbourhoods) is an EU-funded ERC project aiming to create an automated pipeline from aerial laser scanning to city-scale computational modelling. I joined RETURN in 2013 as a PhD candidate at UCD School of Civil and Environmental Engineering.
My contributions to RETURN include:
This aerial data collection acquired in March 2015 include an exclusively dense airborne LiDAR dataset (discrete and full waveform at more than 300 pulses/m2), and a photogrammetry dataset (oblique and multi-spectral nadir imagery data). I was tasked with evaluating and indexing the data records for publishing on the NYU Spatial Data Repository.
This research published on ISPRS integrates an octree spatial structure with a region growing segmentation algorithm to accelerate the data processing speed without significantly compromising the output accuracy. The octree structure is used in this paper to: (1) organize the point data (i.e. indexing); (2) construct a rasterised representation of input point cloud (i.e. a simplification); and (3) define groups of neighboring points for feature estimation.
This paper published on ISPRS presents an approach for storing and indexing full waveform laser scanning data in a relational database environment, while considering both the spatial and the temporal dimensions of that data. The purpose of data indexing is, of course, to make the data searchable or accessible. Two indexing concepts (i.e. simple index and hybrid index) were implemented and evaluated. The implementations were based on Data Cartridges, an Oracle framework for developing software plugins to extend the database's behaviours (i.e. full waveform indexes and spatio-temporal queries).
This work proposes an approach for the management of discrete LiDAR point clouds in a relational database. The approach uses multiple data indexing layers: a top layer with two-dimensional, Hilbert coded, rectangular grid and a bottom layer with multiple, in-memory, 3D octrees. The hybrid indexing mechanism is aimed to speed up spatial queries. In addition, the index is able to support a region growing algorithm running on the server side.
This work, winning the First Prize in the 2015 Data Fusion Contest, introduces a strategy for detecting roads from a fusion of a dense LiDAR dataset and a high resolution imagery dataset. The data processing strategy is based on the high variations of slope and elevation along road curbs. In addition, the paper describes a complete out-of-core data pipeline, including data fusion, management, classification, and processing, which is a robust workflow for analysing a large amount of airborne LiDAR data.