Behind the Scenes of LiDAR Data Processing: A Mission Coordinator’s Perspective

LiDAR (Light Detection and Ranging) technology is an essential tool for geospatial data collection and analysis, allowing for the creation of highly accurate 3D models of the environment. However, the data collected by LiDAR sensors need to be processed and analyzed before it can be used for further purposes. To learn more about this process, we talked to Matthew Berry, mission coordinator at FlyGuys, who is responsible for processing all of our LiDAR jobs. Matthew broke it down for us, discussing the steps involved in collecting,  processing and analyzing LiDAR data.

Software Used for Processing and Analyzing LiDAR Data

LiDAR data is processed using specialized software designed for point cloud creation and editing. According to Berry, the most commonly used software for processing and analyzing LiDAR data includes

• Rock point cloud creation software: PCMasterGL, PCPainterGL
• Point cloud editing software: LiDAR360

Generating Point Clouds from LiDAR Data

To generate point clouds from LiDAR data, Berry explains that the trajectory of the drone used for data collection is used to assemble the points into a 3D georeferenced point cloud. However, it is important to remove noise and outliers from the data to produce cleaner and more accurate point clouds.

The process for removing noise and outliers from LiDAR data involves using feature data from the point cloud, which is input into software algorithms. Berry notes, “This process helps identify and remove false points created by interference with the sensor and outliers, which allows for the production of cleaner and more accurate point clouds.”

Classifying Ground, Buildings, and Vegetation in LiDAR Data

According to Berry, the classification of ground, buildings, and vegetation in LiDAR data is essential for building highly accurate models. The ground is classified using feature data such as building size, ground slope, point distance, and vegetation density, which is then input into TIN software algorithms to identify and classify all points that make up the ground surface. Buildings and vegetation are classified using a combination of manual classification and machine learning. The LiDAR processor manually classifies a small section of the project that contains key examples of the features in the data. This model is then used to classify the remaining section through machine learning, which is then inspected and edited as needed by the LiDAR processor.

Assessing and Verifying the Accuracy of LiDAR Data

“Accuracy is a crucial aspect of LiDAR data,” Berry states. “And it needs to be assessed and verified before it can be used for further purposes.” He further explains that FlyGuys takes ground control data using a sub-centimeter RTK rover in conjunction with LiDAR data to ground truth and assess the accuracy of the data. 2’x2′ white and black targets are used, and check shots are taken across the site, with horizontal and vertical accuracy needing to be within 6 cm for all controls. “Accuracy reports are included on all jobs,” he adds.

Common Data Formats for LiDAR Data

“There are several common data formats used for LiDAR data, including point clouds, rasters, and vectors.” Berry breaks it down for us.

• Point clouds are a collection of individual data points in a three-dimensional plane with each point having a set coordinate on the X, Y, and Z axis.
  • LAS: This is the most common file type for uncompressed 3d point clouds, contains XYZ geographic information of the points along with various other data, often very large
  • LAZ: a compressed LAS
  • TXT: text file usually comma delimited containing xyz position of specific points (ex. PNEZ, notepad)
  • CSV: a table structured text file usually comma delimited containing xyz position of specific points (ex. PNEZ, excel)
• Rasters are a type of digital image data that is composed of a grid of cells, each with a value representing some attribute of the terrain or surface being captured.
  • TIF: 2D images that contain X and Y geographic information as well as an additional user-defined attribute, usually elevation (geographic Z). Often used in GIS/CAD software (ex. DEM, Orthomosaic)
• Vectors are mathematical representations of a quantity that has both magnitude and direction.
  • SHP: a user-defined 3D drawing that contains XYZ geographic information, used primarily in GIS software (ex. Contours, TIN)
  • DWG: a user-defined 3D drawing that contains XYZ geographic information, used primarily in CAD software (ex. Contours, TIN)
  • DXF: an interchange format for moving files between CAD software

Integration of LiDAR Data with Other Types

“Most LiDAR data already includes GPS coordinates for each point in the point cloud, making it easy to integrate with GPS data from other sources,” explains Berry. “In fact, by combining GPS data from different sources, it’s possible to create a more accurate and comprehensive representation of the environment.” The process involves extracting feature data from the LiDAR data and overlaying it on top of other datasets. “For example, contours can be overlaid over Orthomosaic images to provide a more detailed representation of the area’s topography.”

Integrating LiDAR data with other datasets can also improve the accuracy of the data. Berry continues, “For instance, overlaying LiDAR data with aerial imagery can help identify features that may not be visible in the LiDAR data alone, such as tree canopies or building rooftops. This information can then be used to refine the LiDAR data and improve the accuracy of the resulting 3D models.”

“The process is relatively straightforward, and the resulting datasets can be used to gain valuable insights into the environment, particularly for applications such as land management, urban planning, and disaster response.”

Challenges of LiDAR

Despite its many benefits, processing and analyzing LiDAR data can also present challenges. One of the main challenges is ensuring the accuracy of the data. “With jobs consisting of multiple days worth of flights, each flight needs to be carefully monitored for geographic accuracy,” Berry explains. “The accuracy can be affected by factors such as environmental conditions, sensor calibration, and flight planning.” To overcome this challenge, Berry says that once a flight’s geographic accuracy is confirmed, it then needs to be merged with other flights to create a comprehensive data set. Key features such as buildings, ground control points (GCPs), and roads can help ensure the accuracy of the data is maintained.

Another challenge is the sheer volume of data generated by LiDAR. The vast number of data points can make it difficult to manage and process the data effectively. Furthermore, analyzing and interpreting the data requires specialized software and expertise.

Data processing can also be time-consuming and expensive. Cleaning, filtering, and classifying the point cloud data to produce useful outputs can be a complex and time-consuming process.

To address these challenges, Berry acknowledges how important it is to have clear communication between the client and sales team regarding the purpose of the job and data needs. “It is also essential to have experienced and skilled data analysts and technicians who can manage the data and apply appropriate processing and analysis techniques.” With proper planning and execution, LiDAR data can be a valuable tool for a wide range of applications in fields such as surveying, mapping, forestry, and construction.

How long it takes to process LiDAR data

“Processing LiDAR data can be a time-consuming task,” Berry explains. “The amount of time it takes to process LiDAR data depends on a few factors, including the amount of data collected and the complexity of the data processing required. Typically, it takes about one to two days of processing for each day of flight data collected.”

In addition to the time required for processing, additional time may be required to process specific classes of data. For example, if the LiDAR data needs to be classified into different classes, such as ground, vegetation, and buildings, it may take an additional one to two days of processing time per class.

Once the LiDAR data is received from the pilot, it can take an additional one to two days to process the data. During this time, the data is processed, classified, and analyzed to create a detailed and accurate 3D model of the environment.

It’s worth noting that the time required to process LiDAR data can vary depending on the software used and the processing capabilities of the computer system. In general, it’s important to be patient and allow enough time for the data processing to be completed thoroughly and accurately. Rushing the process can lead to errors and inaccuracies in the final 3D model.

Steps to LiDAR

If you read our last blog interview with Zack, our LiDAR pilot, you know that once he is done flying a mission, he sends the logged data to Matthew. Here, Matthew tells us the steps he takes once he receives the data from Zack. 

“Once I get the logged data from Zack, the first step is to download the data onto the processing computer. The next step is to process the initial cloud in PCMaster, which is the software used to process and analyze LiDAR data. After the initial cloud is processed, it is colorized using PCPainter.

“The processed cloud is then imported into L360, which is another LiDAR processing software, and transformed to the client coordinate system. Outliers are removed, points are smoothed, and ground classification is done to ensure that the data is clean and accurate.

“If the client requests GCPs, they are imported, and an initial alignment is done. A control report is generated, and a second alignment is done. Another control report is generated, and a final alignment is done if needed. If not, the control report is exported.

“If the client requests additional classes, those are done at this point. Finally, the deliverables such as contours, DEMs, TIN, and CSV files are created. The full and ground LAS files are exported, zipped, and uploaded to the portal. The client contact is alerted once the data is ready for download.

“The entire process takes 1-2 days of processing for each day of flight, plus an additional 1-2 days per requested class. The turnaround time may vary depending on the size and complexity of the project.”

The Future of LiDAR

LiDAR technology has come a long way in recent years, and its application in various fields has significantly increased. The availability of more advanced systems and data processing software has made it possible to collect high-resolution data quickly and accurately.

The future of LiDAR is promising and exciting. It is likely that the demand for LiDAR data will continue to increase in the coming years, as more industries recognize its benefits, leading to even more innovative uses for the technology. As technology continues to advance, LiDAR sensors will become smaller, more affordable, and more accessible to industries beyond surveying and mapping. 

With the increased availability of LiDAR data, we can expect to see new applications and analyses that were previously impossible or too expensive to perform. LiDAR data will continue to be integrated with other data types, such as aerial imagery and GPS, providing more comprehensive and accurate information for a wide range of applications. The possibilities are endless, and the potential for LiDAR to transform how we perceive and interact with our environment is truly remarkable.

Discover more with LiDAR. Contact us today to schedule a LiDAR consultation.