geofabrics.dem

This module contains classes associated with loading, generating, and combining DEMs.

Attributes

`delayed_roughness_over_chunk`	Wrap the elevation_over_chunk routine in dask.delayed
`delayed_elevation_over_chunk`	Wrap the elevation_over_chunk routine in dask.delayed
`delayed_elevation_over_chunk_from_nearest`	Wrap the load_tiles_in_chunk routine in dask.delayed

Classes

`CoarseDem`	A class to manage coarse or background DEMs in the catchment context
`DemBase`	An abstract class to manage the different geofabric layers in a
`HydrologicallyConditionedDem`	A class to manage loading in an already created and saved dense DEM that has yet
`LidarBase`	A class with some base methods for reading in LiDAR data.
`RawDem`	A class to manage the creation of a 'raw' DEM from LiDAR tiles, and/or a
`PatchDem`	A class to manage the addition of a DEM to the foreground or background
`RoughnessDem`	A class to add a roughness (zo) layer to a hydrologically conditioned DEM.

Functions

`read_file_with_pdal`(lidar_file, region_to_tile, crs[, ...])	Read a tile file in using PDAL with input and output CRS specified.
`roughness_from_points`(point_cloud, xy_out, xy_ground, ...)	Calculate DEM elevation values at the specified locations using the selected
`elevation_from_points`(point_cloud, xy_out, options[, ...])	Calculate DEM elevation values at the specified locations using the selected
`elevation_from_nearest_points`(point_cloud, ...[, eps, ...])	Calculate DEM elevation values at the specified locations using the selected
`point_elevation`(near_z, near_points, point, options)	Calculate DEM elevation values at the specified locations using the selected
`calculate_idw`(near_points, near_z, point[, smoothing, ...])	Calculate the IDW mean of the 'near_indices' points. This implementation is based
`calculate_interpolate_griddata`(near_points, near_z, ...)	Calculate linear interpolation of the 'near_indices' points. Take the straight
`load_tiles_in_chunk`(lidar_files, source_crs, ...)	Read in all LiDAR files within the chunked region - clipped to within
`roughness_over_chunk`(dim_x, dim_y, tile_points, ...)	Rasterise all points within a chunk.
`elevation_over_chunk`(dim_x, dim_y, tile_points, options)	Rasterise all points within a chunk.
`elevation_over_chunk_from_nearest`(dim_x, dim_y, ...)	Rasterise all points within a chunk.

Module Contents

class geofabrics.dem.CoarseDem(dem_file, extents, set_foreshore=True)

A class to manage coarse or background DEMs in the catchment context

Specifically, clip within the catchment land and foreshore. There is the option to clip outside any LiDAR using the optional ‘exclusion_extent’ input.

If set_foreshore is True all positive DEM values in the foreshore are set to zero.

Parameters:

extents (dict)
set_foreshore (bool)

__del__(): Ensure the memory associated with netCDF files is properly freed.

property dem: xarray.Dataset

Return the DEM over the catchment region

Return type:: xarray.Dataset

property resolution: float

Return the largest dimension of the coarse DEM resolution

Return type:: float

property points: numpy.ndarray

The coarse DEM points after any extent or foreshore value filtering.

Return type:: numpy.ndarray

property extents: geopandas.GeoDataFrame

The extents for the coarse DEM

Return type:: geopandas.GeoDataFrame

property empty: bool

True if the DEM is empty

Return type:: bool

calculate_dem_bounds(dem): Return the bounds for a DEM.

class geofabrics.dem.DemBase(catchment_geometry, chunk_size)

Bases: abc.ABC

An abstract class to manage the different geofabric layers in a catchment context. The geofabruc has a z, and data_source layer and may sometimes also have a zo (roughness length) and lidar_source layer.

It is represented by an XArray dataset and is expected to be saved as a netCDF file.

Standard data catcegories are specified in the SOURCE_CLASSIFICATION variable.

Parameters:

catchment_geometry (geofabrics.geometry.CatchmentGeometry) – Defines the spatial extents of the catchment, land, foreshore, and offshore regions
extents – Defines the extents of any dense (LiDAR or refernence DEM) values already added.
chunk_size (int)

property dem: xarray.Dataset

Abstractmethod:
Return type:: xarray.Dataset

Return the DEM over the catchment region

save_dem(filename, dem, compression=None)

Save the DEM to a netCDF file.

Parameters:

filename (pathlib.Path) – .nc or .tif file to save the DEM.
dem (xarray.Dataset) – the DEM to save.
compression (dict) – the compression instructions if compressing.

save_and_load_dem(filename)

Update the saved file cache for the DEM (self._dem) as a netCDF file.

Parameters:: filename (pathlib.Path)

class geofabrics.dem.HydrologicallyConditionedDem(catchment_geometry, raw_dem_path, interpolation_method, chunk_size)

Bases: DemBase

A class to manage loading in an already created and saved dense DEM that has yet to have an offshore DEM associated with it.

Parameters:

behaviour (Logic controlling) –

interpolation_method
If not None, interpolate using that method. Valid options are ‘linear’, ‘nearest’, and ‘cubic’
catchment_geometry (geofabrics.geometry.CatchmentGeometry)
raw_dem_path (str | pathlib.Path)
interpolation_method (str)

__del__(): Ensure the memory associated with netCDF files is properly freed.

property raw_extents: Return the combined DEM from tiles and any interpolated offshore values

property dem: Return the combined DEM from tiles and any interpolated offshore values

interpolate_ocean_chunked(ocean_points, cache_path, k_nearest_neighbours, use_edge, buffer, method)

Create a ‘raw’’ DEM from a set of tiled LiDAR files. Read these in over non-overlapping chunks and then combine

Parameters:

cache_path (pathlib.Path)
k_nearest_neighbours (int)
use_edge (bool)
buffer (int)
method (str)

Return type:

xarray.Dataset

interpolate_ocean_bathymetry(bathy_contours, method='linear'): Performs interpolation offshore outside LiDAR extents using the SciPy RBF function.

clip_within_polygon(polygon_paths, label)

Clip existing DEM to remove areas within the polygons

Parameters:

polygon_paths (list)
label (str)

add_points_within_polygon_chunked(elevations, method, cache_path, label, include_edges=True)

Performs interpolation from estimated bathymetry points within a polygon using the specified interpolation approach after filtering the points based on the type label. The type_label also determines the source classification.

Parameters:

elevations (geofabrics.geometry.ElevationPoints)
method (str)
cache_path (pathlib.Path)
label (str)
include_edges (bool)

Return type:

xarray.Dataset

add_points_within_polygon_nearest_chunked(elevations, method, cache_path, label, k_nearest_neighbours, include_edges=True)

Performs interpolation from estimated bathymetry points within a polygon using the specified interpolation approach after filtering the points based on the type label. The type_label also determines the source classification.

Parameters:

elevations (geofabrics.geometry.ElevationPoints)
method (str)
cache_path (pathlib.Path)
label (str)
k_nearest_neighbours (int)
include_edges (bool)

Return type:

xarray.Dataset

class geofabrics.dem.LidarBase(catchment_geometry, chunk_size, elevation_range=None)

Bases: DemBase

A class with some base methods for reading in LiDAR data.

Parameters:

catchment_geometry (geofabrics.geometry.CatchmentGeometry) – Defines the geometry of the catchment
elevation_range (list) – The range of valid LiDAR elevations. i.e. define elevation filtering to apply.
chunk_size (int)

__del__(): Ensure the memory associated with netCDF files is properly freed.

property dem: Return the positivly indexed DEM from tiles

abstract add_lidar(lidar_datasets_info, chunk_size, lidar_classifications_to_keep, metadata)

Read in all LiDAR files and use to create a ‘raw’ DEM.

Parameters:

lidar_datasets_info (dict) – A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file.
chunk_size (int) – The chunk size in pixels for parallel/staged processing
lidar_classifications_to_keep (list) – A list of LiDAR classifications to keep - ‘2’ for ground, ‘9’ for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list
meta_data – Information to include in the created DEM - must include dataset_mapping key if datasets (not a single LAZ file) included.
metadata (dict)

class geofabrics.dem.RawDem(catchment_geometry, lidar_interpolation_method, drop_offshore_lidar, zero_positive_foreshore, buffer_cells, elevation_range=None, chunk_size=None)

Bases: LidarBase

A class to manage the creation of a ‘raw’ DEM from LiDAR tiles, and/or a coarse DEM.

Parameters:

drop_offshore_lidar (dict) – If True only keep LiDAR values within the foreshore and land regions defined by the catchment_geometry. If False keep all LiDAR values.
elevation_range (list | None) – Optitionally specify a range of valid elevations. Any LiDAR points with elevations outside this range will be filtered out.
lidar_interpolation_method (str) – The interpolation method to apply to LiDAR during downsampling/averaging. Options are: mean, median, IDW, max, min, STD.
for (buffer_cells - the number of empty cells to keep around LiDAR cells) – interpolation after the coarse DEM added to ensure a smooth boundary.
chunk_size (int | None) – The chunk size in pixels for parallel/staged processing
catchment_geometry (geofabrics.geometry.CatchmentGeometry)
zero_positive_foreshore (bool)
buffer_cells (int)

add_lidar(lidar_datasets_info, lidar_classifications_to_keep, metadata)

Read in all LiDAR files and use to create a ‘raw’ DEM.

Parameters:

lidar_datasets_info (dict) – A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file.
lidar_classifications_to_keep (list) – A list of LiDAR classifications to keep - ‘2’ for ground, ‘9’ for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list
meta_data – Information to include in the created DEM - must include dataset_mapping key if datasets (not a single LAZ file) included.
metadata (dict)

clip_lidar(): Clip the a ‘raw’ DEM. Should be called immediately after the add_lidar function.

class geofabrics.dem.PatchDem(catchment_geometry, patch_on_top, drop_patch_offshore, zero_positive_foreshore, buffer_cells, initial_dem_path, elevation_range=None, chunk_size=None)

Bases: LidarBase

A class to manage the addition of a DEM to the foreground or background of a preexisting DEM.

Parameters:

patch_on_top (bool) – If True only patch the DEM values on top of the initial DEM. If False patch only where values are NaN.
drop_patch_offshore (bool) – If True only keep patch values on land and the foreshore.
elevation_range (list | None) – Optitionally specify a range of valid elevations. Any LiDAR points with elevations outside this range will be filtered out.
initial_dem_path (str | pathlib.Path) – The DEM to patch the other DEM on top / only where values are NaN.
for (buffer_cells - the number of empty cells to keep around LiDAR cells) – interpolation after the coarse DEM added to ensure a smooth boundary.
chunk_size (int | None) – The chunk size in pixels for parallel/staged processing
catchment_geometry (geofabrics.geometry.CatchmentGeometry)
zero_positive_foreshore (bool)
buffer_cells (int)

add_patch(patch_path, label, layer)

Check if gaps in DEM on land, if so iterate through coarse DEMs adding missing detail.

Note the Coarse DEM values are only applied on land and not in the ocean.

Parameters:

add (patch_path - patch file path to try)
data_source (label - either "coarse DEM" or "patch". Defines the)
on (layer - either 'z' or 'zo' and the layer to set the patch)
patch_path (pathlib.Path)
label (str)
layer (str)

property no_values_mask: No values mask from DEM within land and foreshore region

class geofabrics.dem.RoughnessDem(catchment_geometry, hydrological_dem_path, temp_folder, interpolation_method, default_values, drop_offshore_lidar, chunk_size=None, elevation_range=None)

Bases: LidarBase

A class to add a roughness (zo) layer to a hydrologically conditioned DEM.

They STD and mean height of ground cover classified points are calculated from the LiDAR data and z (elevation) layer of the hydrologically conditioned DEM, and used to estimate roughness emperically.

RoughnessDem logic can be controlled by the constructor inputs.

Parameters:

catchment_geometry (geofabrics.geometry.CatchmentGeometry) – Defines the geometry of the catchment
hydrological_dem_path (str | pathlib.Path) – The path to the hydrologically conditioned DEM.
interpolation_method (str) – If not None, interpolate using that method. Valid options are ‘linear’, ‘nearest’, and ‘cubic’.
lidar_interpolation_method – The interpolation method to apply to LiDAR. Options are: mean, median, IDW.
temp_folder (pathlib.Path)
default_values (dict)
drop_offshore_lidar (dict)
chunk_size (int | None)
elevation_range (list)

add_lidar(lidar_datasets_info, lidar_classifications_to_keep, metadata, parameters)

Read in all LiDAR files and use the point cloud distribution, data_source layer, and hydrologiaclly conditioned elevations to estimate the roughness across the DEM.

Parameters:

lidar_datasets_info (dict) – A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file.
lidar_classifications_to_keep (list) – A list of LiDAR classifications to keep - ‘2’ for ground, ‘9’ for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list
meta_data – Information to include in the created DEM - must include dataset_mapping key if datasets (not a single LAZ file) included.
parameters (dict) – The roughness equation parameters.
metadata (dict)

add_roads(roads_polygon)

Set roads to paved and unpaved roughness values.

Parameters:: roads_polygon (dict) – Dataframe with polygon and associated roughness values

geofabrics.dem.read_file_with_pdal(lidar_file, region_to_tile, crs, source_crs=None)

Read a tile file in using PDAL with input and output CRS specified.

Parameters:

lidar_file (str | pathlib.Path)
region_to_tile (geopandas.GeoDataFrame)
crs (dict)
source_crs (dict)

geofabrics.dem.roughness_from_points(point_cloud, xy_out, xy_ground, options, eps=0, leaf_size=10)

Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree

Parameters:

point_cloud (numpy.ndarray)
xy_out (numpy.ndarray)
xy_ground (numpy.ndarray)
options (dict)
eps (float)
leaf_size (int)

Return type:

numpy.ndarray

geofabrics.dem.elevation_from_points(point_cloud, xy_out, options, eps=0, leaf_size=10)

Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree

Parameters:

point_cloud (numpy.ndarray)
options (dict)
eps (float)
leaf_size (int)

Return type:

numpy.ndarray

geofabrics.dem.elevation_from_nearest_points(point_cloud, edge_point_cloud, xy_out, options, eps=0, leaf_size=10)

Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree

Parameters:

point_cloud (numpy.ndarray)
edge_point_cloud (numpy.ndarray)
options (dict)
eps (float)
leaf_size (int)

Return type:

numpy.ndarray

geofabrics.dem.point_elevation(near_z, near_points, point, options)

Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree

Parameters:

near_z (numpy.ndarray)
near_points (numpy.ndarray)
point (numpy.ndarray)
options (dict)

Return type:

float

geofabrics.dem.calculate_idw(near_points, near_z, point, smoothing=0, power=2)

Calculate the IDW mean of the ‘near_indices’ points. This implementation is based on the scipy.spatial.KDTree

Parameters:

near_points (numpy.ndarray)
near_z (numpy.ndarray)
point (numpy.ndarray)
smoothing (float)
power (int)

geofabrics.dem.calculate_interpolate_griddata(near_points, near_z, point, strict, method)

Calculate linear interpolation of the ‘near_indices’ points. Take the straight mean if the points are co-linear or too few for linear interpolation.

Parameters:

near_points (numpy.ndarray)
near_z (numpy.ndarray)
point (numpy.ndarray)
strict (bool)
method (str)

geofabrics.dem.load_tiles_in_chunk(lidar_files, source_crs, chunk_region_to_tile, crs)

Read in all LiDAR files within the chunked region - clipped to within the region within which to rasterise.

Parameters:

lidar_files (List[pathlib.Path])
source_crs (dict)
chunk_region_to_tile (geopandas.GeoDataFrame)
crs (dict)

geofabrics.dem.roughness_over_chunk(dim_x, dim_y, tile_points, xy_ground, options)

Rasterise all points within a chunk.

Parameters:

dim_x (numpy.ndarray)
dim_y (numpy.ndarray)
tile_points (numpy.ndarray)
xy_ground (numpy.ndarray)
options (dict)

Return type:

numpy.ndarray

geofabrics.dem.elevation_over_chunk(dim_x, dim_y, tile_points, options)

Rasterise all points within a chunk.

Parameters:

dim_x (numpy.ndarray)
dim_y (numpy.ndarray)
tile_points (numpy.ndarray)
options (dict)

Return type:

numpy.ndarray

geofabrics.dem.elevation_over_chunk_from_nearest(dim_x, dim_y, points, edge_points, options)

Rasterise all points within a chunk.

Parameters:

dim_x (numpy.ndarray)
dim_y (numpy.ndarray)
points (numpy.ndarray)
edge_points (numpy.ndarray)
options (dict)

Return type:

numpy.ndarray

geofabrics.dem.delayed_roughness_over_chunk: Wrap the elevation_over_chunk routine in dask.delayed

geofabrics.dem.delayed_elevation_over_chunk: Wrap the elevation_over_chunk routine in dask.delayed

geofabrics.dem.delayed_elevation_over_chunk_from_nearest: Wrap the load_tiles_in_chunk routine in dask.delayed