geofabrics.dem ============== .. py:module:: geofabrics.dem .. autoapi-nested-parse:: This module contains classes associated with loading, generating, and combining DEMs. Attributes ---------- .. autoapisummary:: geofabrics.dem.delayed_roughness_over_chunk geofabrics.dem.delayed_elevation_over_chunk geofabrics.dem.delayed_elevation_over_chunk_from_nearest Classes ------- .. autoapisummary:: geofabrics.dem.CoarseDem geofabrics.dem.DemBase geofabrics.dem.HydrologicallyConditionedDem geofabrics.dem.LidarBase geofabrics.dem.RawDem geofabrics.dem.PatchDem geofabrics.dem.RoughnessDem Functions --------- .. autoapisummary:: geofabrics.dem.read_file_with_pdal geofabrics.dem.roughness_from_points geofabrics.dem.elevation_from_points geofabrics.dem.elevation_from_nearest_points geofabrics.dem.point_elevation geofabrics.dem.calculate_idw geofabrics.dem.calculate_interpolate_griddata geofabrics.dem.load_tiles_in_chunk geofabrics.dem.roughness_over_chunk geofabrics.dem.elevation_over_chunk geofabrics.dem.elevation_over_chunk_from_nearest Module Contents --------------- .. py:class:: CoarseDem(dem_file, extents, set_foreshore = True) A class to manage coarse or background DEMs in the catchment context Specifically, clip within the catchment land and foreshore. There is the option to clip outside any LiDAR using the optional 'exclusion_extent' input. If set_foreshore is True all positive DEM values in the foreshore are set to zero. .. py:method:: __del__() Ensure the memory associated with netCDF files is properly freed. .. py:property:: dem :type: xarray.Dataset Return the DEM over the catchment region .. py:property:: resolution :type: float Return the largest dimension of the coarse DEM resolution .. py:property:: points :type: numpy.ndarray The coarse DEM points after any extent or foreshore value filtering. .. py:property:: extents :type: geopandas.GeoDataFrame The extents for the coarse DEM .. py:property:: empty :type: bool True if the DEM is empty .. py:method:: calculate_dem_bounds(dem) Return the bounds for a DEM. .. py:class:: DemBase(catchment_geometry, chunk_size) Bases: :py:obj:`abc.ABC` An abstract class to manage the different geofabric layers in a catchment context. The geofabruc has a z, and data_source layer and may sometimes also have a zo (roughness length) and lidar_source layer. It is represented by an XArray dataset and is expected to be saved as a netCDF file. Standard data catcegories are specified in the SOURCE_CLASSIFICATION variable. :param catchment_geometry: Defines the spatial extents of the catchment, land, foreshore, and offshore regions :param extents: Defines the extents of any dense (LiDAR or refernence DEM) values already added. .. py:property:: dem :type: xarray.Dataset :abstractmethod: Return the DEM over the catchment region .. py:method:: save_dem(filename, dem, compression = None) Save the DEM to a netCDF file. :param filename: .nc or .tif file to save the DEM. :param dem: the DEM to save. :param compression: the compression instructions if compressing. .. py:method:: save_and_load_dem(filename) Update the saved file cache for the DEM (self._dem) as a netCDF file. .. py:class:: HydrologicallyConditionedDem(catchment_geometry, raw_dem_path, interpolation_method, chunk_size) Bases: :py:obj:`DemBase` A class to manage loading in an already created and saved dense DEM that has yet to have an offshore DEM associated with it. :param Logic controlling behaviour: interpolation_method If not None, interpolate using that method. Valid options are 'linear', 'nearest', and 'cubic' .. py:method:: __del__() Ensure the memory associated with netCDF files is properly freed. .. py:property:: raw_extents Return the combined DEM from tiles and any interpolated offshore values .. py:property:: dem Return the combined DEM from tiles and any interpolated offshore values .. py:method:: interpolate_ocean_chunked(ocean_points, cache_path, k_nearest_neighbours, use_edge, buffer, method) Create a 'raw'' DEM from a set of tiled LiDAR files. Read these in over non-overlapping chunks and then combine .. py:method:: interpolate_ocean_bathymetry(bathy_contours, method='linear') Performs interpolation offshore outside LiDAR extents using the SciPy RBF function. .. py:method:: clip_within_polygon(polygon_paths, label) Clip existing DEM to remove areas within the polygons .. py:method:: add_points_within_polygon_chunked(elevations, method, cache_path, label, include_edges = True) Performs interpolation from estimated bathymetry points within a polygon using the specified interpolation approach after filtering the points based on the type label. The type_label also determines the source classification. .. py:method:: add_points_within_polygon_nearest_chunked(elevations, method, cache_path, label, k_nearest_neighbours, include_edges = True) Performs interpolation from estimated bathymetry points within a polygon using the specified interpolation approach after filtering the points based on the type label. The type_label also determines the source classification. .. py:class:: LidarBase(catchment_geometry, chunk_size, elevation_range = None) Bases: :py:obj:`DemBase` A class with some base methods for reading in LiDAR data. :param catchment_geometry: Defines the geometry of the catchment :param elevation_range: The range of valid LiDAR elevations. i.e. define elevation filtering to apply. .. py:method:: __del__() Ensure the memory associated with netCDF files is properly freed. .. py:property:: dem Return the positivly indexed DEM from tiles .. py:method:: add_lidar(lidar_datasets_info, chunk_size, lidar_classifications_to_keep, metadata) :abstractmethod: Read in all LiDAR files and use to create a 'raw' DEM. :param lidar_datasets_info: A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file. :param chunk_size: The chunk size in pixels for parallel/staged processing :param lidar_classifications_to_keep: A list of LiDAR classifications to keep - '2' for ground, '9' for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list :param meta_data: Information to include in the created DEM - must include `dataset_mapping` key if datasets (not a single LAZ file) included. .. py:class:: RawDem(catchment_geometry, lidar_interpolation_method, drop_offshore_lidar, zero_positive_foreshore, buffer_cells, elevation_range = None, chunk_size = None) Bases: :py:obj:`LidarBase` A class to manage the creation of a 'raw' DEM from LiDAR tiles, and/or a coarse DEM. :param drop_offshore_lidar: If True only keep LiDAR values within the foreshore and land regions defined by the catchment_geometry. If False keep all LiDAR values. :param elevation_range: Optitionally specify a range of valid elevations. Any LiDAR points with elevations outside this range will be filtered out. :param lidar_interpolation_method: The interpolation method to apply to LiDAR during downsampling/averaging. Options are: mean, median, IDW, max, min, STD. :param buffer_cells - the number of empty cells to keep around LiDAR cells for: interpolation after the coarse DEM added to ensure a smooth boundary. :param chunk_size: The chunk size in pixels for parallel/staged processing .. py:method:: add_lidar(lidar_datasets_info, lidar_classifications_to_keep, metadata) Read in all LiDAR files and use to create a 'raw' DEM. :param lidar_datasets_info: A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file. :param lidar_classifications_to_keep: A list of LiDAR classifications to keep - '2' for ground, '9' for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list :param meta_data: Information to include in the created DEM - must include `dataset_mapping` key if datasets (not a single LAZ file) included. .. py:method:: clip_lidar() Clip the a 'raw' DEM. Should be called immediately after the add_lidar function. .. py:class:: PatchDem(catchment_geometry, patch_on_top, drop_patch_offshore, zero_positive_foreshore, buffer_cells, initial_dem_path, elevation_range = None, chunk_size = None) Bases: :py:obj:`LidarBase` A class to manage the addition of a DEM to the foreground or background of a preexisting DEM. :param patch_on_top: If True only patch the DEM values on top of the initial DEM. If False patch only where values are NaN. :param drop_patch_offshore: If True only keep patch values on land and the foreshore. :param elevation_range: Optitionally specify a range of valid elevations. Any LiDAR points with elevations outside this range will be filtered out. :param initial_dem_path: The DEM to patch the other DEM on top / only where values are NaN. :param buffer_cells - the number of empty cells to keep around LiDAR cells for: interpolation after the coarse DEM added to ensure a smooth boundary. :param chunk_size: The chunk size in pixels for parallel/staged processing .. py:method:: add_patch(patch_path, label, layer) Check if gaps in DEM on land, if so iterate through coarse DEMs adding missing detail. Note the Coarse DEM values are only applied on land and not in the ocean. :param patch_path - patch file path to try add: :param label - either "coarse DEM" or "patch". Defines the data_source: :param layer - either 'z' or 'zo' and the layer to set the patch on: .. py:property:: no_values_mask No values mask from DEM within land and foreshore region .. py:class:: RoughnessDem(catchment_geometry, hydrological_dem_path, temp_folder, interpolation_method, default_values, drop_offshore_lidar, chunk_size = None, elevation_range = None) Bases: :py:obj:`LidarBase` A class to add a roughness (zo) layer to a hydrologically conditioned DEM. They STD and mean height of ground cover classified points are calculated from the LiDAR data and z (elevation) layer of the hydrologically conditioned DEM, and used to estimate roughness emperically. RoughnessDem logic can be controlled by the constructor inputs. :param catchment_geometry: Defines the geometry of the catchment :param hydrological_dem_path: The path to the hydrologically conditioned DEM. :param interpolation_method: If not None, interpolate using that method. Valid options are 'linear', 'nearest', and 'cubic'. :param lidar_interpolation_method: The interpolation method to apply to LiDAR. Options are: mean, median, IDW. .. py:method:: add_lidar(lidar_datasets_info, lidar_classifications_to_keep, metadata, parameters) Read in all LiDAR files and use the point cloud distribution, data_source layer, and hydrologiaclly conditioned elevations to estimate the roughness across the DEM. :param lidar_datasets_info: A dictionary of information for each specified LIDAR dataset - For each this includes: a list of LAS files, CRS, and tile index file. :param lidar_classifications_to_keep: A list of LiDAR classifications to keep - '2' for ground, '9' for water. See https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_4_r13.pdf for standard list :param meta_data: Information to include in the created DEM - must include `dataset_mapping` key if datasets (not a single LAZ file) included. :param parameters: The roughness equation parameters. .. py:method:: add_roads(roads_polygon) Set roads to paved and unpaved roughness values. :param roads_polygon: Dataframe with polygon and associated roughness values .. py:function:: read_file_with_pdal(lidar_file, region_to_tile, crs, source_crs = None) Read a tile file in using PDAL with input and output CRS specified. .. py:function:: roughness_from_points(point_cloud, xy_out, xy_ground, options, eps = 0, leaf_size = 10) Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree .. py:function:: elevation_from_points(point_cloud, xy_out, options, eps = 0, leaf_size = 10) Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree .. py:function:: elevation_from_nearest_points(point_cloud, edge_point_cloud, xy_out, options, eps = 0, leaf_size = 10) Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree .. py:function:: point_elevation(near_z, near_points, point, options) Calculate DEM elevation values at the specified locations using the selected approach. Options include: mean, median, and inverse distance weighing (IDW). This implementation is based on the scipy.spatial.KDTree .. py:function:: calculate_idw(near_points, near_z, point, smoothing = 0, power = 2) Calculate the IDW mean of the 'near_indices' points. This implementation is based on the scipy.spatial.KDTree .. py:function:: calculate_interpolate_griddata(near_points, near_z, point, strict, method) Calculate linear interpolation of the 'near_indices' points. Take the straight mean if the points are co-linear or too few for linear interpolation. .. py:function:: load_tiles_in_chunk(lidar_files, source_crs, chunk_region_to_tile, crs) Read in all LiDAR files within the chunked region - clipped to within the region within which to rasterise. .. py:function:: roughness_over_chunk(dim_x, dim_y, tile_points, xy_ground, options) Rasterise all points within a chunk. .. py:function:: elevation_over_chunk(dim_x, dim_y, tile_points, options) Rasterise all points within a chunk. .. py:function:: elevation_over_chunk_from_nearest(dim_x, dim_y, points, edge_points, options) Rasterise all points within a chunk. .. py:data:: delayed_roughness_over_chunk Wrap the `elevation_over_chunk` routine in dask.delayed .. py:data:: delayed_elevation_over_chunk Wrap the `elevation_over_chunk` routine in dask.delayed .. py:data:: delayed_elevation_over_chunk_from_nearest Wrap the `load_tiles_in_chunk` routine in dask.delayed