data_sources.nwp

NWP data sources and functions

data_sources.nwp.nwp_model

Model for output of NWP data

NWP Objects

class NWP(DataSourceOutput)

Class to store NWP data as a xr.Dataset with some validation

model_validation

@classmethod
def model_validation(cls, v)

Check that all values are not NaNs

data_sources.nwp.nwp_data_source

NWP Data Source

NWPDataSource Objects

@dataclass
class NWPDataSource(ZarrDataSource)

NWP Data Source (Numerical Weather Predictions)

Args (for init): filename: The base path in which we find '2018_1-6', etc.

Attributes:

  • _data - xr.DataArray of Numerical Weather Predictions, opened by open(). x is left-to-right. y is top-to-bottom. Access using public nwp property.
  • consolidated - Whether or not the Zarr store is consolidated.
  • channels - The NWP forecast parameters to load. If None then don't filter. The available params are: t : Temperature in Kelvin. dswrf : Downward short-wave radiation flux in W/m^2 (irradiance). prate : Precipitation rate in kg/m^2/s. r : Relative humidty in %. sde : Snow depth in meters. si10 : 10-meter wind speed in m/s. vis : Visibility in meters. lcc : Low-level cloud cover in %. mcc : Medium-level cloud cover in %. hcc : High-level cloud cover in %.

__post_init__

def __post_init__(image_size_pixels: int, meters_per_pixel: int)

Post init

Arguments:

  • image_size_pixels - number of pixels in image
  • meters_per_pixel - how many meteres for each pixel

open

def open() -> None

Open NWP data

We don't want to open_sat_data in init. If we did that, then we couldn't copy NWPDataSource instances into separate processes. Instead, call open() after creating separate processes.

get_batch

def get_batch(t0_datetimes: pd.DatetimeIndex, x_locations: Iterable[Number], y_locations: Iterable[Number]) -> NWP

Get batch data

Arguments:

  • t0_datetimes - list of timstamps
  • x_locations - list of x locations, where the batch data is for

  • y_locations - list of y locations, where the batch data is for

  • Returns - batch data

datetime_index

def datetime_index() -> pd.DatetimeIndex

Returns a complete list of all available datetimes

open_nwp

def open_nwp(filename: str, consolidated: bool) -> xr.Dataset

Open The NWP data

Arguments:

  • filename - filename must start with 'gs://' if it's on GCP.

  • consolidated - consolidate the zarr file?

  • Returns - nwp data