data_sources.nwp

NWP data sources and functions

data_sources.nwp.nwp_data_source

NWP Data Source

NWPDataSource Objects

@dataclass
class NWPDataSource(ZarrDataSource)

NWP Data Source (Numerical Weather Predictions)

Attributes:

  • _data - xr.DataArray of Numerical Weather Predictions, opened by open(). x is left-to-right. y is top-to-bottom (after reversing the y index in open_nwp()).
  • consolidated - Whether or not the Zarr store is consolidated.
  • channels - The NWP forecast parameters to load. If None then don't filter.
  • See - http://cedadocs.ceda.ac.uk/1334/1/uk_model_data_sheet_lores1.pdf All of these params are "instant" (i.e. a snapshot at the target time, not accumulated over some time period). The available params are: cdcb : Height of lowest cloud base > 3 oktas, in meters above surface. lcc : Low-level cloud cover in %. mcc : Medium-level cloud cover in %. hcc : High-level cloud cover in %. sde : Snow depth in meters. hcct : Height of convective cloud top, meters above surface.
  • WARNING - hcct has NaNs where there are no clouds forecast to exist! dswrf : Downward short-wave radiation flux in W/m^2 (irradiance) at surface. dlwrf : Downward long-wave radiation flux in W/m^2 (irradiance) at surface. h : Geometrical height, meters. t : Air temperature at 1 meter above surface in Kelvin. r : Relative humidty in %. dpt : Dew point temperature in Kelvin. vis : Visibility in meters. si10 : Wind speed in meters per second, 10 meters above surface.
  • wdir10 - Wind direction in degrees, 10 meters above surface. prmsl : Pressure reduce to mean sea level in Pascals. prate : Precipitation rate at the surface in kg/m^2/s.

__post_init__

def __post_init__(image_size_pixels_height: int, image_size_pixels_width: int,
                  meters_per_pixel: int)

Post init

Arguments:

  • image_size_pixels_height - number of pixels in image height
  • image_size_pixels_width - number of pixels in image width
  • meters_per_pixel - how many meteres for each pixel

open

def open() -> None

Open NWP data

We don't want to open_nwp() in init. If we did that, then we couldn't copy NWPDataSource instances into separate processes. Instead, call open() after creating separate processes.

get_data_model_for_batch

@staticmethod
def get_data_model_for_batch()

Get the model that is used in the batch

datetime_index

def datetime_index() -> pd.DatetimeIndex

Returns a complete list of all available datetimes

sample_period_minutes

@property
def sample_period_minutes() -> int

Override the default sample minutes

open_nwp

def open_nwp(zarr_path: str, consolidated: bool) -> xr.DataArray

Open The NWP data

Arguments:

  • zarr_path - zarr_path must start with 'gs://' if it's on GCP.

  • consolidated - Is the Zarr metadata consolidated?

  • Returns - NWP data.

data_sources.nwp.nwp_model

Model for output of NWP data

NWP Objects

class NWP(DataSourceOutput)

Class to store NWP data as a xr.Dataset with some validation

model_validation

@classmethod
def model_validation(cls, v)

Check that all values are not NaNs