config

Configuration of the dataset

config.model

Configuration model for the dataset

General Objects

class General(BaseModel)

General pydantic model

Git Objects

class Git(BaseModel)

Git model

InputData Objects

class InputData(BaseModel)

Input data model

All paths must include the protocol prefix. For local files, it's sufficient to just start with a '/'. For aws, start with 's3://', for gcp start with 'gs://'.

OutputData Objects

class OutputData(BaseModel)

Output data model

Process Objects

class Process(BaseModel)

Pydantic model of how the data is processed

seq_length_30_minutes

@property
def seq_length_30_minutes()

How many steps are there in 30 minute datasets

seq_length_5_minutes

@property
def seq_length_5_minutes()

How many steps are there in 5 minute datasets

history_minutes_divide_by_30

@validator("history_minutes")
def history_minutes_divide_by_30(cls, v)

Validate 'history_minutes'

forecast_minutes_divide_by_30

@validator("forecast_minutes")
def forecast_minutes_divide_by_30(cls, v)

Validate 'forecast_minutes'

Configuration Objects

class Configuration(BaseModel)

Configuration model for the dataset

set_base_path

def set_base_path(base_path: str)

Append base_path to all paths. Mostly used for testing.

set_git_commit

def set_git_commit(configuration: Configuration)

Set the git information in the configuration file

Arguments:

• configuration - configuration object

• Returns - configuration object with git information

config.save

Save functions for the configuration model

save_yaml_configuration

def save_yaml_configuration(configuration: Configuration, filename: Optional[Union[str, Pathy]] = None)

Save a local yaml file which has the a configuration in it.

If filename is None then saves to configuration.output_data.filepath / configuration.yaml.

Will save to GCP, AWS, or local, depending on the protocol suffix of filepath.

config.load

Loading configuration functions

load_yaml_configuration

def load_yaml_configuration(filename: Union[str, Pathy]) -> Configuration

Load a yaml file which has a configuration in it

Arguments:

• filename - the file name that you want to load. Will load from local, AWS, or GCP depending on the protocol suffix (e.g. 's3://bucket/config.yaml').

Returns:pydantic class

load_configuration_from_gcs

def load_configuration_from_gcs(gcp_dir: str, bucket: str = "solar-pv-nowcasting-data", filename: str = "configuration.yaml") -> Configuration

Load configuration from gcs

gcp_dir: the directory where the configruation is saved bucket: the gcs bucket to load from filename: the filename that will be loaded

Returns: configuration class