4.1 - Source Descriptions
4.1.1 FTP Server
METEOLOGICA (FTP server):
CSV files containing data about the current and forecast energy prices by country - Spain, Italy, Germany, France.
Folders:
- /20160301/Europe/Spain/PowerPrice/Forecast/Meteologica/Total/Hourly
- /20160301/Europe/Spain/PowerPrice/Forecast/ECMWF-ENS/Total/Hourly
- /20160301/Europe/Spain/PowerPrice/Observation/Total/Hourly
- /20160301/Europe/Italy/PowerPrice/Forecast/Meteologica/CNOR/Total/Hourly
- /20160301/Europe/France/PowerPrice/Forecast/Meteologica/Total/Hourly
- /20160301/Europe/Germany/PowerPrice/Forecast/Meteologica/Total/Hourly
Folder structure (Example, /20160301/Europe/France/PowerPrice/Forecast/Meteologica/Total/Hourly for 2023.08.26):
- France_PowerPrice_Forecast_Meteologica_post-ECMWF-ENS_Total_Hourly_202308260000.csv (data from 10 am)
- France_PowerPrice_Forecast_Meteologica_Total_Hourly_202308261333.csv (data from 2 pm)
- France_PowerPrice_Forecast_Meteologica_Total_Hourly_202308261533.csv (data from 4 pm)
- France_PowerPrice_Forecast_Meteologica_post-ECMWF-ENS_Total_Hourly_202308261200.csv (data from 10 pm)
Files with the same date contain the same data, provided 4 times a day.
Structure of the files (example France_PowerPrice_Forecast_Meteologica_post-ECMWF-ENS_Total_Hourly_202308260000.csv):
All four files are consolidated into one file with the additional column datetime_forecasted.
The STG/ODS table format is given below (table structures will be provided as a part of the Talend integration documentation):
| Column name | Data type | Mode | Description |
|---|---|---|---|
datetime_from | TIMESTAMP | NULLABLE | Date from |
utc_offset_from | STRING | NULLABLE | UTC offset |
datetime_to | TIMESTAMP | NULLABLE | Date to |
utc_offset_to | STRING | NULLABLE | UTC offset |
price | FLOAT | NULLABLE | Price |
datetime_forecasted | TIMESTAMP | NULLABLE | Date and time forecasted - generated column |
meta_run_id | STRING | NULLABLE | Meta column |
meta_md5_hsh | STRING | NULLABLE | Meta column |
meta_file_name | STRING | NULLABLE | Meta column |
meta_file_path | STRING | NULLABLE | Meta column |
meta_execution_id | STRING | NULLABLE | Meta column |
meta_bucket_load_date | STRING | NULLABLE | Meta column |
meta_business_date | TIMESTAMP | NULLABLE | Meta column |
meta_stg_insert_date | TIMESTAMP | NULLABLE | Meta column |
meta_source_system | STRING | NULLABLE | Meta column |
The different format is presented only in the SPAIN_ENS data source where there are 50 ens column with the presented prices, and the top, bottom and average price columns.
4.1.2 Database
VENDOHM DATABASE:
Gas & electricity (weekly, monthly, yearly values by location)
- Tables with prices with the same structure (mapping is provided in the Excel file "List of tables_filters for Vendohm database of Robustify")
List of tables_filters for Vendohm database of Robustify - Meteo table curve_n_ometeo_870 (mapping with the definitions and curve IDs provided in the Excel file "Vendohm_meteo_mapping" -this should be provided as a Metadata table)
Vendohm_meteo_mapping
- Tables with prices with the same structure (mapping is provided in the Excel file "List of tables_filters for Vendohm database of Robustify")
Example of the structure (Vendohm data):
IRM DATABASE:
Information about energy deals by market, site, product, etc., with prices.
Example of the structure (IRM data):
4.1.3 Google Sheets
GSHEETS:
Example of the structure:
Responsible & contact points:
- Alessandro Mainardi - Project Owner
- Simon Bourguignon - Delivery Manager
- Alba Carrero/ Gaetan Frenoy - Product Owner
- Rui Ferraz - Project Manager