Description

The data source is the Pure server (ftp.credit360.com), which Talend will load the file via FTP (more detail from PURE click on this link).  The Talend project EHS_PURE load these data first and keep to GCP project solvay-ind-conso-[env] on the dataset ehs_pure_[env]_mig, which is existing and not part of Operational Dashboard project. 

Source FTP server = "ftp.credit360.com"  / User = "solvay" by using Private key file and it keep in remote engine GCP at this folder \DATA\DEV\EHS\Pure\InOut\pure_sftp_ssh_key (control by context variable l_CNX_EHS_PURE_SFTP_private_key )

Talend Project = EHS_PURE

Talend jobs  = F004_Connect_to_SFTP  + F005_Data_Prep, which are not part of this project.

Load following file  to /DATA/DEV/EHS/Pure/Tmp

GCS folder = ehs_pure_dev_mig

Talend Plan = PL_EHS_PURE_HOURL_New run every hour in PROD


Then, Operation Dashboard project using these data to prj-data-industrial-dash-[dev] project by create views

GCP dataset = solvay-ind-conso-dev.DS_prj_data_industrial_dash

V_core_hd_monthly  → prj-data-industrial-dash-[env].DataOcean_solvay_conso.V_OP_HSE_Transversal_Monthly → DPL.V_OP_HSE_Transversal_Monthly

v_core_hd_quarterly →  prj-data-industrial-dash-[env].DataOcean_solvay_conso.V_OP_HSE_Transversal_Quarterly → DPL.V_OP_HSE_Transversal_Quarterly

V_os_data →  prj-data-industrial-dash-[env].DataOcean_solvay_conso.V_OP_HSE_OccupationalSafety_IncidentsList → DPL.V_OP_HSE_OccupationalSafety_IncidentsList

V_ts_data →  prj-data-industrial-dash-[env].DataOcean_solvay_conso.V_OP_HSE_TransportSafety_IncidentsList → DPL.V_OP_HSE_TransportSafety_IncidentsList

V_ps_data →  prj-data-industrial-dash-[env].DataOcean_solvay_conso.V_OP_HSE_ProcessSafety_IncidentsList → DPL.V_OP_HSE_ProcessSafety_IncidentsList