...
| FlowName | Tasks | Description | Source_table/query | Target_Table | Complex transformation, if Yes,then logic |
| F001_QVBW_QRY_MVSDSO57_0002_to_ODS_MAIN | F001_QVBW_QRY_MVSDSO57_0002_to_ODS | Delivery query to get number of delivery | Xtract job TALEND_DEV_QVBW_QRY_MVSDSO57_0002 | prj-data-dm-marketing-dev. ODS_BWH_0000_F001_F_D_qvbw_qry_mvsdso57_0002 | |
| F020_SFC_case_customer_complaint_to_ODS | J020_SFC_case_customer_complaint_to_ODS | number of complaints | Salesforce on case table | prj-data-dm-marketing-dev. ODS_SFC_0000_F001_F_M_case | |
| F010_FACT_customer_complaints | J010_FACT_customer_complaints | Fact table of customer complaint | ODS_BWH_0000_F001_F_D_qvbw_qry_mvsdso57_0002 ODS_SFC_0000_F001_F_M_case | prj-data-industrial-dash-dev. FACT_customer_complaints | count distinct of delivery number and case id. Also calculate rolling 12 months by this script |
Tools: Talend
From source to ODS
...
- Connect to Salesforce (SFC)
- Query SFC on the case table ( select only required fields FROM Case
WHERE CreatedDate >= 2020-01-01T00:00:00Z
and PO2_CASE_Organization__c like '%ECO%' ) - Write output to file in local PC
- Put the file to bucket cs-ew1-prj-data-dm-marketing-dev-staging/Case/
- Delete the output file in local PC
- Write log on the main flow job
...
- Check execution time format
- Select data from DataOcean.V_sfc_case sfc and DataOcean.V_BWH_no_of_delivery_orders (script)
- Create a template internal table (period x gbu x plant x serivity)
calendar to have period in format YYYY-MM back to n month, which control by variable l_VAR_IND_DASH_customer_complaint_nb_month_to_DM
gbu_mapping to have text and key, which get from variable l_VAR_IND_DASH_customer_complaint_gbu_mapping = internal table for gbu, add more in the variable if we have new gbu
severity_mapping to have list of severity value = not specified, Low, High, Major, Medium, Anomaly, Critical and Standard which get from variable l_VAR_IND_DASH_customer_complaint_severity_mapping
site_cal to have all plant and gbu from V_sfc_case sfc cross join with severity_mapping and calendar to have template table that have all period, gbu, plant and serivity
- Create an aggregate internal table of customer complaint (cc_aggr ) from V_sfc_case sfc
- Create an aggregate internal table of delivery (del_aggr ) from V_BWH_no_of_delivery_orders
- Select the final data by calculate rolling 12 months
- Create a template internal table (period x gbu x plant x serivity)
- Delete all data in DM.FACT_customer_complaints
- Load data from point 2 to FACT_customer_complaints
- In case of error, email inform DataOps (l_VAR_IND_DASH_email)
- Update log
...