Page tree


You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Description

Tools: Talend

Detail job

  • F010_Helix_Incidents


  • F040_Helix_Measurement


  • F050_Helix_Request


  • F030_Helix_ServiceRequestStub


  • F021_Helix_WorkInfo


  • F011_Helix_Worklog


  • F020_Helix_WorkOrder_json

Flow job

Below are the list of plan names which are used to invoke the above mentioned Talend jobs.

  • PL_DT_F020_Helix_WorkOrder_json
  • PL_DT_F011_Helix_Worklog
  • PL_DT_F021_Helix_WorkInfo
  • PL_DT_F030_Helix_ServiceRequestStub
  • PL_DT_F050_Helix_Request
  • PL_DT_F040_Helix_Measurement
  • PL_DT_F010_Helix_Incidents


Access rights

Source

Format

  • JSON

Destination

Location

  • Bucket = cs-ew1-prj-data-dm-dt-[dev]-staging/xxx
  • DataOean GCP = prj-data-dm-dt-[env]
  • STG Table names =
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_HD_incidents
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_Measurement
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_ServiceRequest
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_ServiceRequestStub
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_Workinfo
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_worklog
    • prj-data-dm-dt-[env].STG.STG_HLX_0000_0000_F001_I_H_workorder_json
  • ODS Table names =
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_Incidents
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_Measurement
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_ServiceRequest
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_ServiceRequestStub
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_Workinfo
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_worklog
    • prj-data-dm-dt-dev.ODS.ODS_HLX_0000_F001_I_H_workorder_json


  • DPL View names =  
    •  prj-data-dm-dt-[env].DPL.V_FACT_hlx_work_order
    • prj-data-dm-dt-[env].DPL.V_FACT_hlx_Workinfo
    • prj-data-dm-dt-[env].DPL.V_FACT_hlx_service_request_stub
    • prj-data-dm-dt-[env].DPL.V_FACT_hlx_measurement
    • prj-data-dm-dt-[env].DPL.V_DIM_hlx_incident
    • prj-data-dm-dt-[env].DPL.V_FACT_hlx_ServiceRequest
    • prj-data-dm-dt-[env].DPL.V_DIM_hlx_status

Format

  • columnar format

Sizing


Assessment

How to validate that the generated output is valid: 

Loading

1.1 Incremental Load

1.2 Full load

1.3. Reloading data

1.4 Plan to schedule

1.5 Timing

The average time expected for  loading:

Criticality

High/Medium/Low

Logging