...

1. Define variable and Generate meta_run_id
2. Call reference job to load from BW extract from prj-data-dq-selfservice-test.DataOcean_industrialreporting_workspace.VariableCosts and save the output file to bucket cs-ew1-prj-data-industrial-dash-dev-staging/vc/z
3. Load from Bucket to STG and ODS
4. Move data from ODS to DM
5. Update log

...

It is required to access prj-data-dq-selfservice-dev test project

Source

BigQuery

Project = prj-data-dq-selfservice-dev

...

• ODS Table names =

  • ODS_FIL_0000_F001_F_M_fc

• DM Table names = 
  • FACT_variableCosts
• DPL View names =
  • V_FACT_variableCosts

Product

• GCP = prj-data-industrial-dash-[env]
• DataOcean
  • V_FACT_VariableCosts 

Format

columnar format

Sizing

• TG_FIL_0000_0000_F001_F_M_fc  vc         around 7,392 records

...

It is scheduled by plans below on WS_DATA_OCEAN_DOMAIN_MARKETINGIND_DASHBOARD

• TASK_DailyExecution_VariableCosts - Weekly every 30 min starting at 07:00 07am CET ~ 6pm CET

1.4 Timing

• Case 5 minutes from source to ODS (full)

...