...
In Mar 2020, the data flow was updated to PCM 2.0
In 2021, BW change the source system SDK_PCM (SQL) to BigQuery(#4749200)
In Feb 2023, Po2 project requires to segregate data of Eco/Sco, therefore, company master data file is modified to add C_AUTHMA(authorization scope) to the openhub OH_PCM_05
In Dec 2023, Po2 spin off project has the new credit controlling area (CCA) = SYEN. DTP of DBFIAR20 need to add CCA = SOLV and SYEN and master data of c_compcde and c_custid are need to add c_scope field in order to use to calculate interco in BQ
Roles & Access
Aligned on FIAR Roles & Access
...
- check that no other run are in progress (a) and then extract data from BW to PCM
- Extract master data in full mode to csv files through openhub
- Load FIAR and CAMS data into specific DSO build for PCM project (DBFIAR20 and DBFIAR21),then extract to csv through openhub
- Load the data from the file via Talend
- treatment on PCM GCP
- Load full with deletion DPFIAR13 from Big Query with new data source DTS_FI_PCM_01 by PC_FI_PCM_10. SQL server was replaced by Big Query
BW to Talend
...
Big Query to BW
DPFIAR13 data loading
...
PC_FI_PCM_V2 load by job RSPROCESS workday at 7am, 1pm , 7pm CET
Schedule by PC at 7 AM and schedule by program on job name BI_PROCESS_TRIGGER_1PM and BI_PROCESS_TRIGGER_7PM
Average performance
| Key Figure | Estimation |
|---|---|
| ~ Average Process Chain Runtime | |
| ~ Average nb of rows loaded per load | |
| ~ Total nb of rows loaded (if full) | |
| ~ Average Runtime for 10k lines |
...
- Copy the new JSON file to the folder defined in gbqadapter_prod.properties on dp agent server (acew1dhcahca1)
- Change the KeyFilePath to the folder of the new JSON file (JSON file provide by DataOps team: dataops@solvay.com )
Remote source in BW configuration is not required to change.
...