...
Architecture - High Level Design (HLD)
Link.
Architecture - Low Level Design (LLD)
Link.
Architecture Data Flow
| Google Drive Live Link | ||
|---|---|---|
|
|
|
|
DataPrep Flow
Schema showing the different STEPS of the application flow - with the data involved at each step
Steps descriptions on Big Query
Describe the data and process involved at each step
SAP PF1
Description
This project has the data from SAP on QM, material document and Vendor master data tables loading into Industrial data ocean for all data in PF1 basing on the create and modification date. Then, the view on top will filter only Soda Ash data.
Note: Soda ash data has only in PF1, therefore, this project will load only from PF1. (It is not include WP1)
Tools
Talend
Access rights
To access to SAP, the Talend user RFC_TAL_PF1 is required
Source
Location
...
Table
...
Name of Table
...
Create Date
...
Modify Date
...
MCH1
...
Batches (if Batch Management Cross-Plant)
...
ERSDA
...
LAEDA
...
MCHA
...
Batches
...
ERSDA
...
LAEDA
...
QALS
...
Inspection lot record (header)
...
ERSTELDAT
...
AENDERDAT
...
QAMR
...
Characteristic results during inspection processing
...
ERSTELLDAT
...
AENDERDAT
...
QAMV
...
Characteristic specifications for inspection processing
(Average / Summary result)
...
ERSTELLDAT
...
AENDERDAT
...
QAVE
...
Inspection processing: Usage decision
...
VDATUM
...
VAEDATUM
...
MSEG
...
Document Segment: Material
...
CPUDT_MKPF
...
/BEV2/ED_AEDAT
...
LFA1
...
Vendor Master (General Section)
Format
Destination
Location
DataOean GCP = prj-data-dm-industrial-dev
Product GCP = prj-data-sad-ebatch-ppd
To save the data into GCP Industrial Data Ocean, service account is required
Format
Same as the source
Sizing
Expected data volume for :
- full process
- incremental process
Assessment
How to validate that the generated output is valid
Scheduling
Is there an automatic schedule ? At what frequency ? What is the trigger ?
Timing
The average time expected for :
- full process
- incremental process
Criticality
High / Medium / Low
Logging
Logging location
DataSource 2
...
Children Display