This page presents the data development documentation for Coatings data for the sites of ...
The data ingestion phase for Coatings follows the standard approach describe on ALB Data Dev Architecture - General except for...
The list of the spreadsheets extracted from JSON files coming from ELN can be found in the Data Mapping of the next section.
| Document Name | Link |
|---|---|
| Data Model for Coatings | Data Model Link |
| New Data Model for Coatings | Data Model Link |
The instrument files are, most of them, manually added to a folder in lab servers.
The spreadsheet Where to find instrument files? has the full list of instrument folders and location:
\\frph2-labpc-backup\labo\W-513825\DATAS\Experiment Results
\\frph2-labpc-backup\LABO\W-510315\Malvern Instruments\Zetasizer\Measurement Data\Coatings\Export Data - Coatings
\\USBRIFS51.ist-priv.rhodia.com\data$\USBRI30293\Malvern Instruments\Zetasizer\Export Data\Coatings
\\USBRIFS51.ist-priv.rhodia.com\data$\USBRI28321\Attension Instrument\Surface Tension\Export Data\Coatings
The jobs F001, F000 and F100 are responsables for orchestrating the Data Ingestion:
The data preparation phase for Coatings follows the standard approach describe on ALB Data Dev Architecture - General.
The spreadsheet below presents all data transformation between the raw files (extracted files) and a BigQuery delta table. Some files are unstructured and semi-structured. This steps aims to structure the files in the target table format and checking if the column's type (schema) are conformed expected.
The data presentation phase for Coatings follows the standard approach describe on ALB Data Dev Architecture - General.
No data mapping available, as there are no transformations for now. This must be created if there are requirements for that.
No need of jobs as there are no steps to load DW/DM. A priori, all data is presented as views.
The data model presents the tables/views presented on DW/DM dataset and the relation between them.
{Add model}
All the jobs are run in sequence under the follow job and project name on TAC/Talend Cloud:
| Project | Job/Flow |
|---|---|
| RnI_ACN_Coatings | F001_RnI_ACN_Coatings_ELN_EP_OrchFlow |
| RnI_ACN_Coatings | F000_RnI_ACN_Coatings_ELN_Paint_Formulation_Orch_Flow |
| RnI_ACN_Coatings | F100_RnI_ACN_Coatings_Instruments_Orch_Flow |
For scheduling details check the Operational documentation.
Tableau workbook documentation : Technical documentation