This page presents the data development documentation for Materials data for the sites of Alpharetta and Bollate.  

Summary

Description

Data Ingestion

The data ingestion phase for Materials follows the standard approach describe on ALB Data Dev Architecture - General except for...

Data Sources

Labware

we retrieve data from an oracle database 

Related documents

Instruments

The instrument files are, most of them, manually added to a Google Driver's folder or a folder in lab servers. 

The spreadsheet Where to find instrument files? has the full list of instrument folders and location:

Data Mapping

No data mapping needed as the data is copied exactly as the source.

Talend Jobs

The jobs Instruments_Flow and Labware_Flow are responsables for orchestrating the Data Ingestion:

Data Preparation or Parsing

The data preparation phase for Materials follows the standard approach describe on ALB Data Dev Architecture - General.

Data Integration or Computing

The data integration phase for Materials follows the standard approach describe on ALB Data Dev Architecture - General.

Data Mapping

The spreadsheet below presents all data transformations between the tables on Staging and ODS. This steps aims to add some calculations and intelligence to the data. Not all tables will need passing through this step.

https://drive.google.com/file/d/1hi1YCa7OZ_sMfMiNAGXAk3XRnCtm4eBJ/view

Data Presentation

The data presentation phase for Materials follows the standard approach describe on ALB Data Dev Architecture - General.

Talend Jobs

No need of jobs as there are no steps to load DW/DM. A priori, all data is presented as views.

Orchestrating Jobs

All the jobs are run in sequence under the follow job and project name on TAC/Talend Cloud:

For scheduling details check the Operational documentation.

Folders

Tables (Staging)

Data Visualization

Tableau workbook documentation : Technical documentation Materials