Description

Wave is a external tool that calculate cost of saving on each project. Talend can get the data via API of this server and the output of the API is excel file. 

It is required to create a connection and setup the access time to allow Talend read the data. If Talend access out side of the allow time, it will not get the file.

The password is also required to reset every 6 months since the password will be expired

Wave Connection

How to create connection

1. Login https://solvay.mckinseywave.com/login/auth

2. Go to Data Connection

3. Click on ADD DATA CONNECTION

4. Enter the name of the connection and data/time that allow to access.

If not set schedule, it is required to login on the web in order to allow the access.

The time is GMT+1 time zone

5. It will generate URL endpoints, username and password.   Click on generate password and save it. 

Please note that each password will be expired every 6 months and the interface is required to access at least every 17 days, otherwise, the connection will be deactivated. 

How to change the access time

1. Login https://solvay.mckinseywave.com/login/auth

2. Go to Data Connections same as create connection

3. Right click on the connection that you want to change and select Update

4. Change date/time and click UPDATE

How to reset password

1. Login https://solvay.mckinseywave.com/login/auth

2. Go to Data Connections same as create connection

3. Right click on the connection that you want to change and select Connect

4. Click on reset password




Talend

The Talend project is on DATA_OCEAN_DOMAIN_INDUSTRIAL

The project will required 4 sheets from the file output from the API. Then, in order to improve the performance of the loading and the size of data, the impactStar and SustainabilityImpactStar sheet are normalize into 2 tables

Detail job



  1. Connect to the source system API by reading context from flow job
  2. Setup loop to get the data
    1. tSetGlobalVar : to set the maximum number of records to read each time and set the variable nb to check when to exit the loop (start with 0)
    2. tLoop : setup the condition to exit the loop when variable nb < 0
    3. tJava: setup the offset of records in order to get new records of each loop
  3. To get data from the source by using start row number from "nb" and max row number from "limit".  It read schema from the source(meta data)
  4. Generate output file and save to DATA\DEV\DATA_OCEAN_DOMAIN_DT\Tmp
  5. Update the offset number "nb" = "nb" + "limit"
  6. Update "nb" = -1 when ((Integer)globalMap.get("tReplace_1_NB_LINE"))<= 0  in order to exit the loop
  7. Upload the files all the folder( cs-ew1-prj-data-dm-dt-[dev]-staging)
  8. Delete all the files in the folder (point number 5)


The Talend process extracts all cases from the ITSM source, and as part of data anonymization, we have anonymized the "Human-Resources" case data.

Below is a list of columns that are anonymized during ingestion. Users will not be able to view the original data, as it will appear as "*** Anonymized info ***." By anonymizing at the ingestion stage, the original data is not stored in Cloud Storage or BigQuery tables.


Flow job


Access rights

It is required to get access from WAVE admin ednamaria.coan@solvay.com

Source

Format

Destination

Location

Format