Talend Context variable standard and naming-convention

I. Purposes

II. Reminder : What is a « context »

III. Centralization

IV. Context

V. Global Context Synthesis

VI. Local Context Synthesis

VII. Project Folder Example

VIII. Other Context vs globalMap

I. Purposes

—Centralize parameters used in Talend projects
—Rationalize / standardize contexts through naming convention and values used
—Distinct three environments : Developement (DEV), Testing/Quality/UAT (UAT) and Production (PROD)
—Initiate Talend development standard which should be completed along with the projects done

II. Reminder : what is a « Context »

—A context in Talend could be assimilate to a parameter which could be used in read/write by any component used in a Talend job
—In programming, it could be refer to a global or a constant variable
—In general, context are defined at Talend project level
—The context value could be define :
—Within Talend project (« hard coded »)
—External files or database through the option « implicit context load »
—In TAC, at deployed job level (job conductor)

According to the latest point and in order to standardize the usage, it is recommended to use the implicit context load from an external data source. It will be easier for Production team to change context value without using any Talend tools. The external data base solution will be kept (only a restricted population should be able to change value in the database).

III. Centralization

A. Target centralization storage

B. Centralization TALEND_PARAM

• All context values will be stored in Talend_param table
• Talend_param table will be available on each server. It will contain all data by environment
• The table which could be used by a Production team will be stored on Talend_exploit database on WDCSLVBP31\SLV1_DBPRD server and should only contain prod values

IV. Context

A. Context - Naming convention

• In general we will distinguish two kind of context, global context vs talend project local context
  1. Global context example
     • Source / target database connexions
     • Source / target systems and/or applications connexions (LDAP, WebService, API, etc.)
     • SMTP server
     • Workspace/target root folders used by Talend projects
  2. Local context example
     • Variable dedicated to a specific project (switch of year date, flags and so on)
     • Dedicated workspace folders (storage, rejects, logs and so on)
     • Dedicated Filenames (storage, rejects, logs and so on)
     • Dedicated e-mail distribution list

B. Global context – Naming convention

• Global context name should start by « g_* »
• The name should then follow by one of the following type :

a. Global context – Naming convention « CNX »

SYNOPSIS : « g_CNX_<CONNECTIONTYPE>_<Application>_Key »

• Global context name should start by « g_CNX_* »
• Followed by one of the type below :

• Followed by the System / application name (LABWARE, PICASSO etc.)
• At the end by the key name (Login, Server, etc.)

b. Global context – Naming convention « DIR »

SYNOPSIS : «g_DIR_<PROJECT>»

• Global context name should start by « g_DIR_* »
• Following by the Talend project name which produced the files

All Talend project directory will be stored as following :

« DiskLetter:/DATA/ » Which located on Talend servers, Depending the talend server we might find the folders DEV, UAT or PROD

* : the slash at the end of the value is important since it simplified the concatenation with a filename in Talend

Suggestion : If possible set a network shared folder instead of storing the data directly on Talend servers

C. Local context – Naming-convention

• Local context name should start by « l_* »
• Followed by one of the context type:

a. Local Context – Naming-convention « VAR »

SYNOPSIS : « l_VAR_<PROJECT>_<Key> »

uLocal variable/parameter context type should start by « l_VAR_* »
uFollowed by the Talend project name
uEnded by an explicit description of the variable usage