The run_jobs log contains the information about a specific job run, i.e. the overall status of a flow from source extraction till data mart.
The job will get context variables as input and use them to create a new entry in the runs_jobs table.
You can simply drag and drop the job at the end of your flow and pass the following contexts:
- l_LOCAL_VAR_JOBID : the id of the job (it corresponds to the system id in Talend)
- l_LOCAL_PATHDIR_GCS_CSV_TO_STAGING_SERVICE_ACCOUNT_PATH: the path where the service account will be stored
- l_LOCAL_VAR_GCS_CSV_TO_STAGING_PROJECTID : the Google Cloud Platform project id
- l_LOCAL_VAR_GCS_TO_STAGING_DATASET : the dataset containing the runs_jobs table ("Staging")
- l_LOCAL_VAR_JOB_START_TIMESTAMP : the timestamp when the job started (this variable must be a timestamp variable)
- l_LOCAL_VAR_JOB_STATUS : the status of the job as it will displayed in the status column
- l_LOCAL_VAR_ERROR_MESSAGE : the error message (if any)
- l_LOCAL_VAR_GCS_CSV_TO_STAGING_RUNS_JOBS_LOG_TABLE : the name of the runs_jobs table
An example of the variables is as follows:
