View Source

This job allows you to upload all the files of a Google Drive folder into a Google Cloud Storage bucket

PREREQUISTES

The GCP service account must have at least read access on the Google Drive folder.

JOB DESCRIPTION

1 - The job tries to connect to the desired Google Drive folder, if the connection fails, the job will stop and raise an error

2 - The job creates a subfolder inside the output folder that you specified. This folder will be used to save locally the Google Drive files during the processing.

3 - The job will download all the files locally.

4 - The files are uploaded into the desired Google Cloud Storage bucket.

5 - The subfolder (with all its files) is deleted

HOW TO USE THIS JOB?

The job is fully dynamic. So you can :

Copy the job from the DATA_OCEAN project and paste it into your project
Drag and drop the copied job into your Talend flow
Provide the necessary parameters to make the job work.

CONTEXT VARIABLE	DESCRIPTION
l_PATHDIR_SERVICE_ACCOUNT_KEY	The full path where the GCP service account is saved
l_VAR_BUCKET_NAME	The name of the GCS bucket that will store the files
l_VAR_BUCKET_FOLDER	The bucket folder where the file will be stored