Procedures
Here is a suggested operation book template:
https://docs.google.com/document/d/1xf3wWoQBgHKQtefaWi04Hx545rPB8HzQ
Procedure guide on how to operate the application
DataPrep Flow
Start
Incremental process
In order to run the incremental process you should execute the Talend flow F001_Daily_flow which is scheduled daily
Termination
Check in the TMC that the job is completed.
Restart
Is the process the same ?
Also for restarting after an error ?
The restart process consists in re-running the incremental process. As the process is incremental (some tables are always loaded in full) even after an error you should simply run the process.
Pause
Procedure
How?
No pause
Alert contacts
Who should be alerted ? Antoine REY (ELN Application Owner / ELN dashboard Product Owner)
Stop
Procedure
How ?
If you need to stop the daily flow you can remove the trigger from the Talend TMC.
Alert contacts
Who should be alerted ? Antoine REY (ELN Application Owner / ELN dashboard Product Owner)
Reset
How ?
No need.
Scheduling
Trigger
What is the start trigger ? Event based ? Time based ?
Are there differences between DataPrep and DataApp ?
The Talend flow is scheduled every day at 6am CET.
Expected results
For each brick, what is the expected output ?
Big Query tables of the project prj-dash-eln-prod will be updated.
Intervention
When is the time frame to intervene ? (when downtime is acceptable or scheduled)
The project is not critical so you can intervene anytime.
Monitoring
Runtime
Where and how can we see the application status (Stopped, waiting, running, etc) ?
Look at the TMC to check the job status.
Run history
Where are the run actions historic ?
What form does it take ? Logs ?
Resources
Memory / Disk / CPU used by application
Additional metrics
According to operational requirements, detail application metrics (Processed Volume, Process duration, ...)
Logging
Where to find each step logs ?
Error handling
As a general guideline, application should stop as soon as possible.
Alerts
- Contacts
- Meaningful message (timestamps, description, criticality)
Specificity
Detail procedure for specific error cases