Google Cloud Platform (GCP) is the platform that allows us to store the data.
For the PCF project, there are 4 GCP environments:
- DEV (prj-carbon-footprint-dev): this environment is used by the developers to develop new features. The first tests are performed by the Product Owner (PO) on this environment.
- UAT (prj-carbon-footprint-test): this environment is used by the key users, via Tableau, to perform further tests to ensure that the developments are correct.
- PRE-PROD (prj-carbon-footprint-pre-prod): this environment is used by the DataOps team to ensure that developments pushed into production are not erroneous.
- PROD (prj-carbon-footprint-prod): this is the production environment and therefore allows end users to access the data via Tableau.
Each of the environments follows the same schema and is therefore composed of the following datasets:
- Staging: Data layer identical to the source
- ODS: Data layer with slight processing on the data without changing its structure
- DW: Layer of data where the data is processed in order to model it as the business wishes. This is the layer that is linked to Tableau.
- CLC: Data layer where all the tables necessary for the various calculations are stored, but which will not be used in Tableau.
- Temp: Data layer where temporary tables appear, used in particular when updating data in the Staging layer.
- Delta: Data layer allowing the Staging layer to be updated incrementally.