Data Platform Development Setup Guide

1. Purpose

This document provides step‑by‑step instructions for setting up the local development environment for developers working on the Data Platform project.

2. Prerequisites

• Access to project GitHub repositories
• SQO Account and Azure account access (Entra ID)

3. Software Installation Steps

Download Link for all required software:  Software

3.1 Install Python

1. Download Python from above mentioned link - python-3.11.00-amd64.exe
2. Run the installer
3. Enable 'Add Python to PATH' option - On the first installation screen, look for the checkbox at the bottom labeled: “Add Python 3.11 to PATH”. The installer will add Python to your system PATH automatically
4. Verify installation using command: python --version

Install Git

1. Download Git from above mentioned link - Git-2.51.0-64-bit.exe
2. Install using default configuration
3. Verify installation using command: git –version

3.3 Install Visual Studio Code

1. Download VS Code from above mentioned link – VSCodeUserSetup-x64-1.104.0 (1).exe
2. Install using default settings
3. Open VS Code after installation
4. Also install the extension - Azure tools, Python, Bicep, Github Copilot

3.4 Install Azure CLI

1.   Download Azure CLI from Microsoft website

1. Install the package
2. Verify installation using command: az –version

3.5 Install Notepad ++ 

1. Download exe from mentioned link - npp.8.8.5.Installer.x64.exe
2. Install the package

3.6 Install DBeaver or SSMS

1. Download exe from mentioned link - dbeaver-ce-25.1.3-x86_64-setup (1).exe/vs_SSMS.exe
2. Install the package

4. VS Code Development and Repository Setup

4.1 Required VS Code Extensions

• Python
• Pylance
• Azure Tools, Azure Function, Azure CLI Tool 
• Bicep
• Github Action, Github Copilot

4.2 Configure Python Environment

Open Visual Studio Code, below page will appear

4.2.1 Clone the repositories 

1. Clone Git Repository from UI -  
   Video Link - https://drive.google.com/file/d/1L1JXEpYiYjT5wHacRBNaB2yD0UYhiF1b/view?usp=drive_link

       2. Using Terminal – Open Terminal and use below command to clone the repo

git clone https://github.com/SQO-SySight/Ingest-SFDC

4.2.2 Install dependencies 

• Check the repo folder and search for requirement.txt (file that contains details of libraries and other git files required in the project)
• Once found run pip install, so that all required libraries are installed for the project - python -m pip install -r ./requirements.txt

Note – Make sure to give exact path for requirement.txt

5. Code Repository and Branching Strategy

5.1 Pull latest code from dev branch

1. Fetch the latest changes from the remote repository: git fetch

Switch to the dev branch: git checkout develop

1. Make sure your dev branch is up to date: - git pull origin develop
2. Create and switch to your new feature branch: - git checkout -b feature/your-feature-name

Replace your-feature-name with a descriptive name for your feature.

5.2 Develop and test locally

Make necessary changes to the code. This will be explained in the different sessions.

5.3 Commit and push code to GitHub

1. Stage your changes, i.e. add the files you want to commit. 

For all changes: git add

Or, to add specific files: git add path/to/your/file.py

1. Commit your changes. Use a clear and descriptive commit message:

git commit -m "Describe your changes here"

1. Push your changes to GitHub
   git push origin feature/your-feature-name

Note – All of these steps can also be performed using the VS user interface. I will explain the process using the UI in the related video.

5.4 Raise Pull Request

1. Push Your Feature Branch to GitHub
2. Go to the Repository on GitHub
   Open your web browser and navigate to your repository on https://github.com/SQO-SySight/{Project_name}.
3. Switch to Your Feature Branch. Click the “Branch” dropdown and select your feature branch.
4. Click “Compare & pull request”
   GitHub will often show a banner with a green button labeled “Compare & pull request”. Click this button.
• If you don’t see the banner, click the “Pull requests” tab, then “New pull request”.
• Set the base branch (usually develop) and compare it with your feature branch.
5. Fill in the Pull Request Details
• Title: Give your pull request a clear, descriptive title.
• Description: Explain what changes you made and why.
• Add reviewers or assignees if required by your workflow.
6. Create the Pull Request
   Click the “Create pull request” button to submit your PR for review.

5.5 Peer review and approval

5.6 Merge into dev branch

6. Troubleshooting

Python not recognized – ensure PATH is configure

1. Find where Python is installed
   Example:
   C:\Users\<YourUsername>\AppData\Local\Programs\Python\Python3x\
2. Copy the folder path
• Open the folder in File Explorer.
• Click the address bar and copy the path.
3. Open Environment Variables
• Press Win + S and type Environment Variables.
• Click Edit the system environment variables.
• In the new window, click Environment Variables….
4. Edit the Path variable
• Under User variables, find and select Path.
• Click Edit….
5. Add Python paths
• Click New and paste the Python folder path.
• Click New again and add the Scripts folder path (e.g., C:\Users\<YourUsername>\AppData\Local\Programs\Python\Python3x\Scripts\).
6. Save and close
• Click OK on all windows to save.
7. Restart Command Prompt
• Close and reopen your command prompt.
8. Test Python
• Type python --version and press Enter.
• You should see the Python version.

Note – If you are not able to add variables, please connect with Anup Mule for assistance 

Dependency installation errors – check pip version

1. Open Command Prompt or Terminal

Check pip version: pip --version

1. Upgrade pip (if needed): python -m pip install --upgrade pip

Try installing your package again: pip install package-name

(Replace package-name with the name of the package you want to install.)

Authentication issues – login again using Azure CLI

1. Open Command Prompt or Terminal

Login to Azure: az login

1. Authenticate
• A browser window will open.
• Sign in with your Azure account credentials.
2. Wait for Confirmation
• Once logged in, the terminal will show your account details.