How to : start a new development

Create a dedicated feature branch

• From the dev project (SPOT_CS), fetch and make sure there is nothing to push/pull.

• Create a new branch, chose the "duplicate project" option and name it based on its Jira ticket.

• Store it in the "Pricing" folder and select "full duplication" if you want to retrieve all the intermediate datasets.

• Push your new branch for the first time so it appears on Gitlab. It is recommended to create the merge request in "Draft" in Gitlab directly.

Develop on the new branch

You can then develop the functionality on the new branch, be careful that any csv stored in the folders of the flow will not be merged back on the dev branch at the end of the development.

During the development, it is advised to frequently rebase the dev to stay updated with the latest changes, especially if other branches have been merged on dev in the meantime.

Rebase your branch 

IMPORTANT: NEVER REBASE DEV OR MASTER !!! This only applies to feature branches or hotfixes that we want to merge to dev.

Because dev branch might have changed since you created your branch, you should update your branch as often as possible to include those changes and stay as close as possible to dev.

There are 2 main ways: merge and rebase, with pros and cons for each. In our workflow, we will favor rebasing.

One of the drawbacks of rebasing is to deal with conflicts multiple times with different commits. In order to avoid that we can first squash some commits by using an interactive rebase without changing the starting point.

Rebasing

# Clean state
git fetch
git checkout my_branch
git pull --rebase

# Rebase in-place with squash: 
# first find forking point between your branch and dev
git merge-base HEAD origin/dev
# this will return a commit number. let's call it base_commit
git rebase -i base_commit
# this opens editor with the list of all your branch commits that are not in dev
# keep the first two as pick
# then squash the others as you want, for example squash all the following together by picking the 3rd commit and squash for all following
# save and close, a new commit is made: rewrite properly its description with something meaningful, save and close

# Your branch now has only 3 commits: first 2 automatically done by DSS, the 3rd has all your branch changes. check status
git status

# Rebase to get dev updates
git fetch
git rebase origin/dev
# deal with potential conflicts up to 3 times only for each commit

# Verify your branch state and synchronize it to remote
git push --force-with-lease

Merge back to dev

Merging feature branch to dev:

1. Make sure the full scenario has run successfully on your branch
2. If not already done, create a merge request on gitlab : assign to yourself, and add a reviewer
3. Reviewer does the review. Relaunch a scenario if there are any changes.

4. Interactive rebase to remove first two commits automatically generated by Dataiku when creating the branch : we don't want to keep them because they would overwrite some parameters on dev (project title, key, etc.).

   git fetch
   git checkout my_branch
   git pull --rebase
   git rebase -i origin/dev
   # editor is opened with the list of commits on the branch that are not in origin/master
   # remove unwanted lines for commits you don't want to keep, save and close
   # (remark: interactive rebase can also be used to squash together multiple consecutive commits)
   #if any conflicts rebase is paused, make sure to fix them: fix file, then 
   git add my_file_with_conflicts
   # make sure currently rebased commit doesn't contain anymore conflicts
   git status 
   # should be all green
   # continue rebase
   git rebase --continue
   #repeat above if conflicts until end of rebase
   #make sure branch is where you want it and push the changed branch
   git status
   git push --force-with-lease

5. Merge on target.
   
   

   git checkout dev
   git pull --rebase
   git merge --no-ff my_branch
   git push

6. Update the dev project (SPOT_CS) by doing a fetch and pull. Check that you have the latest changes including the merge commit.
   
7. Verify that the merge request is closed in gitlab.

8. Delete the Dataiku branch project