Table of

...

Contents 

...

Testing Strategy Definition

• Overview of Testing Strategies
• Importance of a Defined Strategy
• Components of a Testing Strategy

...

Defining the Testing Process

1. Define testing type per environment Monthly Release

INT/QA

...

Functional Testing ::

...

Definition: Test the scenarios Identified under the Stories

Includes:

1. Define testing type per environment Weekly Release

2. Define Test scripts repository (JIRA folder) for each testing type

3. Define how to Manage test Plans and Executions and Progress Reporting

4. Define Testing process including Bug creation and fix

5. Define Guidelines on how to report a Bug

Define testing type per environment Monthly Release

INT/QA

Functional Testing :: Test the scenarios Identified under the Stories

Happy path scenarios :: This confirms the system's ability to successfully complete transactions and workflows without encountering errors, thereby validating the basic functionality and reliability of the integrated system

(new) Alternative Flows :: Alternative flows testing in an integration environment involves validating the system's behaviour when users take non-standard paths or encounter less common scenarios. This type of testing ensures that the system can handle variations in user actions or data inputs that deviate from the primary, expected workflow (the happy path)

(new) Negative (-Ve) Testing :: Identify Scenarios related to -Ve testing. Ex ::Boundary Values , Invalid Formats , Empty or Null Values testing , Calling API’s with incorrect endpoints , Concurrency issues etc…

Exploratory Testing : Conduct to identify edge cases and unexpected behaviours. (no scripts necessary, only applicable if dedicated testing team - recommended)

UAT Readiness :: (to be included into UAT Internal Readiness)Stories moving into UAT are bug free and all tests under the stories are passed. All

...

bugs raised in the

...

user stories have been resolved. If any bugs remain unfixed, they should be discussed with the Product Owner (PO) and added to the project backlog in JIRA.

UAT

Functional Testing :: Same as above

NRT /Regression Testing (Functional Happy Path) ::

...

  Continuous execution of regression tests to ensure that new changes do not introduce new defects.

Regression test suite

...

's should be updated regularly to cover new features and fixed defects. (These tests scripts should be highly sophisticated such that

...

these can be automated in future) .

Happy Path test cases from all previous functional testing can be incorporated into the Regression test suite. This ensures that critical functionality is consistently validated and that any changes or updates do not disrupt existing features. Integrating these test cases into the regression suite enhances test coverage and reliability, providing greater confidence in the stability of the system.

...

Prod Readiness :

...

: Ensures that system is fully prepared for deployment to the live environment, with all critical functionalities tested, performance optimized , All bugs are tested and passed, if any bugs are not fixed , then they are created as Project backlog’s in Jira.

E2E : (Optional)

Definition: E2E testing simulates real-world user scenarios to validate that the entire application, including its interfaces and interactions, performs seamlessly. This testing is crucial for identifying integration issues, ensuring data integrity, and providing confidence in the system's readiness for deployment.

Frequency: E2E tests should be less frequent due to their complexity and extensive coverage, while regression tests should run more frequently to validate continuous code changes.

Performance Testing <Topic to be discussed with Hugo>

Prod Readiness :: Ensures that system is fully prepared for deployment to the live environment, with all critical functionalities tested, performance optimized , All bugs are tested and passed, if any bugs are not fixed , then they are created as Project backlog’s in Jira.

PROD

Sanity Testing ::

...

Testers should use specific test cases designed to validate essential operations without

...

diving into extensive testing. The goal is to quickly confirm system stability and functionality, minimizing user disruption while ensuring that the production environment remains operational and reliable.

Dedicated Test Accounts: Create dedicated test accounts that are isolated from actual user data. These accounts should mimic various user roles and permissions.

Data Tagging: Tag test data set distinctly so that it can be easily identified and segregated from real user data. This helps in monitoring and clean-up.

Delete created test records: testers should have the permission to delete data, directly using delete Button (system might need to be reviewed to allow or a person appointed to do this task)

Define testing type per environment Weekly Release

Assumption: weekly release will not include changes on automations (Flows, triggers), if the case, then a more robust testing should be considered: Happy Path, Alternative Flows, Negative (-Ve).

UAT

Functional Testing ::

→ Test the scenarios Identified under the Stories

→ Happy path scenarios < This confirms the system's ability to successfully complete transactions and workflows without encountering errors, thereby validating the basic functionality and reliability of the integrated system>

PROD Readiness :: Stories moving into UAT are bug free and all tests under the stories are passed. All the bugs raised in the stories are fixed , if any bugs are not fixed , then they are created as Project backlog’s in Jira.

...

• Key Steps in the Testing Process
• Roles and Responsibilities
• Entry and Exit Criteria

...

Managing Test Plans

• Overview of Test Plans
• How to Create and Update a Test Plan
• Managing Changes to Test Plans

...

Test Scripts Repository

• Defining a Test Scripts Repository
• Organizing Test Scripts
• Version Control for Test Scripts

...

Define Test scripts repository (JIRA folder) for each testing type

Test X-Ray Test Repository :: This will be our test case Library with following limitation ie test repo is a project level feature, so each test repository can only contain test from its projects (we cannot mix tests from multiple projects into a single repository and organize them together)

All test cases which we create end up in Test Repo by default.

X-Ray Test Sets :: Is a group of tests , we can consider this to be the equivalent of a single folder inside the test repository.

...

• Folders - will be used to structured the repository by Application >> Process >> Test Type - this folders will contain all test scripts that will be created by BAs, QA Lead will maintain the structure, and Test Scripts cannot be cloned.

Image RemovedImage Added

• Test Sets - will be used to include all scripts needed for a certain release, this will be managed by QA Lead

• Test Sets will contain the regression pack, EtE test per release, this will be managed by QA Lead

Define how to Manage test Plans and Executions and Progress Reporting

• Test Plan - Create test plan according to Monthly Releases for INT , UAT & Prod testing 

...

Reporting :: Reports can be executed reports section of X-Ray based on Tests List, Test Set list , Test Plan List , Test Execution List.

• Test Executions

  • Types of Test Execution
  • Monitoring Test Execution Progress
  • Handling Test Execution Failures

• Bug Reporting

  • Guidelines for Reporting Bugs
  • Information to Include in a Bug Report
  • Bug Reporting Workflow

• Bug Creation

  • How to Create a Bug Report
  • Common Bug Types
  • Tools for Bug Creation

• Fixing a Bug

  • Steps in Bug Fixing
  • Coordination with Development Teams
  • Verifying the Bug Fix

Define Testing process including Bug creation and fix

Define Testing process including Bug creation and fix

Once a story is created Once a story is created on Jira, the testing process generally follows these steps:

...

Core_CCCME-6789 :: Verify that the user successfully logged into Core application.

Test Execution: Once the development is complete , and the code is deployed to the testing environment (usually in , typically during the Integration Testing (INT) phase, the developer hands over the story by providing a high-level demo to the QA and BA teams. This demo helps verify whether all the Acceptance Criteria (AC) have been implemented in the story. After the handover, the QA team executes proceeds with executing the test cases.

If any Any issues or bugs encountered are found, they are logged as separate Jira tickets, . The story is marked as failed and , the bug is linked to the original story, and it is assigned to the relevant responsible developer for resolution.

Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).

...

INT_iCARe :: User unable to log into iCARe application.

PO Testing :: After Integration Testing (INT)  PO testing is a phase where the Product Owner (PO) personally verifies that the completed story meets the business requirements and acceptance criteria before it moves to User Acceptance Testing (UAT). During PO Testing, the PO reviews the functionality to ensure it aligns with the expected outcomes and overall product vision. If the PO identifies any issues, they can request changes or adjustments, and any bugs found are logged in Jira for the development team to address. Once the PO is satisfied with the implementation, the story is typically approved to proceed to UAT.

Bug Fixing and Retesting: Developers work on fixing any reported bugs, updating the status of the bug tickets in Jira (e.g., from "Open" to "In Progress," and then "Resolved/ Done"). The QA team then retests the fixes and updates the status of the bug tickets accordingly.

Once the bug is passed and closed, QA Team retests the associated Story and marks its status accordingly.

User Acceptance Testing (UAT): User Acceptance Testing (UAT) is the final testing phase, where the story is validated by business users or stakeholders after it has passed Integration Testing (INT). In UAT, these users ensure that the implementation meets the defined acceptance criteria and aligns with business requirements. If any issues or bugs are discovered during this phase, they are logged in Jira, where they are tracked, updated, and reassigned to developers for resolution. The process continues until the users are satisfied that the system functions correctly and meets their needs, signalling that the story is ready for production.

Note :: The bugs raised on UAT should also be tested in lower environments i.e INT.

Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).

Ex :: UAT_iCARe :: Description of Bug/ UAT_Core :: Description of the bug

UAT_Core :: User unable to log into Core application.

Jira Bug Status and Associated Roles

• Open/New

  • Role: Reporter (could be a QA Engineer, Developer, or User)

  • Description: A new bug is reported and logged in Jira. The bug is in the queue, awaiting triage or assignment.

• Triage/Backlog

  • Role: BA, Project Manager (PM), Product Owner (PO), or Lead Developer

  • Description: The bug is reviewed for severity, priority, and assignment. The decision is made whether to work on it immediately or add it to the backlog.

• In Dev

  • Role: Developer

  • Description: The assigned developer starts working on the bug fix. This stage involves coding and preparing a solution.

• In Review/Code Review (Optional)

  • Role: Peer Developer or Lead Developer

  • Description: The fix is completed and the code is reviewed by another developer to ensure quality and adherence to standards.

• Ready For QA

  • Role: Developer / Test team

  • Description: The bug fix is complete and marked as Ready for QA. The resolution is ready for verification, typically by a QA Engineer.

• IN QA

  • Role: QA Team

  • Description: The fix is tested by the QA team to ensure it works as expected and doesn’t introduce new issues.

• Closed/Done

  • Role: QA Engineer or Product Owner

  • Description: After successful verification and testing, the bug is closed, marking the end of its lifecycle.

• Rejected/Won’t Fix

  • Role: Project Manager (PM), Product Owner (PO)

  • Description: The bug is deemed invalid, a duplicate, or is decided not to be fixed. This status is used to close issues that won't be addressed.

Bug Priority :: Low , Medium , High , Critical

Flow Diagram ::

Image Removed

Story Closure: Marks the final step in the development process, where a user story is formally completed and closed in Jira. This occurs after the story has successfully passed all testing phases, including Integration Testing (INT), Product Owner (PO) Testing, and User Acceptance Testing (UAT). The QA team or product owner verifies that all acceptance criteria have been met, all associated bugs have been resolved, and the functionality works as intended in the production environment. The Jira story's status is then updated to "Done" or "Closed," signalling that no further work is required and the feature is ready for release. Story closure also involves ensuring that any related documentation is updated and that the story is archived for future reference.

User Acceptance Testing (UAT): User Acceptance Testing (UAT) is the final testing phase, where the story is validated by business users or stakeholders after it has passed Integration Testing (INT). In UAT, these users ensure that the implementation meets the defined acceptance criteria and aligns with business requirements. If any issues or bugs are discovered during this phase, they are logged in Jira, where they are tracked, updated, and reassigned to developers for resolution. The process continues until the users are satisfied that the system functions correctly and meets their needs, signalling that the story is ready for production.

Story Closure: Marks the final step in the development process, where a user story is formally completed and closed in Jira. This occurs after the story has successfully passed all testing phases, including Integration Testing (INT), Product Owner (PO) Testing, and User Acceptance Testing (UAT). The QA team or product owner verifies that all acceptance criteria have been met, all associated bugs have been resolved, and the functionality works as intended in the production environment. The Jira story's status is then updated to "Done" or "Closed," signalling that no further work is required and the feature is ready for release. Story closure also involves ensuring that any related documentation is updated and that the story is archived for future reference.

Bug Fixing and Retesting: Developers work on fixing any reported bugs, updating the status of the bug tickets in Jira (e.g., from "Open" to "In Progress," and then "Resolved/ Done"). The QA team then retests the fixes and updates the status of the bug tickets accordingly.

Once the bug is passed and closed, QA Team retests the associated Story and marks its status accordingly.

Note :: The bugs raised on UAT should also be tested in lower environments i.e INT.

Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).

Ex :: UAT_iCARe :: Description of Bug/ UAT_Core :: Description of the bug

UAT_Core :: User unable to log into Core application.

Jira Bug Status and Associated Roles

• Open/New

  • Role: Reporter (could be a QA Engineer, Developer, or User)

  • Description: A new bug is reported and logged in Jira. The bug is in the queue, awaiting triage or assignment.

• Triage/Backlog

  • Role: BA, Project Manager (PM), Product Owner (PO), or Lead Developer

  • Description: The bug is reviewed for severity, priority, and assignment. The decision is made whether to work on it immediately or add it to the backlog.

• In Dev

  • Role: Developer

  • Description: The assigned developer starts working on the bug fix. This stage involves coding and preparing a solution.

• In Review/Code Review (Optional)

  • Role: Peer Developer or Lead Developer

  • Description: The fix is completed and the code is reviewed by another developer to ensure quality and adherence to standards.

• Ready For QA

  • Role: Developer / Test team

  • Description: The bug fix is complete and marked as Ready for QA. The resolution is ready for verification, typically by a QA Engineer.

• IN QA

  • Role: QA Team

  • Description: The fix is tested by the QA team to ensure it works as expected and doesn’t introduce new issues.

• Closed/Done

  • Role: QA Engineer or Product Owner

  • Description: After successful verification and testing, the bug is closed, marking the end of its lifecycle.

• Rejected/Won’t Fix

  • Role: Project Manager (PM), Product Owner (PO)

  • Description: The bug is deemed invalid, a duplicate, or is decided not to be fixed. This status is used to close issues that won't be addressed.

Bug Priority :: Low , Medium , High , Critical

Flow Diagram ::

Image Added

Define Guidelines on how to report a Bug

Guidelines for Reporting a Bug

...

Example of Bug ::

Title :: INT_iCare Migratiopn Migration 2 Flows :: System generating an error message when trying to add a Relationship to an account.

...