Table of
...
Testing Strategy Definition
- Overview of Testing Strategies
- Importance of a Defined Strategy
- Components of a Testing Strategy
...
Defining the Testing Process
Contents
Define testing type per environment Monthly Release
INT/QA
Functional Testing ::
Definition: Test the scenarios Identified under the Stories
Define testing type per environment Weekly Release
Define Test scripts repository (JIRA folder) for each testing type
Define how to Manage test Plans and Executions and Progress Reporting
Define Testing process including Bug creation and fix
Define Guidelines on how to report a Bug
Define testing type per environment Monthly Release
INT/QA
Functional Testing :: Test the scenarios Identified under the StoriesIncludes:
Happy path scenarios :: This confirms the system's ability to successfully complete transactions and workflows without encountering errors, thereby validating the basic functionality and reliability of the integrated system
...
UAT Readiness :: (to be included into UAT Internal Readiness)Stories moving into UAT are bug free and all tests under the stories are passed. All the bugs raised in the stories are fixed , if not triaged (fixed) with product owner to be created as Project backlog’s in Jirauser stories have been resolved. If any bugs remain unfixed, they should be discussed with the Product Owner (PO) and added to the project backlog in JIRA.
UAT
Functional Testing :: Same as above
NRT /Regression Testing (Functional Happy Path) ::Definition: Continuous Continuous execution of regression tests to ensure that new changes do not introduce new defects.
Regression test suite is 's should be updated regularly to cover new features and fixed defects. (These tests scripts should be highly sophisticated such that this these can be automated in future) .
Happy Path test cases from all previous functional testing can be incorporated into the Regression test suite. This ensures that critical functionality is consistently validated and that any changes or updates do not disrupt existing features. Integrating these test cases into the regression suite enhances test coverage and reliability, providing greater confidence in the stability of the system.
...
Prod Readiness :
...
: Ensures that system is fully prepared for deployment to the live environment, with all critical functionalities tested, performance optimized , All bugs are tested and passed, if any bugs are not fixed , then they are created as Project backlog’s in Jira.
E2E : (Optional)
Definition: E2E testing simulates real-world user scenarios to validate that the entire application, including its interfaces and interactions, performs seamlessly. This testing is crucial for Definition: E2E testing simulates real-world user scenarios to validate that the entire application, including its interfaces and interactions, performs seamlessly. This testing is crucial for identifying integration issues, ensuring data integrity, and providing confidence in the system's readiness for deployment.
...
Prod Readiness :: Ensures that system is fully prepared for deployment to the live environment, with all critical functionalities tested, performance optimized , All bugs are tested and passed, if any bugs are not fixed , then they are created as Project backlog’s in Jira.
PROD
Sanity Testing :: Testers Testers should use specific test cases designed to validate essential operations without delving diving into extensive testing. The goal is to quickly confirm system stability and functionality, minimizing user disruption while ensuring that the production environment remains operational and reliable.
...
Delete created test records: testers should have the permission to delete data, directly using delete Button (system might need to be reviewed to allow or a person appointed to do this task)
Define testing type per environment Weekly Release
Assumption: weekly release will not include changes on automations (Flows, triggers), if the case, then a more robust testing should be considered: Happy Path, Alternative Flows, Negative (-Ve).
...
PROD Readiness :: Stories moving into UAT are bug free and all tests under the stories are passed. All the bugs raised in the stories are fixed , if any bugs are not fixed , then they are created as Project backlog’s in Jira.
- Key Steps in the Testing Process
- Roles and Responsibilities
- Entry and Exit Criteria
Managing Test Plans
- Overview of Test Plans
- How to Create and Update a Test Plan
- Managing Changes to Test Plans
Test Scripts Repository
- Defining a Test Scripts Repository
- Organizing Test Scripts
- Version Control for Test Scripts
Define Test scripts repository (JIRA folder) for each testing type
Define Test scripts repository (JIRA folder) for each testing type
X-Ray Test Test Repository :: This will be our test case Library with following limitation ie test repo is a project level feature, so each test repository can only contain test from its projects (we cannot mix tests from multiple projects into a single repository and organize them together)
All test cases which we create end up in Test Repo by default.
X-Ray Test Sets :: Is a group of tests , we can consider this to be the equivalent of a single folder inside the test repository.
...
Folders - will be used to structured the repository by Application >> Process >> Test Type - this folders will contain all test scripts that will be created by BAs, QA Lead will maintain the structure, and Test Scripts cannot be cloned.
Test Sets - will be used to include all scripts needed for a certain release, this will be managed by QA Lead
Test Sets will contain the regression pack, EtE test per release, this will be managed by QA Lead
Define how to Manage test Plans and Executions and Progress Reporting
Test Plan Test Plan - Create test plan according to Monthly Releases for INT , UAT & Prod testing
...
Reporting :: Reports can be executed reports section of X-Ray based on Tests List, Test Set list , Test Plan List , Test Execution List.
Test Executions
- Types of Test Execution
- Monitoring Test Execution Progress
- Handling Test Execution Failures
Bug Reporting
- Guidelines for Reporting Bugs
- Information to Include in a Bug Report
- Bug Reporting Workflow
Bug Creation
- How to Create a Bug Report
- Common Bug Types
- Tools for Bug Creation
Fixing a Bug
- Steps in Bug Fixing
- Coordination with Development Teams
- Verifying the Bug Fix
Define Testing process including Bug creation and fix
Define Testing process including Bug creation and fix
Once a story is created on Jira, the testing process generally Once a story is created on Jira, the testing process generally follows these steps:
Story Refinement Session: A story refinement session is a collaborative meeting where the development team, product owner, and stakeholders review and clarify user stories in the product backlog. The goal is to ensure that stories are well-understood, properly scoped, and ready for upcoming sprints. During the session, the team clarifies requirements, estimates effort, discusses technical challenges, reviews and refines acceptance criteria, and prioritizes the stories. The outcome is a set of clearly defined, estimated, and prioritized stories that are ready for development, helping streamline the subsequent sprint planning process.
...
Core_CCCME-6789 :: Verify that the user successfully logged into Core application.
Test Execution: Once the development is complete , and the code is deployed to the testing environment (usually in , typically during the Integration Testing (INT) phase, the developer hands over the story by providing a high-level demo to the QA and BA teams. This demo helps verify whether all the Acceptance Criteria (AC) have been implemented in the story. After the handover, the QA team executes proceeds with executing the test cases.
If any Any issues or bugs encountered are logged found, they are logged as separate Jira tickets, . The story is marked as failed and , the bug is linked to the original story, and it is assigned to the relevant responsible developer for resolution.
Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).
...
INT_iCARe :: User unable to log into iCARe application.
PO Testing :: After Integration Testing (INT) PO testing is a phase where the Product Owner (PO) personally verifies that the completed story meets the business requirements and acceptance criteria before it moves to User Acceptance Testing (UAT). During PO Testing, the PO reviews the functionality to ensure it aligns with the expected outcomes and overall product vision. If the PO identifies any issues, they can request changes or adjustments, and any bugs found are logged in Jira for the development team to address. Once the PO is satisfied with the implementation, the story is typically approved to proceed to UAT.
Bug Fixing and Retesting: Developers work on fixing any reported bugs, updating the status of the bug tickets in Jira (e.g., from "Open" to "In Progress," and then "Resolved/ Done"). The QA team then retests the fixes and updates the status of the bug tickets accordingly.
Once the bug is passed and closed, QA Team retests the associated Story and marks its status accordingly.
User Acceptance Testing (UAT): User Acceptance Testing (UAT) is the final testing phase, where the story is validated by business users or stakeholders after it has passed Integration Testing (INT). In UAT, these users ensure that the implementation meets the defined acceptance criteria and aligns with business requirements. If any issues or bugs are discovered during this phase, they are logged in Jira, where they are tracked, updated, and reassigned to developers for resolution. The process continues until the users are satisfied that the system functions correctly and meets their needs, signalling that the story is ready for production.
Note :: The bugs raised on UAT should also be tested in lower environments i.e INT.
Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).
Ex :: UAT_iCARe :: Description of Bug/ UAT_Core :: Description of the bug
UAT_Core :: User unable to log into Core application.
Jira Bug Status and Associated Roles
Open/New
Role: Reporter (could be a QA Engineer, Developer, or User)
Description: A new bug is reported and logged in Jira. The bug is in the queue, awaiting triage or assignment.
Triage/Backlog
Role: BA, Project Manager (PM), Product Owner (PO), or Lead Developer
Description: The bug is reviewed for severity, priority, and assignment. The decision is made whether to work on it immediately or add it to the backlog.
In Dev
Role: Developer
Description: The assigned developer starts working on the bug fix. This stage involves coding and preparing a solution.
In Review/Code Review (Optional)
Role: Peer Developer or Lead Developer
Description: The fix is completed and the code is reviewed by another developer to ensure quality and adherence to standards.
Ready For QA
Role: Developer / Test team
Description: The bug fix is complete and marked as Ready for QA. The resolution is ready for verification, typically by a QA Engineer.
IN QA
Role: QA Team
Description: The fix is tested by the QA team to ensure it works as expected and doesn’t introduce new issues.
Closed/Done
Role: QA Engineer or Product Owner
Description: After successful verification and testing, the bug is closed, marking the end of its lifecycle.
Rejected/Won’t Fix
Role: Project Manager (PM), Product Owner (PO)
Description: The bug is deemed invalid, a duplicate, or is decided not to be fixed. This status is used to close issues that won't be addressed.
Bug Priority :: Low , Medium , High , Critical
Flow Diagram ::
Story Closure: Marks the final step in the development process, where a user story is formally completed and closed in Jira. This occurs after the story has successfully passed all testing phases, including Integration Testing (INT), Product Owner (PO) Testing, and User Acceptance Testing (UAT). The QA team or product owner verifies that all acceptance criteria have been met, all associated bugs have been resolved, and the functionality works as intended in the production environment. The Jira story's status is then updated to "Done" or "Closed," signalling that no further work is required and the feature is ready for release. Story closure also involves ensuring that any related documentation is updated and that the story is archived for future reference.
User Acceptance Testing (UAT): User Acceptance Testing (UAT) is the final testing phase, where the story is validated by business users or stakeholders after it has passed Integration Testing (INT). In UAT, these users ensure that the implementation meets the defined acceptance criteria and aligns with business requirements. If any issues or bugs are discovered during this phase, they are logged in Jira, where they are tracked, updated, and reassigned to developers for resolution. The process continues until the users are satisfied that the system functions correctly and meets their needs, signalling that the story is ready for production.
Story Closure: Marks the final step in the development process, where a user story is formally completed and closed in Jira. This occurs after the story has successfully passed all testing phases, including Integration Testing (INT), Product Owner (PO) Testing, and User Acceptance Testing (UAT). The QA team or product owner verifies that all acceptance criteria have been met, all associated bugs have been resolved, and the functionality works as intended in the production environment. The Jira story's status is then updated to "Done" or "Closed," signalling that no further work is required and the feature is ready for release. Story closure also involves ensuring that any related documentation is updated and that the story is archived for future reference.
Bug Fixing and Retesting: Developers work on fixing any reported bugs, updating the status of the bug tickets in Jira (e.g., from "Open" to "In Progress," and then "Resolved/ Done"). The QA team then retests the fixes and updates the status of the bug tickets accordingly.
Once the bug is passed and closed, QA Team retests the associated Story and marks its status accordingly.
Note :: The bugs raised on UAT should also be tested in lower environments i.e INT.
Naming Convention for Bug Description (This helps in identifying in which env the bug was raised).
Ex :: UAT_iCARe :: Description of Bug/ UAT_Core :: Description of the bug
UAT_Core :: User unable to log into Core application.
Jira Bug Status and Associated Roles
Open/New
Role: Reporter (could be a QA Engineer, Developer, or User)
Description: A new bug is reported and logged in Jira. The bug is in the queue, awaiting triage or assignment.
Triage/Backlog
Role: BA, Project Manager (PM), Product Owner (PO), or Lead Developer
Description: The bug is reviewed for severity, priority, and assignment. The decision is made whether to work on it immediately or add it to the backlog.
In Dev
Role: Developer
Description: The assigned developer starts working on the bug fix. This stage involves coding and preparing a solution.
In Review/Code Review (Optional)
Role: Peer Developer or Lead Developer
Description: The fix is completed and the code is reviewed by another developer to ensure quality and adherence to standards.
Ready For QA
Role: Developer / Test team
Description: The bug fix is complete and marked as Ready for QA. The resolution is ready for verification, typically by a QA Engineer.
IN QA
Role: QA Team
Description: The fix is tested by the QA team to ensure it works as expected and doesn’t introduce new issues.
Closed/Done
Role: QA Engineer or Product Owner
Description: After successful verification and testing, the bug is closed, marking the end of its lifecycle.
Rejected/Won’t Fix
Role: Project Manager (PM), Product Owner (PO)
Description: The bug is deemed invalid, a duplicate, or is decided not to be fixed. This status is used to close issues that won't be addressed.
Bug Priority :: Low , Medium , High , Critical
Flow Diagram ::
Define Guidelines on how to report a Bug
Guidelines for Reporting a Bug
...
Example of Bug ::
Title :: INT_iCare Migratiopn Migration 2 Flows :: System generating an error message when trying to add a Relationship to an account.
...