Automation framework is very important aspect of any QA automation project. It supports the testing effort to ultimately achieve test coverage and shorter time lines with consistency for each cycle. To achieve these, the framework should be effective enough to address the challenges encountered while transforming the test cases to test scripts.
The different factors that decide the effectiveness of the framework are:
- Defined folder structure: Framework should have a well-defined folder structure to easily trace the items
- Portability: The framework should work as expected with any folder hierarchy, any drive location and type of application
- Error handling and recovery scenarios: The process to be followed when an error or exception occur have to be handled in framework rather than in the scripts
- Function libraries for general use and project specific purpose should be in separate folders for ease of maintenance
- Test data to be maintained on module basis to update them as when required without touching the scripts. Security to be provided to the test data
- Most importantly the framework should be environment independent. By implementing few things that can be configured like browser, OS etc. we can make the framework environment independent.
- Selection of the test scripts to be executed and batch creation should be provided
- Scheduling the execution of batches should be provided
- Test results should be easily understood by any tester with more details on the condition the application under test and test data used. This will help the tester to reproduce the defect. Capturing the screen shot will also add value to decide if it is a defect or script failure
- Configurable Emailing of the results with selected recipients should be provided
- Scripting guidelines should be documented
- Naming conventions and standard to be followed should be documented
- Proper trace-ability should be visible to understand the coverage of the automation and execution to the defects
- After the execution, the framework should provide some data for analysis and metric preparation
- Test framework should be easy to expand and maintain with less references to paid tools or other libraries
Above all these, there should be a continuous improvement plan to make the framework more matured. This will lead to reducing the manual effort and replace with the batch execution.