Automation framework is very important aspect of any QA automation project. It supports the testing effort to ultimately achieve test coverage and shorter time lines with consistency for each cycle. To achieve these, the framework should be effective enough to address the challenges encountered while transforming the test cases to test scripts.

The different factors that decide the effectiveness of the framework are:

  1. Defined folder structure: Framework should have a well-defined folder structure to easily trace the items
  2. Portability: The framework should work as expected with any folder hierarchy, any drive location and type of application
  3. Error handling and recovery scenarios: The process to be followed when an error or exception occur have to be handled in framework rather than in the scripts
  4. Function libraries for general use and project specific purpose should be in separate folders for ease of maintenance
  5. Test data to be maintained on module basis to update them as when required without touching the scripts. Security to be provided to the test data
  6. Most importantly the framework should be environment independent. By implementing few things that can be configured like browser, OS etc. we can make the framework environment independent.
  7. Selection of the test scripts to be executed and batch creation should be provided
  8. Scheduling the execution of batches should be provided
  9. Test results should be easily understood by any tester with more details on the condition the application under test and test data used. This will help the tester to reproduce the defect. Capturing the screen shot will also add value to decide if it is a defect or script failure
  10. Configurable Emailing of the results with selected recipients should be provided
  11. Scripting guidelines should be documented
  12. Naming conventions and standard to be followed should be documented
  13. Proper trace-ability should be visible to understand the coverage of the automation and execution to the defects
  14. After the execution, the framework should provide some data for analysis and metric preparation
  15. Test framework should be easy to expand and maintain with less references to paid tools or other libraries

Above all these, there should be a continuous improvement plan to make the framework more matured. This will lead to reducing the manual effort and replace with the batch execution.