Nowadays, Agile has become the most commonly used methodology in software development life cycle, which promotes continuous iteration of development and testing concurrently, unlike the Waterfall model.

It has pre-defined short iteration cycles for 2-3 weeks to deliver quality product. QA teams have to work with developers and vice versa to achieve the goal of BVT.

In the DevOps model, use of a BVT methodology is an essential step in continuous integration maturity.

In this whole cycle, BVT has an important role in terms of delivering a high quality product on time. In BVT, QA teams run a set of pre-defined test cases on each new build to ensure that they are a stable build for further testing.

These pre-defined test cases are for core functionality so that the product’s core functionality is not broken, with further eligibility to release this build to QA team for a thorough testing.

You can see the below highlighted area, which falls under BVT.

The entire cycle takes at least 3-4 days, which depends on product complexity that requires handing over the product to QA team for further testing. And this would happen at the end of each iteration i.e. after each 2-3 weeks.

BVT contains below sub-processes:

• Generate build based on developers checked-in code
• Setup/create VMs/machines for testing
• Setup environment
  • Server installation, if there is multi-cluster setup, you have to setup server on each node
  • Database installation
  • Setup load balancer
• Build deployment
  • Deploy server application builds
  • Execute DB script
• Verify Build
  • Run pre-defined test cases for core functionality verification
• Inform QA team to start further testing

Now think about it, how does it feel when you can automate the whole thing?

Amazing, right? Yes, if we can automate whole BVT process it can easily save around 3-4 man-days on each iteration. Also, these automated processes are more reliable, and without manual errors.

BVT is a surefire mechanism to ensure no regression defects have been introduced in DevOps methodology.

But the bigger question is how? What would this approach look like?

Please see the steps below.

First step is to setup Bamboo or Jenkins, in such a way that nightly test builds would be started at the end of every sprint OR manually, which pulls the latest code from the GIT/SVN, executes unit test cases, generates build and uploads that build on Artifactory.

Once build is uploaded, it will kick-off automated BVT plan. Let’s dig into more detail in each step of BVT process.

Generate VMs:

There are multiple options to generate VMs automatically:

1. Amazon Web Service
2. Dockers
3. VMWare ESXi Server

Each one of the options have their own Rest API to clone VMs from pre-defined VM templates. These pre-defined VMs have the required software already installed. We can call upon these APIs using Python, Java OR even shell script, whatever is applicable.

Setup environment and configure servers:

Once VMs are available, we have to verify that the required web application server and database software have been installed successfully. Many of the web application server have an option to install in silent-mode.

For example, in Weblogic server, you have to answer to all the questions requested during the installation process and run the command “weblogic.exe -mode=silent -silent_xml=path_to_silent.xml”
for silent installation.

To automate this step, Ansible is a really good option.

Deploy Build from Artifactory:

Get the latest build from the Artifactory and deploy it on Web application server using Ansible. Once build is deployed on server, one has to make sure that it is successfully deployed to access a web page.

Execute defined Test cases using scripts:

Once build is deployed successfully, start executing pre-defined test cases, which cover the core functionality of the product. You can automate those test cases based on your requirement using any language and tool like, Java, Python, shell script, selenium etc.

Add Test Results into Jira using API from Test case:

It is really a good idea to add test step results into Jira directly from Test script, so that there are no manual interventions or dependencies.

There is a Zephyr plugin for Jira to achieve this. Zephyr exposes rest API, ZAPI, to add/update/delete test case/step result in Jira directly. You can use that ZAPI and develop script based on your requirement and add result into Jira during test case execution.

Send Final Status Email to Manual QA Team:

This is an important step where one has to inform the manual QA team that the build is stable/not stable for further testing. This is done using a final status email.

Conclusion:

This is just a general approach to automate build verification testing process in Agile.

We have collected quantifiable data from multiple low/average size projects, and based on that, we realize that it is possible to save 88% of efforts from manual BVT using automated BVT.

Because of automated BVT, one can save a number of man-hours, which can then be utilized for productive work, even while managing the project within the planned timelines.

To know more about how eInfochips can help your organization achieve to BVT in your QA practice, drop us an email at marketing@einfochips.com You may also want to check out our unified automation solution – EzTest.