Often, people think agile testing only can be implemented on an agile project. Actually agile testing is a set of engineering practices, and it can be implemented on any kind of project.

For example, as a typical agile testing practice, test-driven requirements can be leveraged in any kind of project. Even for a traditional waterfall project, we can do test case/script development with requirement development in parallel. All unclear points for preconditions, scenarios, testing data will be found and clarified earlier. It is easy to help a team to make requirements ready as early as possible, and it will avoid a lot of costs caused by bad quality in the requirements.

Then which kind of agile testing practice can be leveraged for any kind of project? It will include but not be limited to the following:

• Involves all members for testing, with special expertise contributed by testers
• Testing from the customer perspective as early as possible
• Test-driven requirement
• Test-driven development
• Pair testing
• Automated regression testing
• Continuous integration with automated testing

For those teams with less testing and automated testing experience, I recommend you try item 1, 2 and 3 first. Please feel free to try the left ones, especially automated ones, if your team has strong automation skills. You will gain value quickly due to automation.

I am not requesting that you implement all of above practices. But it will be very helpful to improve your project’s quality if you can try several of them. And I will post a separate blog post soon about those agile testing practices one by one if you are interested.

Recently one of the projects that I was involved in was finished. The testing was blocked at the end phase, and the cost was almost exhausted even though the testing had not been finished when testers were rolled off.

People always talk about quality, and most people agree that quality should be the most important aspect of a successful project/product, but the quality I am talking about is based on the business vision and value instead of only considering technical perspectives. The reality is that often times, teams will cut quality to meet a fixed schedule or cost when issues arise.

As a quality assurance analyst, I would like to outline a method for building up quality in your project. It can be likened to baking a cake. You cannot spend half the time to bake a cake with double the temperature. When your end goal is to produce a quality product, whether it’s a cake or a technical project, it is nearly impossible to spend less time to deliver the same level of quality deliverables, with more team members. Even worse, sometimes teams involve more team members at the middle or end of a project. In these cases, a project can end in complete chaos.

So we have to bake quality into each project in every single unit, step by step, including each user story, every different phase and in different sprints/iterations, especially at the beginning of the project.

Here is something I tried in different projects to help build up quality. I hope it will be helpful for you as you work on quality assurance in upcoming projects:

• Make sure the requirement is testable
• Introduce quality when designing
• Test earlier as possible
• Implement automation testing in proper time
• Build & Deployment management
• Risk management for 3^rd party integrations and dependencies
• Sprint/Iteration reviews with the client

I may talk more about different aspects of quality assurance in upcoming blog posts if you are interested in it. Start to try to bake your own quality “cake” from now on and you will find less pain at the end of your project. Good luck!

I am travelling in our Detroit office today. I am working with our U.S. team to leverage Scrum for automation and load testing on a current project for a client. Typically, this process includes system testing with waterfall, which cannot include Agile methodology. However, it does work for our current project. And I will share with you what our solution is and how it works.

Automation Testing

My U.S. teammates plan to use automation testing to replace manual testing. So I helped them to create a one-month plan for Phase1 development using Agile since we have one team member from our Global Delivery Center in China to work on it onshore.

Week1: Create a sampler by using the automation tool to cover all components testing, including image, URLs, integration and so on.

Week2: Create common components’ framework for automation scripts and integrate with sample’s testing.

Week3: Finish all common components testing.

Week4: Finish smoke testing by calling different common components.

The final goal for phase1 is to create smoke test scripts and integration test scripts which could be executed and get reliable results. But for every single week, we will have Daily stand up, Demo for deliverables, review activities for next week’s plan and ready automation testing scripts. It will help us to control the issues and risk easily. Even if we have to stop the work for automation during this month, we still have several automation test cases that can be used to reduce our current manual testing effort.

Load Testing

For our load testing team, we have one U.S. colleague designated to work on this project from our Detroit office as a lead and then several team members working offshore at our Global Delivery Center in China. We have couple of requests for load testing enhancement.  I discussed with our team lead all of the enhancement including scope, testing strategy, load testing execution and testing results analysis. We wrote down all user stories and prioritized them according to the impact for the client.

I created one planning board in a wiki  as the following to share all kinds of enhancements for load testing.

Then it is easy for both of our teams to share ideas about user stories, task break down and progress.

Every two weeks, our U.S. team lead and GDC offshore team review a task break down and estimation together. Our team will pick up the user stories and track the status in our daily Scrum meeting. Every week, our offshore team will demo the finished enhancements to the US team lead. Though we are working on enhancements for our load testing, I believe the same method can be adapted for load testing as well.

Download our latest whitepaper on Improving the Efficiency and Effectiveness of Software Testing with Automated Testing.

These days we are preparing next year’s CMMI Level 5 reassessment and delivered a series of training about CMMI. So people are easy to compare CMMI and Agile when we were doing training. And somebody will say they are different and conflict. How can we implement them together?

So what is the difference between CMMI and Agile?

From the perspective of people who support CMMI, they think the gap between CMMI and agile:

But for people who support Agile, their opinion is totally different:

CMMI focuses on process and emphasizes the transparence of the process. It seems that conflicts with Agile soul, reducing additional management effort, self-organized and embracing change.

Based on my understanding, Agile is a kind of culture. It respects people’s contributions and takes into consideration everyone’s ideas. It encourages initiative, innovation and teamwork for all kinds of activities.  Also Agile includes a lot of best practices in software engineering, such as continuous integration, daily stand up and retrospective meeting.

Also, you will find that the CMMI model never mentioned that you need heavy documents or standard processes. Actually, CMMI just defines the different process areas and goals, recommends some samples for best practices, but does not request that you implement all kinds of things with heavy documents, complex processes and tough management.

At this point, we can combine Agile and CMMI together, using simple methods, such as slight documents or tools, to achieve the requests and goals of CMMI. At the same time, we can refer to the systematic thinking of CMMI, for example quantitative management from CMMI, and we can try to collect useful data from small scrum teams to generate organizational metrics.

I will talk about one detailed example of how to combine them together in daily work in my next blog post. Hope you will like it:)

Today, one tester complained to me:” Developer A always asks me to help him to test his code earlier in dev environment without any testing by himself!”

I heard the same complains form the different projects. When I was doing testing in different projects, I often encountered the similar situation as well. From my perspective, I think this kind of developers is not bad at least if some developer asked to help to test his code earlier. Because he has good understanding of ensuring quality and he knows we should find defects as soon as possible to reduce the cost of defects.

And in agile today, people are eager to test earlier for finding defects and can deliver real shippable things at the end of sprints or iterations. But the question is that what is the better way to test earlier? Obviously, it is not to ask testers to help to test our code in development environment without any other testing.

Here is some experience shared what I got from my different projects about how to test earlier before delivering.

Before checking in code in integrated environment

• Code review: I prefer it is a kind of testing as well. It is a not special and has been come up for years, but the difficulties are how to persist on it in the project. Do not save time for code review, it will help us to save more time later on.
• Unit Test: Test Coverage is 100% percentage is everyone’s dream. But I seldom saw it happened in reality. So the realistic code coverage and efficient measurement will be helpful for us.
• Basic functional testing manually if possible: It is not popular now, but recommend project to try this way before team check in their code. It is an easiest way and high ROI (return on investment) to find the obvious defects. It can also improve the developer’s understanding of requirements as well.
• Automated smoke testing if possible: Is it possible to run automated smoke testing in your local test environment? Of course, automated test scripts can be checked out in our local environment to run it and it can help us to find if we produce a new defect or break existing function before checking in our code into integration environment to make disaster.

After checking in code in integrated environment

• Automated smoke testing: It could be triggered automatically after finish build in integration environment. And test results can be sent to the whole team by email directly.
• Basic functional testing manually: It will focus on main work flow or high priorities functions and be supplement of automated smoke testing.
• Automated regression testing: Every sprint or iteration automated smoke testing scripts can be added in this pool. Stabled basic functional testing case in previous sprint or iteration can be exchanged to be automation as well.

The precondition for the above testing, we need to

• Set up a continuous integration environment to do build automatically firstly.
• Team need to get commitment to do basic functional testing manually by everyone at the beginning of the project.
• All check in procedure needs to be followed up strictly
• Need to plan the different level’s testing at the beginning of project

Now, can we test earlier?

Currently more and more projects are multi-shore in a lot of big companies. Clients always ask us one question: How much effort needs to be spent to communicate with offshore teams if we work with one. It is one concern from clients that they need to spend more time to communicate with offshore team and usually they do not want to communicate with an offshore team directly.

So the question is how much effort should really be spent communication with offshore teams?

Basing on my experience, communication with an offshore team will not be much different from communicating with your onshore team if you have a good communication plan at the beginning of the project and implement your communication using some useful tools according to plan during the project. We already finished half a year’s project which has one US lead from client side to communicate with our team directly. And the following lists the best practices from my previous project on how to communicate with our US client directly and more smoothly.

Daily Tasks & Status

• Define the efficient way to assign daily tasks to offshore team members, such as email, wiki, SharePoint or JIRA
• All tasks should have clear priorities to guide team members to finish them by sequence
• One team member should be responsible for daily status and send it to the US client before the end of working day according to a template
• Recommend marking every task’s effort in a daily status and help the US client understand our productivity level
• Recommend using tools to store daily status and make sure it can be tracked easily

Requirement Communication

• Set up one method to do daily requirement communication. It can be WIKI page, share point, one shared document and so on
• Session can be set up if there are a lot of new requirements coming up. It can be conducted with client onshore team together and it will not spend extra time.
• The meeting can be recorded and shared with the offshore team due to the different time zones

Weekly Report & Sync-up Call

• Weekly report needs to be sent to the US client. It will contain an overview about the offshore team’s achievements, next week’s plan, coming milestones and risks & issues. Recommend focusing on communication about risk & issues and coming plan since we already exchanged daily status information
• A short Sync-up Call will be another good method. Usually it will not include too many details and just sync-up every week’s status between US client and offshore team. Demo is also a good way to show your achievements in sync-up call

Contact point & Emergency contact

• Contact point and responsibilities should be indicated clearly both of client and offshore side
• Emergency contact information need to be published for both of sides at the beginning of project

Others

• Do not recommend using heavy documents to do communication with client or offshore team
• Do not recommend writing long emails
• Instant massenger can be used to communicate as well

Now let us see how much effort a client needs to spend with an offshore team and what is the difference with an onshore team?

If you are a client, can you tell me how much time you spend communicating with your offshore team or how much time was saved communicating as recommended above?