StartEast, Test Process Improvement (TPI)

As part of the STAR conference, they offer two one-day tutorials during the first two days of the conference. One of the tutorials I chose to attend was on Test Process Improvement (TPI), taught by Martin Pol (co-author of the TPI book) and Rick Craig.

The first time I heard about TPI was when I attended my first STAR conference shortly after moving to the QA team. At the time our QA department was just forming, so other than reading through most of the book, not much was done.  Two years later, however, out department had started to mature and I felt it was a good time to start making some serious attempts at improving our department.

One thing really interesting about the TPI approach is its flexibility, in the sense you can choose the areas you need/want to improve on and you can start working on these improvements in your department without having to get a lot of buy-in from other departments (initially).  True, for larger companies and QA/testing departments there might be a lot of additional work needed to do the initial assessment, but the model is setup in a way that an individual or even a small team in a larger group can use it to help improve themselves.

I’m a firm believer in the adage, “the only person you can change is yourself,” and in the early stages this model fits in well with this idea. My mission was to find out what I could do to help improve things in my department, and this approach seems to be a good start.

The model is set up with 20 different categories (test strategy, metric, test-automation, etc) with each category having “maturity” levels of A (beginning) up to D (advanced). The TPI book itself guides you through the improvement process by specifying the criteria you would need to meet to advance to the next level. These criteria could be certain process or tasks that need to be in place, or dependencies on other areas in the model needing to be at a certain level. To begin the process you would do an initial assessment to figure out the level of each of the 20 categories and from there you can determine what to work on improving.

After doing the initial assessment I was able to determine the area I wanted to work on improving first was the Life-cycle modal category. The reasons I chose this category was because several of the other categories depend on this being at a certain level before meeting their requirements, and the only additional thing I need to do to meet the requirements was to begin writing out some form of test plan.

Up to this point we had never written out a test plan because we worked in small teams and figured we knew what was going on, and what needed to be done. Plus, when looking over test plan templates from other books and resources, it appeared a lot of the information contained in them really didn’t apply or wasn’t really necessary. I thought writing up a test plan was a waste of time, but this was something I could start doing to help make the improvements I knew we needed. So with the help of one of the testers attending the conference with me, we came up with a working outline of our test plan template.

For the next couple of weeks I worked on completing and adjusting the test plan prototype with the project I was currently working on. In the end, I wasn’t too surprised to discover the amount of time we need to test the project was a longer than had been allocated but I didn’t give it much thought because I had assumed we would just need to do as much testing as possible in our allocated time frame. One thing that amazed me was to learn by doing risk assessments and quality characteristic analysis, I was able to get a pretty good idea for what we would need to focus on when it came time to testing.

After I had finished up the plan I decided I would show the prototype to the product manager (and my boss) to get his opinions on how the test plan was set up. I really didn’t expect a whole lot to come of the specific test plan for the product in question because I was looking for feedback on the idea of the test plan. When I showed the plan to the product manager he seemed to really like the idea, but he wanted me to actually update the plan to show how long the testing period would need to last if we wanted to test what the plan had said. I was a little surprised by the request because I had assumed our testing efforts would need to fit into the time allocated. I updated the plan and showed it to him and quickly learned the other teams were assuming the testing time would be able to fit into the time allocated. Because of this, some discussions and re-evaluation to took place.

Just by taking a beginning step, there has already been a significant impact on how things will be done with future projects. And since this was something I would do as the QA manager, and wanted to do, I was able to starting making improvements in my department, and the company, by changing myself.

If you find yourself looking for ways to improve your QA/testing group, but aren’t sure how to start, I would recommend checking into this approach. Even if you don’t follow the process completely, it should at least give you some things to think about if you want to make improvements.


The test plan scheme I used to help create our working test plan came from the TMap book. Software Testing: a Guide to the TMap Approach, by Martin Pol, Ruud Teunissen and Erik van Veenendaal.

posted @ Saturday, June 12, 2004 10:27 PM