I just finished reading Are Your Lights On? by Donald C. Gause and Gerald M. Weinberg. The subtitle of this book is “How to Figure Out What the Problem REALLY Is”. The subtitle, coupled with the Customer Reviews on Amazon, is what drove my decision to buy/read the book.
My favorite chapter is Chapter 10, "Mind Your Meaning". This chapter deals with the words that are used in written communication and how there are multiple meanings that can be taken from what is said. The first example in the book is about a sign hanging in a window that says, “Nothing is too good for our customers”. The example goes on to question what that statement means. Two optional meanings listed in the book are, “There is no thing in the world that is too good for our customers” and “Giving them nothing would be giving them something too good for them”.
While the book approaches the problem with problems in a humorous way, there is much to learn from it that can be applied to everyday situations. The reason that Chapter 10 is my favorite chapter is that is where I need the most work. It will be a bit of a challenge to/for me. When one person communicates with another, there is often the assumption that the other party “sees” what is in their head and "understands" what their intention is. This is not necessarily true. Unless people build a relationship between them it is often difficult to be on the same page.
The book describes a problem as “a difference between things as desired and things as perceived”. This definition certainly did give me a lot to think about considering that it widens the circle of what I would consider a problem in my little world with my little biases. If someone feels like they have a problem with me, then they do have a problem with me? And vice versa? Then how do I know if the problem is mine? Or someone else’s? This book has some great (and humorous) examples of problems and addresses “whose problem” it could be by listing out the potentials.
“Are Your Lights On?” is divided into 6 parts:
Part 1: What is the Problem?
Part 2: What is the Problem?
Part 3: What is the Problem Really?
Part 4: Whose Problem is It?
Part 5: Where Does it Come From?
Part 6: Do We Really Want to Solve It?
I found the book thoughtful, fun to read, and insightful.
Followers
Wednesday, March 25, 2009
Sunday, March 22, 2009
Experimenting with "COTS"
The company I work for has hired 3 Temporary “COTS” testers, and by this I mean, “Came in Off the Street”, not Commercial Off the Shelf. They were hired with the understanding that 1 position could become permanent. The reasoning behind the hires in the first place is the growth of the products that we are testing. The basic job they will fill is running scripted test cases for legacy products while the more experienced testers are working on the new one.
We have no set training program for new testers, especially under these circumstances. Information ends up passed on as needed from multiple sources. This was no different when I was hired on in a similar situation. Self-learning, self-motivation, self-management, and self-leadership are essential for developing a career in software testing.
Although I just stated that this “was no different”, it actually is different in one respect. The team I work with was previously exposed to Rapid Software Testing. This exposure caused me to wonder what would happen if these new folks were exposed to some of these ideas early on? I am not a teacher by nature, but when I have a passion for something, I have been known to inspire. And a real live experiment was available, how could I resist?
The products/applications that I am test lead of were the only ones left in an RC period, so I inquired of my manager if I could have the temps. This was agreed to so I set about on my mission. Please be advised that much of the following information is based upon what I gleaned from the Rapid Software Testing course, and I give credit to such.
Day One:
1. I went in their shared office and opened up a five minute conversation with them by asking “What is a bug?” and following that with “who are the people that matter?”.
2. History of the product that I was having them look at
3. Mission: “You are an alien from another planet; you have landed on Planet “Insert Product Name Here”. Your prime directive is to gather information on this product to bring home to your people.”
4. Tools: Something to present your information on, decide this for yourself
5. Questions and Answers
6. Task: Email the information to me at the end of the day
I visited them a few times during the day to see how it was going and ask what type of information they were finding about the product. One of them was way outside his typical comfort zone given a mission with so “few guidelines”. I explained a bit about Exploratory Testing and Rapid Software Testing to them.
When I received their Information Sheets at the end of the day, it was amazing how different each one’s focus was – in fact one had no focus, one appeared to have scripted test cases, and one had a few product enhancements.
I commented each of these out and emailed them back to the appropriate owner. My comments/questions included things like “what would happen if you did this” or “how is this information important to which stakeholder”. I pointed out the good and bad in each report. One spent so much time making the Information look like a Presentation that I commented about how more time spent in this document is less time spent testing/exploring the product. (I deliberately did not give them guidelines on this when I asked them to do it, this way I would have some sort of an understanding about what they feel is important information.)
My manager stopped by to ask me what I had them doing. I explained the above assignment to him and he asked me if I had given them any oracles. I said they are available, but I had not given them to the group. I wanted to see what they would find out on their own. It is easier to give a lesson in something when someone sees the value of it. A couple of them did seek out the Help files on their own. And one of the persons found a viable bug in the product.
Day Two:
1. Went into their office after giving them a half-hour to go over yesterday’s returned Information Sheets
2. First Assignment – without helping each other, go through and uninstall the application from yesterday. Gather information on this and ask yourself questions about it. Do I like the uninstall process? Why or why not? I set them to task to investigate and gather information on it. This was a timed 30 minute session.
3. Day two’s five minute conversation was about the constant changes and challenges in testing. Being able to switch from one task to the next without too much time in between is essential in our department.
4. Discussed the uninstall process of the product – this particular product has an ugly uninstall process and leaves too much behind. I wanted to see if anyone found it all. No one did, but I showed them where everything was and explained why this product did not meet our company’s standards for uninstall, that this was a bug and there are plans to fix it.
5. History: I presented them with product two that is in the group of products that I am responsible for and gave them the history on it.
6. Mission: Same as Day One
7. Oracles: I gave them the definition of an oracle and presented them with three of them. User guide, help files, and marketing material.
8. Questions and Answers
9. Tools: Something to present your information on. Decide this for yourself.
10. Task: Email the information to me at the end of the day.
Once again, I popped in and out of their office to make myself available for questions and to add little bits of information to the task. The same person who found a viable bug in Product One/Day One, found a bug in this Product Two/Day Two.
When I received the emails at the end of the day, only two of the temps had sent me their Information Sheets. (Not sure what happened to the third and I hate to make assumptions.) This time I gave them each other’s sheets to check out. This sets out to instill the “constant change” and to set up for the Day Three conversation.
Day Three:
1. Went into their office after giving them a half hour to review each other’s Information Sheets
2. Day Three’s five minute conversation was on sharing information. After acknowledging to them that I am aware that they may feel that they are in a competition for the possible permanent position at the end of the temp season, I informed them that their competitive edge is what they do with what they learn and that most of what they will need for this edge will take place on their own time, not within the office that they share. I spoke with them about the necessity of sharing information and used the example of my father to drive home the point (Why Share). I discussed how they would need to become reliant upon each other and build a bit of a mini-team within the team. Information sharing and teamwork are essential for success.
3. First task of the day was a 30 minute session on uninstalling the product from Day Two and to gather the information about it together. We then discussed what they found out about it.
4. History: I gave them the history of the product that I was having them look at on Day Three. Being part of the test team for this one from the ground up allowed me to explain what it was like to test a product with little documentation and how the project people had to focus on getting the product/application changes implemented, tested, and out the door. This product/application is much more complex than the other two that the temps had installed. It had scared off a few experienced testers when it originally came in house.
5. Mission: Same as Day One, only by all means help each other out.
6. Oracles: Same as Day Two.
7. Questions and Answers
8. Tools: The template that our team uses when conducting Exploratory Testing.
9. Task: email me the ET Sheets at the end of the day.
That was Friday. When I briefly glanced at their ET Sheets before heading out for the weekend, I was pleased with the results. Each one has their own style of writing, but each sheet was more focused on information and contained less “filler” material.
While they are not being utilized on another project (which is subject to change while I type); I will continue to share information with them. The above is a brief summary of what was brought to them in the past week, but within each “lesson” there were other “embedded lessons” as well. I have not planned out the “lessons” but looked for what was lacking or not understood in each previous lesson in order to drive the next.
I have no idea if what I am doing is going to produce any benefits to them, to the team, or to the stakeholders. Time will tell.
We have no set training program for new testers, especially under these circumstances. Information ends up passed on as needed from multiple sources. This was no different when I was hired on in a similar situation. Self-learning, self-motivation, self-management, and self-leadership are essential for developing a career in software testing.
Although I just stated that this “was no different”, it actually is different in one respect. The team I work with was previously exposed to Rapid Software Testing. This exposure caused me to wonder what would happen if these new folks were exposed to some of these ideas early on? I am not a teacher by nature, but when I have a passion for something, I have been known to inspire. And a real live experiment was available, how could I resist?
The products/applications that I am test lead of were the only ones left in an RC period, so I inquired of my manager if I could have the temps. This was agreed to so I set about on my mission. Please be advised that much of the following information is based upon what I gleaned from the Rapid Software Testing course, and I give credit to such.
Day One:
1. I went in their shared office and opened up a five minute conversation with them by asking “What is a bug?” and following that with “who are the people that matter?”.
2. History of the product that I was having them look at
3. Mission: “You are an alien from another planet; you have landed on Planet “Insert Product Name Here”. Your prime directive is to gather information on this product to bring home to your people.”
4. Tools: Something to present your information on, decide this for yourself
5. Questions and Answers
6. Task: Email the information to me at the end of the day
I visited them a few times during the day to see how it was going and ask what type of information they were finding about the product. One of them was way outside his typical comfort zone given a mission with so “few guidelines”. I explained a bit about Exploratory Testing and Rapid Software Testing to them.
When I received their Information Sheets at the end of the day, it was amazing how different each one’s focus was – in fact one had no focus, one appeared to have scripted test cases, and one had a few product enhancements.
I commented each of these out and emailed them back to the appropriate owner. My comments/questions included things like “what would happen if you did this” or “how is this information important to which stakeholder”. I pointed out the good and bad in each report. One spent so much time making the Information look like a Presentation that I commented about how more time spent in this document is less time spent testing/exploring the product. (I deliberately did not give them guidelines on this when I asked them to do it, this way I would have some sort of an understanding about what they feel is important information.)
My manager stopped by to ask me what I had them doing. I explained the above assignment to him and he asked me if I had given them any oracles. I said they are available, but I had not given them to the group. I wanted to see what they would find out on their own. It is easier to give a lesson in something when someone sees the value of it. A couple of them did seek out the Help files on their own. And one of the persons found a viable bug in the product.
Day Two:
1. Went into their office after giving them a half-hour to go over yesterday’s returned Information Sheets
2. First Assignment – without helping each other, go through and uninstall the application from yesterday. Gather information on this and ask yourself questions about it. Do I like the uninstall process? Why or why not? I set them to task to investigate and gather information on it. This was a timed 30 minute session.
3. Day two’s five minute conversation was about the constant changes and challenges in testing. Being able to switch from one task to the next without too much time in between is essential in our department.
4. Discussed the uninstall process of the product – this particular product has an ugly uninstall process and leaves too much behind. I wanted to see if anyone found it all. No one did, but I showed them where everything was and explained why this product did not meet our company’s standards for uninstall, that this was a bug and there are plans to fix it.
5. History: I presented them with product two that is in the group of products that I am responsible for and gave them the history on it.
6. Mission: Same as Day One
7. Oracles: I gave them the definition of an oracle and presented them with three of them. User guide, help files, and marketing material.
8. Questions and Answers
9. Tools: Something to present your information on. Decide this for yourself.
10. Task: Email the information to me at the end of the day.
Once again, I popped in and out of their office to make myself available for questions and to add little bits of information to the task. The same person who found a viable bug in Product One/Day One, found a bug in this Product Two/Day Two.
When I received the emails at the end of the day, only two of the temps had sent me their Information Sheets. (Not sure what happened to the third and I hate to make assumptions.) This time I gave them each other’s sheets to check out. This sets out to instill the “constant change” and to set up for the Day Three conversation.
Day Three:
1. Went into their office after giving them a half hour to review each other’s Information Sheets
2. Day Three’s five minute conversation was on sharing information. After acknowledging to them that I am aware that they may feel that they are in a competition for the possible permanent position at the end of the temp season, I informed them that their competitive edge is what they do with what they learn and that most of what they will need for this edge will take place on their own time, not within the office that they share. I spoke with them about the necessity of sharing information and used the example of my father to drive home the point (Why Share). I discussed how they would need to become reliant upon each other and build a bit of a mini-team within the team. Information sharing and teamwork are essential for success.
3. First task of the day was a 30 minute session on uninstalling the product from Day Two and to gather the information about it together. We then discussed what they found out about it.
4. History: I gave them the history of the product that I was having them look at on Day Three. Being part of the test team for this one from the ground up allowed me to explain what it was like to test a product with little documentation and how the project people had to focus on getting the product/application changes implemented, tested, and out the door. This product/application is much more complex than the other two that the temps had installed. It had scared off a few experienced testers when it originally came in house.
5. Mission: Same as Day One, only by all means help each other out.
6. Oracles: Same as Day Two.
7. Questions and Answers
8. Tools: The template that our team uses when conducting Exploratory Testing.
9. Task: email me the ET Sheets at the end of the day.
That was Friday. When I briefly glanced at their ET Sheets before heading out for the weekend, I was pleased with the results. Each one has their own style of writing, but each sheet was more focused on information and contained less “filler” material.
While they are not being utilized on another project (which is subject to change while I type); I will continue to share information with them. The above is a brief summary of what was brought to them in the past week, but within each “lesson” there were other “embedded lessons” as well. I have not planned out the “lessons” but looked for what was lacking or not understood in each previous lesson in order to drive the next.
I have no idea if what I am doing is going to produce any benefits to them, to the team, or to the stakeholders. Time will tell.
Sunday, March 15, 2009
The Perfect Storm, Part II
Wow, a lot can change in a few weeks. Evidently, more people on the project realized the extent of the problems and some things have been changed.
Most importantly, Point of Contacts were created for the applications/products/systems. In the case of the primary project that I am responsible for testing, I now report to an on-site individual with a background in programming. This has made a significant, positive impact on the project in very little time. No more trying to explain the implications of a bug. The POC has an interest in the project success because it reflects upon himself. Pride/ownership is now instilled in the project at the next level. There are now three of us, in two geographic locations, driving the test effort. All three are testing differently. One is primarily unit testing, one is calculation driven, and one is program/system testing. Usability, functionality, and integration are being tested by all three of us, but each with a different approach. We share information and the desire to see the product/application become a success. This simple change has enabled me to test more deeply, provide more information, and enjoy doing so without the added tension that was in the project before.
Now, here is where the story gets funny. My first meeting with the POC was in order for him to see what “QA” was responsible for and what coverage we were providing. I had written some regression test cases for the Automation team to implement. These were run through one build. While in that meeting, it was made apparent to me that even these were going to be rendered useless. A large part of the program has been changed. Along with the large change, several smaller usability issues were finally addressed as well. By the time the meeting was over, there were two test cases left that automation could run for the regression testing! I do find this quite humorous. First because I hate writing/running scripted test cases, and secondly because my heart belongs to Exploratory Testing.
I was also able to discuss with him my concerns on the project, up to an including the people problems and the problems with development not providing any information about what was done to fix a bug. (When a bug was sent back to be retested, all it said was “completed”. Now, seriously, what does that mean? What was changed? When I get bugs backed from developers with no information, it “bugs” me.) He has promised to get these issues to a place where they are not issues any longer.
The major program/application changes will be continuing until at least the August 2009 release. In fact, that particular release will hold the most risk due to the changes that are pending for it. I look forward to seeing the program/application develop. I look forward to testing the changes. I look forward to seeing the positive customer feedback due to the changes. I can work with this team. And because we share the same common goal, it is much easier for me to do my job and enjoy it as well.
The two other applications now have a POC as well. This is an unknown variable until after the March release, but I can now look ahead with an optimistic attitude to the changes that will come about. My one problem, which is mine alone, with the other two products is that I cannot seem to stick within the boundaries that I have been given. I am supposed to only focus on the direct integration portion of the products, but I cannot help but explore past the boundaries that have been given to me. Sometimes this affects how other testers relate to me. Crossing over my boundaries heads directly into their “territory”. I have been reminded of my boundaries by management, but not prohibited from pursuing my course (I argue well). I take this as a “go ahead to do it, but not with my permission” attitude. I have explained that I cannot help myself. I only hope that eventually the other tester who is the lead of the project will see that I do not want his project, but I do want the customers to be satisfied. I hope he will see that we are on the same side and that an “extra set of eyes” is a good thing. And I am respectful of the lead testers in all the projects that I test on. I provide them with information and only go around them if an issue is such that it affects another project.
Time will tell how things go with the other two applications/products. One thing I do know, when I pointed out the issues that were outside of the scope of testing the product/application, this information was just as helpful to management as the information on the project. The powers that be saw the project in a different light, took into account what I was frustrated with, and changed things. These are “people that matter” (James Bach) to the project. Everyone really wants success over failure and reporting the issues with the processes (even if they are people related) can be just as important as providing information about the product/application/system in test.
Most importantly, Point of Contacts were created for the applications/products/systems. In the case of the primary project that I am responsible for testing, I now report to an on-site individual with a background in programming. This has made a significant, positive impact on the project in very little time. No more trying to explain the implications of a bug. The POC has an interest in the project success because it reflects upon himself. Pride/ownership is now instilled in the project at the next level. There are now three of us, in two geographic locations, driving the test effort. All three are testing differently. One is primarily unit testing, one is calculation driven, and one is program/system testing. Usability, functionality, and integration are being tested by all three of us, but each with a different approach. We share information and the desire to see the product/application become a success. This simple change has enabled me to test more deeply, provide more information, and enjoy doing so without the added tension that was in the project before.
Now, here is where the story gets funny. My first meeting with the POC was in order for him to see what “QA” was responsible for and what coverage we were providing. I had written some regression test cases for the Automation team to implement. These were run through one build. While in that meeting, it was made apparent to me that even these were going to be rendered useless. A large part of the program has been changed. Along with the large change, several smaller usability issues were finally addressed as well. By the time the meeting was over, there were two test cases left that automation could run for the regression testing! I do find this quite humorous. First because I hate writing/running scripted test cases, and secondly because my heart belongs to Exploratory Testing.
I was also able to discuss with him my concerns on the project, up to an including the people problems and the problems with development not providing any information about what was done to fix a bug. (When a bug was sent back to be retested, all it said was “completed”. Now, seriously, what does that mean? What was changed? When I get bugs backed from developers with no information, it “bugs” me.) He has promised to get these issues to a place where they are not issues any longer.
The major program/application changes will be continuing until at least the August 2009 release. In fact, that particular release will hold the most risk due to the changes that are pending for it. I look forward to seeing the program/application develop. I look forward to testing the changes. I look forward to seeing the positive customer feedback due to the changes. I can work with this team. And because we share the same common goal, it is much easier for me to do my job and enjoy it as well.
The two other applications now have a POC as well. This is an unknown variable until after the March release, but I can now look ahead with an optimistic attitude to the changes that will come about. My one problem, which is mine alone, with the other two products is that I cannot seem to stick within the boundaries that I have been given. I am supposed to only focus on the direct integration portion of the products, but I cannot help but explore past the boundaries that have been given to me. Sometimes this affects how other testers relate to me. Crossing over my boundaries heads directly into their “territory”. I have been reminded of my boundaries by management, but not prohibited from pursuing my course (I argue well). I take this as a “go ahead to do it, but not with my permission” attitude. I have explained that I cannot help myself. I only hope that eventually the other tester who is the lead of the project will see that I do not want his project, but I do want the customers to be satisfied. I hope he will see that we are on the same side and that an “extra set of eyes” is a good thing. And I am respectful of the lead testers in all the projects that I test on. I provide them with information and only go around them if an issue is such that it affects another project.
Time will tell how things go with the other two applications/products. One thing I do know, when I pointed out the issues that were outside of the scope of testing the product/application, this information was just as helpful to management as the information on the project. The powers that be saw the project in a different light, took into account what I was frustrated with, and changed things. These are “people that matter” (James Bach) to the project. Everyone really wants success over failure and reporting the issues with the processes (even if they are people related) can be just as important as providing information about the product/application/system in test.
Subscribe to:
Posts (Atom)