Today I am a very grateful software tester. I am grateful for the developer lead on the project that I have been testing on during this latest RC cycle.
Release time: T minus 6 hours.
Issue: Huge Vista related problem.
Resolved: Within two hours.
Reason: Developer worked with Tester to find the issue and resolve it.
After a longer than usual day of testing yesterday, I entered the building with a better brain than the night before. I ran through the remainder of a series of tests that I needed to run in Vista. I found something that had a true potential of being devastating. This particular release had already been promised to be on time and highly successful to everyone from the janitor to the CEO of the company. Can you feel the pressure?
I spent my up-to-30-minutes-allowed-during-an-RC- investigation time (I allotted this to myself because during an RC I believe breadth is more important than depth for coverage of applications of this magnitude). If I bypass my allotted time, I will ask for help. I did just that. I sent out a high priority email to all the other testers on my team and the key development characters in the release.
The developer who leads this project came in and immediately went through some scenarios with the issue. She set me to task on some of the other potential causes. Together we ended up with information on not just the original, potentially lethal defect, but several auxiliary ones as well.
The Release went out earlier than expected. The team that was assigned to the project went home earlier than expected. The customer will be happier than expected.
The gains in this project far outweighed the potential losses. We all, developers and testers, learned quite a bit more than was expected from this experience.
For this education I would personally like to thank Microsoft for all the wonderful issues they have created for us to explore in Vista.
Followers
Thursday, September 25, 2008
Sunday, September 21, 2008
"don't should on me"
I never take things at face value. I don’t believe what I am told unless there are solid examples to draw from and evidence to support them. Maybe this is due in part to the “J” in the INTJ that I am, or maybe it is just a by-product of being exposed to too many misleading television commercials and presidential elections. This “Judging” has benefited me as a software tester.
Here is a list of statements that I have heard in the past couple of years:
1. The state of the data should not matter; the process should be the same.
2. Changes in this feature should not affect any others.
3. The updates should not affect the processes.
4. We should be able to get this out the door on time.
5. The new build should have all the fixes in it.
6. Both products should play nice with each other.
7. This should work in all supported operating systems.
8. Testing this morph of the product should cover all others as well.
A substitute camp counselor, I met quite a few years ago, talked philosophically with me and my fellow campmates about the word “should”. The conversation was about how people are always saying what they should change or what they should do, though they rarely ever actually did any of the changing or doing. One statement he made in the conversation has stuck with me for all these years, “don’t should on me”. Because of this, I listen up whenever that word is used in a sentence. I actually worry a bit when it is used in conversation during a release cycle. It generally means late nights.
Since I have already interpreted this word to be a watch phrase in communication, what do I do when I hear it?
I have to analyze the situation. Is there any other information that I can gather to help me do this? If there is a change list available, that can be helpful. To be able to see what areas the code was changed in will enable me to make some decisions as to where I will test first. Product knowledge will help as well. If I know the product well enough I can project what may be affected by the changes. I can begin to jot down some “what if” questions for testing as well in order to evaluate both my thinking and the information that I have been able to gather. This is important in order to look for pieces of information that I may be missing.
At this point I will have some logical assumptions that I can follow through on in testing. I will have created reasons to test certain areas and I can justify why I have chosen those particular areas.
While testing I need to be on the lookout for anything that could change my reasoning. If this occurs I must be ready to change the course of action as well. I actively keep notes while I am testing this way and this is helpful in many ways. One of these ways is that it enables me to see where I faltered in my own analysis. This can expose some of my own biases and help me to look past them the next time around.
This process can happen very rapidly during a release cycle and I need to pay attention to what I am actually thinking. I also need to pay attention to the “how” and “why” of my thinking. I practice exercises like this in various ways continuously in exploratory testing. It keeps me alert and eager to see where it leads me because it evolves continuously. And it keeps me from being “should on”.
Here is a list of statements that I have heard in the past couple of years:
1. The state of the data should not matter; the process should be the same.
2. Changes in this feature should not affect any others.
3. The updates should not affect the processes.
4. We should be able to get this out the door on time.
5. The new build should have all the fixes in it.
6. Both products should play nice with each other.
7. This should work in all supported operating systems.
8. Testing this morph of the product should cover all others as well.
A substitute camp counselor, I met quite a few years ago, talked philosophically with me and my fellow campmates about the word “should”. The conversation was about how people are always saying what they should change or what they should do, though they rarely ever actually did any of the changing or doing. One statement he made in the conversation has stuck with me for all these years, “don’t should on me”. Because of this, I listen up whenever that word is used in a sentence. I actually worry a bit when it is used in conversation during a release cycle. It generally means late nights.
Since I have already interpreted this word to be a watch phrase in communication, what do I do when I hear it?
I have to analyze the situation. Is there any other information that I can gather to help me do this? If there is a change list available, that can be helpful. To be able to see what areas the code was changed in will enable me to make some decisions as to where I will test first. Product knowledge will help as well. If I know the product well enough I can project what may be affected by the changes. I can begin to jot down some “what if” questions for testing as well in order to evaluate both my thinking and the information that I have been able to gather. This is important in order to look for pieces of information that I may be missing.
At this point I will have some logical assumptions that I can follow through on in testing. I will have created reasons to test certain areas and I can justify why I have chosen those particular areas.
While testing I need to be on the lookout for anything that could change my reasoning. If this occurs I must be ready to change the course of action as well. I actively keep notes while I am testing this way and this is helpful in many ways. One of these ways is that it enables me to see where I faltered in my own analysis. This can expose some of my own biases and help me to look past them the next time around.
This process can happen very rapidly during a release cycle and I need to pay attention to what I am actually thinking. I also need to pay attention to the “how” and “why” of my thinking. I practice exercises like this in various ways continuously in exploratory testing. It keeps me alert and eager to see where it leads me because it evolves continuously. And it keeps me from being “should on”.
Sunday, September 14, 2008
This reminds me of when I was a waitress…
Years back I spent time working in a restaurant. I have a tendency to look for structure and patterns in just about everything I do and it did not take me long to figure out how to be successful as a waitress in the establishment that I was employed at.
As a software tester I have benefitted from this experience.
First there are the customers. They want their deliverables on time, they want to be pleased by the product, and they want to feel important enough to matter if there is an issue.
Then there are the cooks. They develop and execute the recipes. They take pride in their work and can be difficult to deal with.
The customers did not want to have inferior meals or service. The cooks wanted to create specialty dishes. Both of these groups determined my success. If the cooks did not prepare great meals, and the customers were not satisfied, my tips suffered.
At first glance it may appear that these two groups of people are extremely different. But they have the same basic commonalities and needs. Both groups want to feel important and respected. Both groups want to feel appreciated for what they do; one for creating, the other for paying.
Understanding that both groups determined my success helped me to learn how to forge relationships with people. The relationships that I created with the cooks were different from the ones that I created with the customers. Before opening the restaurant each evening, there is a setup time where only staff is in the building. I could have rushed through all my doings in short order. Instead I took a few moments each day to look over the product and ask a lot of questions. Most of the time customers did not want to know much about the food they were ordering, but on occasion someone wanted to know exactly what a dish contained. I needed this information from the cooks in order to inform the customers. I needed to understand the features and why a customer might want them.
The cooks had a certain pride in what they created. I was generally able to taste new creations which helped me to further inform the customers and create a bond with the cooks. This bond enabled me to make special requests from customers and have them met.
The customers wanted to feel as if they mattered. They wanted to get what they paid for and find the cost worthwhile. They want a good experience with a good product created by people who appreciate their business.
Being a software tester puts me pretty much in the middle of two groups once again. The developers and the customers determine my success in the company I work for. If either one is removed from the equation, so is my job. Seeking to understand both of these groups and creating positive relationships with both is vital.
Unlike restaurant work, I do not have the face to face contact with the customer, but I am provided information about them through customer service liaisons, marketing, and research. But I still must be able to advocate for them to the developers. In order to do this successfully I need to forge relationships with the developers. And I have found that most of the developers I have worked with have a lot in common with the cooks.
The products are a lot lighter than they were in the restaurant, and I spend way too much time off my feet these days (thank god for treadmills); but the people lessons are quite the same.
As a software tester I have benefitted from this experience.
First there are the customers. They want their deliverables on time, they want to be pleased by the product, and they want to feel important enough to matter if there is an issue.
Then there are the cooks. They develop and execute the recipes. They take pride in their work and can be difficult to deal with.
The customers did not want to have inferior meals or service. The cooks wanted to create specialty dishes. Both of these groups determined my success. If the cooks did not prepare great meals, and the customers were not satisfied, my tips suffered.
At first glance it may appear that these two groups of people are extremely different. But they have the same basic commonalities and needs. Both groups want to feel important and respected. Both groups want to feel appreciated for what they do; one for creating, the other for paying.
Understanding that both groups determined my success helped me to learn how to forge relationships with people. The relationships that I created with the cooks were different from the ones that I created with the customers. Before opening the restaurant each evening, there is a setup time where only staff is in the building. I could have rushed through all my doings in short order. Instead I took a few moments each day to look over the product and ask a lot of questions. Most of the time customers did not want to know much about the food they were ordering, but on occasion someone wanted to know exactly what a dish contained. I needed this information from the cooks in order to inform the customers. I needed to understand the features and why a customer might want them.
The cooks had a certain pride in what they created. I was generally able to taste new creations which helped me to further inform the customers and create a bond with the cooks. This bond enabled me to make special requests from customers and have them met.
The customers wanted to feel as if they mattered. They wanted to get what they paid for and find the cost worthwhile. They want a good experience with a good product created by people who appreciate their business.
Being a software tester puts me pretty much in the middle of two groups once again. The developers and the customers determine my success in the company I work for. If either one is removed from the equation, so is my job. Seeking to understand both of these groups and creating positive relationships with both is vital.
Unlike restaurant work, I do not have the face to face contact with the customer, but I am provided information about them through customer service liaisons, marketing, and research. But I still must be able to advocate for them to the developers. In order to do this successfully I need to forge relationships with the developers. And I have found that most of the developers I have worked with have a lot in common with the cooks.
The products are a lot lighter than they were in the restaurant, and I spend way too much time off my feet these days (thank god for treadmills); but the people lessons are quite the same.
Tuesday, September 9, 2008
Still dealing with bugs...
Sometimes when I am researching something online, I stumble across an unrelated, yet totally interesting topic.
Today I happened across an article titled Scientists find bugs that eat waste and excrete petrol.
Sounds gross doesn't it?
But what makes it extremely funny is that one of the project participants is a former software executive - and he is still dealing with bugs!
Today I happened across an article titled Scientists find bugs that eat waste and excrete petrol.
Sounds gross doesn't it?
But what makes it extremely funny is that one of the project participants is a former software executive - and he is still dealing with bugs!
Saturday, September 6, 2008
Applying the Model
There were several blogs that I read while researching OODA. Most of what I read was in regards to Agile Testing environments. Phil Kirkham pointed out several links to me in his comments from my last blog entry. I found Eject!Eject!Eject! to be the most informative. In expressing the need for change in the Pentagon to be more agile, it showed me that I cannot depend on my testing environment to be agile. Although I would prefer it to be so, it may not be. That does not prevent me from practicing agility in my personal testing efforts.
Here is what I have come up with to connect OODA with my Exploratory Testing. This is a rough draft for me to consider while testing.
Observation begins with Unfolding Circumstances - this could be the reason for the upcoming software release or update. This could be a new product in test. Outside Information - Specs, if available, or customer reported defects that resulted in the need for an update or change to the program. What were the customers doing to get the error? What is the "fix" for? What is the purpose of the new software?
Feed Forward
Orientation begins with Cultural Traditions - I can translate this to Company Traditions. I can use this in regards to the software life cycle that is used in the company that I work for. What does this life-cycle look like? Where does my testing fit into the cycle? What is expected from my department and how much time do I have?
Genetic Heritage - Is this a .NET application? What behavior is generally expected with .NET? What should I be looking for? What if users have previous or later versions of .NET installed?
Previous Experience - What defects have I generally found when such changes have been made? Does the developer have a disposition to forget something? Spell incorrectly?
New Information - Any new OS updates that could affect the software? New updates to the software in test? Change list available?
Analysis and Synthesis - (Understanding how I think is important for this to work.) The dictionary definitions of these two words are quite helpful in understanding what they entail. System Analysis is also helpful to remember in the generic use of the words: are the desired objectives being met? How is the procedural method working?
Feed Forward
Decision - What feature, path, or function should I focus on in this loop? Do I go with a positive or negative testing approach this time through?
Feed Forward
Action - Testing - plain and simple. Applying what I just acquired through the beginning of the strategic method.
From here I take the Feedback of what I have just learned and seen back to Observation. According to the model, I need to take two things back with me. Implicit Guidance and Control - What questions did I come up with during the Action? These can help guide me through the next loop. Unfolding Interaction with Environment - What changed in the system with (or since) my last Action? Data change? File changes? System Crash?
This is a rough draft, but I can already see how Boyd found this strategic model to be effective.
Friday, September 5, 2008
An Interesting Model To Test
I don’t watch much television, but on occasion I find myself hearing or seeing something that catches my attention. The other day I happened to be in earshot of a program that evidently involved a military investigation group. One of the cast members said, “Object, Orient, Decide, Act”. This caught my attention. The cast proceeded to speak about OODA. Not sure what that was, but quite curious, I jotted it down to research when I had the time.
What I found was a military strategic model designed by Col. John Boyd of the US Air Force called "The OODA Loop". When I saw it, I thought, this just might be a fascinating model for Exploratory Testing. Wikipedia has some information on the model as well as a decent image of it. It is a loop with an ongoing, many-sided, cross-referencing process. Action is equated with Test in the loop. It appears to have some interesting similarities with Exploratory Testing.
I happen to have a quote, attributed to General George Patton, hanging on the wall in my office: "A violently executed plan today, is better than a perfect plan next week." I think I will print off this model and hang it on the wall under the quote just to see what ideas I may get for testing.
Monday, September 1, 2008
I am good at what I do, but how do I know if what I am doing is good?
This question has been bothering me on and off for about six months now. It became even more complicated when I attempted to understand exactly what the next level is that I should be attempting to reach in my career as a software tester.
The official title of my occupation in the company that I am employed at is Quality Analyst. I want the title to reflect both what I do and who I am. I want to be a quality analyst as well as the person who analyzes the quality of the software.
After frustrating myself, and quite possibly others, I decided to look elsewhere for assessment. This was not an easy thing for me to do. I believe in self-assessment and I use it constantly. I like to challenge myself and I attempt to learn continuously through reading and then trying to apply what I have read/learned to the task at hand. But I am also biased and somewhat blinded. Sometimes I can be too easy on myself and sometimes too hard. I had to look for an alternative source.
I decided to sign up on the waiting list for the BBST Foundations course available through the Association for Software Testing (AST). I read through quite a bit of the information available on the site in regards to the course. I developed an intense fear of looking like a fool. I also developed a bit of a stomach ache. I had a choice to make. Allow the fear of failure to impede my progress or allow failure to continue to be an option. Allow myself to be assessed by an outside source that will enable me to understand what I need to work on and where I stand as a software tester or allow myself to be the chief assessor in spite of the obvious flaws that entails.
Ultimately I need to grow. I cannot grow if I do not have an accurate understanding of where I am right now. So, in spite of the fear of finding out that I am only good for the position that I am currently in, doing what I currently do for the company that I currently work for…. I signed up. I want to be the best that I can be. I cannot be the best that I can be if I do not know where my strengths and weaknesses lie.
I believe that by taking this course I will be able to begin to understand if “what I am doing is good”. And then, perhaps, I can get to the next step, “how can I do what I do better”.
The official title of my occupation in the company that I am employed at is Quality Analyst. I want the title to reflect both what I do and who I am. I want to be a quality analyst as well as the person who analyzes the quality of the software.
After frustrating myself, and quite possibly others, I decided to look elsewhere for assessment. This was not an easy thing for me to do. I believe in self-assessment and I use it constantly. I like to challenge myself and I attempt to learn continuously through reading and then trying to apply what I have read/learned to the task at hand. But I am also biased and somewhat blinded. Sometimes I can be too easy on myself and sometimes too hard. I had to look for an alternative source.
I decided to sign up on the waiting list for the BBST Foundations course available through the Association for Software Testing (AST). I read through quite a bit of the information available on the site in regards to the course. I developed an intense fear of looking like a fool. I also developed a bit of a stomach ache. I had a choice to make. Allow the fear of failure to impede my progress or allow failure to continue to be an option. Allow myself to be assessed by an outside source that will enable me to understand what I need to work on and where I stand as a software tester or allow myself to be the chief assessor in spite of the obvious flaws that entails.
Ultimately I need to grow. I cannot grow if I do not have an accurate understanding of where I am right now. So, in spite of the fear of finding out that I am only good for the position that I am currently in, doing what I currently do for the company that I currently work for…. I signed up. I want to be the best that I can be. I cannot be the best that I can be if I do not know where my strengths and weaknesses lie.
I believe that by taking this course I will be able to begin to understand if “what I am doing is good”. And then, perhaps, I can get to the next step, “how can I do what I do better”.
Subscribe to:
Posts (Atom)