This is a follow-up to my recent post describing a tool I created to analyze Q sorts. In that post I briefly called attention to a button on the analysis screen titled "Are You Like Me?":
In this post, I will explain how this special analysis works. Although this analysis option is specifically meant to support my goal of using the Q sort process within an instructional context, the idea behind it originated much earlier, and the analysis itself can be applied to other survey data types besides Q sorts. So, allow me to give a little background first.
Those in the field of instructional design like to think that one of its strongest elements is learner analysis. This is a major phase of instructional design and is well documented within all of the best known models. Even though we devote a lot of attention to learner analysis in the literature, my own opinion is that we don't practice it well. Yes, we are fond of giving pretests and surveys, but these usually only provide very superficial information about the learners for whom we are designing instructional materials. So, I feel that the theory of learner analysis falls far short of its reality and importance in practice. Perhaps I feel this way based on my formative years as a public school teacher. It was only after spending many months with my students for five or six hours per day did I begin to feel as if I really knew them. So, I tend to think we know very little about the people we are designing instruction for despite following the advice and procedures in our canons and even though we might feel comfortable in thinking that we do.
We also don't teach learner analysis very well, at least, I don't think I do. I've long been trying to come up some innovative activities to help my instructional design students recognize both the importance of learner analysis and the difficulty in doing it well. At the very least, I want to instill in them the idea that we should be very cautious and skeptical about thinking that we really understand who our learners are, what they know, and what they want. So, I've been trying to design some class activities over the past few years that get at some of these deeper principles. I've yet to succeed. But a few years ago I came up with a class activity called "Are You Like Lloyd?" that seemed to hold some promise. I asked a series of questions to see how many students in the class had similar life experiences to me. For example, one question was "Do you have at least three siblings?" I thought this was a good question because coming from a family of five, I feel certain that growing up with a few brothers and sisters will make a big difference in how you see the world. After asking about 10 questions like this, a nice discussion usually ensued. I always had the sense that I was on the verge of designing an interesting game based on this activity, but I could never quite figure out what the game's goal would be. Is it to ask the least amount of questions before I demonstrated that everyone in the room was different than me in some way? Heck, that would be easy for me just with the question "Do you play an accordion?" Maybe the goal should be to ask questions to show how much people shared. That seemed easy to do too and not very interesting (e.g. Do you like music? Do you like ice cream?) In the end, just asking some questions that I thought reflected important influences on how I saw the world to find out how many in the room were like or different than me was interesting.
The "Are You Like Lloyd?" activity is the inspiration for my current work with using Q sorts as instructional activity. I've been playing around with the idea of comparing each person in the group to every other person in the group in terms of their individual Q sort responses. The best way to explain this is with an example. Let's consider a very short and simple Q sort that has only four statements. The Q sort board ranges from least to most favorite with column values of -1, 0, and +2 with two slots in the 0 column:
And, to keep the example as concrete as possible, let's imagine these ice cream flavors are the four statements of the sorting activity:
In this post, I will explain how this special analysis works. Although this analysis option is specifically meant to support my goal of using the Q sort process within an instructional context, the idea behind it originated much earlier, and the analysis itself can be applied to other survey data types besides Q sorts. So, allow me to give a little background first.
Those in the field of instructional design like to think that one of its strongest elements is learner analysis. This is a major phase of instructional design and is well documented within all of the best known models. Even though we devote a lot of attention to learner analysis in the literature, my own opinion is that we don't practice it well. Yes, we are fond of giving pretests and surveys, but these usually only provide very superficial information about the learners for whom we are designing instructional materials. So, I feel that the theory of learner analysis falls far short of its reality and importance in practice. Perhaps I feel this way based on my formative years as a public school teacher. It was only after spending many months with my students for five or six hours per day did I begin to feel as if I really knew them. So, I tend to think we know very little about the people we are designing instruction for despite following the advice and procedures in our canons and even though we might feel comfortable in thinking that we do.
Doing a Better Job at Teaching Learning Analysis
We also don't teach learner analysis very well, at least, I don't think I do. I've long been trying to come up some innovative activities to help my instructional design students recognize both the importance of learner analysis and the difficulty in doing it well. At the very least, I want to instill in them the idea that we should be very cautious and skeptical about thinking that we really understand who our learners are, what they know, and what they want. So, I've been trying to design some class activities over the past few years that get at some of these deeper principles. I've yet to succeed. But a few years ago I came up with a class activity called "Are You Like Lloyd?" that seemed to hold some promise. I asked a series of questions to see how many students in the class had similar life experiences to me. For example, one question was "Do you have at least three siblings?" I thought this was a good question because coming from a family of five, I feel certain that growing up with a few brothers and sisters will make a big difference in how you see the world. After asking about 10 questions like this, a nice discussion usually ensued. I always had the sense that I was on the verge of designing an interesting game based on this activity, but I could never quite figure out what the game's goal would be. Is it to ask the least amount of questions before I demonstrated that everyone in the room was different than me in some way? Heck, that would be easy for me just with the question "Do you play an accordion?" Maybe the goal should be to ask questions to show how much people shared. That seemed easy to do too and not very interesting (e.g. Do you like music? Do you like ice cream?) In the end, just asking some questions that I thought reflected important influences on how I saw the world to find out how many in the room were like or different than me was interesting.
Analyzing a Q Sort: Person to Group Comparison
The "Are You Like Lloyd?" activity is the inspiration for my current work with using Q sorts as instructional activity. I've been playing around with the idea of comparing each person in the group to every other person in the group in terms of their individual Q sort responses. The best way to explain this is with an example. Let's consider a very short and simple Q sort that has only four statements. The Q sort board ranges from least to most favorite with column values of -1, 0, and +2 with two slots in the 0 column:
And, to keep the example as concrete as possible, let's imagine these ice cream flavors are the four statements of the sorting activity:
- Chocolate
- Vanilla
- Strawberry
- Peach
Let's imagine a group of five people sorted these four statements as follows:
Let's consider Bob's sorting of these flavors. He likes vanilla the most, chocolate the least, and is neutral about strawberry and peach.
The idea of the "Are You Like Me?" activity is to consider how similar Bob is to all of his classmates, then to do the same comparison for the other people, one by one. Let's stick with Bob for now and consider what this analysis looks like if we use Excel to do the comparisons. Here is the entire spreadsheet:
The absolute value of the difference between Bob's scores and those of the other people are shown in orange. Let's compare Bob to Jane. Bob rated chocolate as -1 (his least favorite), but Jane was neutral about chocolate (0), for a difference of 1. We use the absolute value because we are only concerned about the "distance" between the two scores. (The formula for cell B12 is: "=ABS(B$5-B6)".) We don't really care who has the higher or lower score for any particular flavor. Vanilla was the favorite flavor for both Bob and Jane, so their difference was 0. So, when it comes to vanilla, Bob and Jane are exactly alike. Bob and Jane were also exactly alike in their rating of peach, but they difference by 1 on their rating of strawberry. If we sum of the differences between Bob and Jane, as shown in blue, we get a sum of 2. Now, take a look at how different Bob is from Jim, Sarah, and Susan. If you scan the sum of differences you see that Bob and Sarah are exactly the same, whereas he is most dissimilar to Jim and Susan with sums of 4 for his comparisons to them. If we sum up the sum of differences - the "grand sum" shown in green - we get a rough idea of how different Bob is from the group on ice cream flavor preferences.
Automatizing the Difference Analyses Between All Group Members
Great, now what? Well, we just need to do the same analysis for all other members of this group. With a group of five, I'm sure you could take it from here and finish my Excel spreadsheet. But, it would be rather tedious to set it all up. And imagine if you had a group of 10, 20, or 100 people. For this reason, I decided to program LiveCode do the analysis for me. I set it up in such a way that it doesn't matter how many statements there are or how many people are in the group.
To demonstrate, I plugged the hypothetical data above into my LiveCode program, clicked on the "Are You Like Me?" button and here is the output:
The left side gives me an overall comparison summary. A quick click of the "Copy to Clipboard" button on the left allows me to paste the data here:
Participant,Sum of Comparisons
Bob,10
Sarah,10
Jane,12
Jim,16
Susan,16
As noted above, the group is arranged from low to high. Bob and Sarah have the fewest differences with their classmates, and Jim and Susan have the greatest differences.
A click of the other "Copy to Clipboard" button allows me to easily paste the results of all the comparisons below - this is actual text output complete with commas in key spots to allow for each formatting after pasting into Excel with the "Text to Columns" option (ordered low to high within each group):
Participant,Difference (Absolute Value),Note: The comparison person's name is in the first row of each block.
Bob
Sarah,0
Jane,2
Jim,4
Susan,4
SUM,10
Jane
Bob,2
Sarah,2
Jim,4
Susan,4
SUM,12
Jim
Bob,4
Jane,4
Sarah,4
Susan,4
SUM,16
Sarah
Bob,0
Jane,2
Jim,4
Susan,4
SUM,10
Susan
Bob,4
Jane,4
Jim,4
Sarah,4
SUM,16
Now, obviously, this dummy data doesn't yield very interesting results, but I hope you get the idea.
Is it better to be similar to everyone else, or different from everyone else? Neither. I think it's important not to impose any value-laden interpretation on the results. That is, it is not good or bad to be similar or different, but only to recognize that there are similarities and differences and to explore why. My hope is that doing this analysis right after a group completes a Q sort will stimulate some lively discussion with a slightly better understanding about each other. And, if these are people studying to become instructional designers, maybe it will give them a deeper understanding and appreciation about learner analysis.
Here's an interesting anecdote based on some early trials. I've tried this activity out with one of my doctoral classes in the fall 2015. The course was required by our majors, but it was open to nonmajors too. We only had one nonmajor take the course. He brought it to my attention in a hallway conversation during a break that he noticed he tended to be the person at the bottom of each comparison group. We both found it interesting that the sorting activities "revealed" him to have a different view or perspective about the statements being sorted than the others in the class.
There may be times that it is useful to group people who have similar or different points of view. Perhaps a certain project would benefit from having people who shared a similar point of view. Or, conversely, perhaps you wanted to maximize the diversity of ideas within a team of people as they worked on a project. It's important to remember that the original purpose of a Q sort activity is to identify a small number of profiles or categories of the people who complete the activity. That involves a lengthy and sophisticated analysis (that I like to refer to as an upside-down factor analysis) using a special statistical software program. Yet, comparing everyone's answers to each other person in the group as shown above seems to be the start to an alternative process for achieving an analysis with a similar goal. I think I need to do some further follow-up computations - yet invented - to tease out subgroups or profiles.
As I end this blog post, I want to reiterate that this analysis will work on any survey involving quantitative data, such as surveys based on the more familiar Likert scale. So, even if you are not interested in Q sorts, you might find the ideas behind this analysis intriguing and useful if you are someone who wants to know more about how a group of people tick.
Also, I really don't know if any of this work will yield anything particularly useful. In the end, I may simply be generating an overly-complicated way of conducting an icebreaker activity. Yet, there seems something inherently important and useful in it. I look forward to exploring this issue in my Q Sort research. So, we'll see.
What Does It All Mean?
Is it better to be similar to everyone else, or different from everyone else? Neither. I think it's important not to impose any value-laden interpretation on the results. That is, it is not good or bad to be similar or different, but only to recognize that there are similarities and differences and to explore why. My hope is that doing this analysis right after a group completes a Q sort will stimulate some lively discussion with a slightly better understanding about each other. And, if these are people studying to become instructional designers, maybe it will give them a deeper understanding and appreciation about learner analysis.
Here's an interesting anecdote based on some early trials. I've tried this activity out with one of my doctoral classes in the fall 2015. The course was required by our majors, but it was open to nonmajors too. We only had one nonmajor take the course. He brought it to my attention in a hallway conversation during a break that he noticed he tended to be the person at the bottom of each comparison group. We both found it interesting that the sorting activities "revealed" him to have a different view or perspective about the statements being sorted than the others in the class.
Final Thoughts
As I end this blog post, I want to reiterate that this analysis will work on any survey involving quantitative data, such as surveys based on the more familiar Likert scale. So, even if you are not interested in Q sorts, you might find the ideas behind this analysis intriguing and useful if you are someone who wants to know more about how a group of people tick.
Also, I really don't know if any of this work will yield anything particularly useful. In the end, I may simply be generating an overly-complicated way of conducting an icebreaker activity. Yet, there seems something inherently important and useful in it. I look forward to exploring this issue in my Q Sort research. So, we'll see.
No comments:
Post a Comment