Saturday, December 12, 2015

Creating a Custom Tool to Analyze Q Sort Data

December is finally here. It's been a wonderful semester at the University of Georgia. They seem to get better the longer I'm here. Although I haven't written many posts lately, I have been doing quite a bit of work with LiveCode. In particular, I've done much more work on my Q sort tool. In case anyone wants some background, here are links to my previous three posts about my Q sort tool:
  1. Creating a Q Sort with LiveCode
  2. Lloyd's Q Sort Project: Importing Data from an Internet File
  3. Latest on My Q Sort Prototype: Enhancing the User Experience and Inventing an Instructional Strategy
In short, a Q sort is a quantitative way to measure subjectivity. I know, that sounds completely contradictory. Think of it as a ranking procedure with a few twists. It's a procedure that's been around since the 1930s. One of my recent accomplishments is creating a much improved Q sort tool. The design improvements have been significant enough to warrant a full step in the version number - I'm now up to version 3.1. However, this post is not about that. Instead, it's about creating a completely new tool that focuses on analyzing the data that results from the Q sort activity.

As I've explained in previous posts, I've been trying to come up with an instructional strategy using a Q sort as the main class activity. In my early field trials, I really struggled to provide the class participants with the sorting results as quickly as possible. The reason is that I feel that the Q sort activity promotes a very active, "minds on" experience for participants. In order to take full advantage of their thinking and engagement, I need to compile, analyze, and report to the class participants the results of the Q sort activity as soon after they complete it as possible. In this post, I'll explain why that was such a challenge and what I've built to make this task much, much easier.

The Q Sort Raw Data

When a person completes a Q sort activity, a data string is created and uploaded to a text file stored on the Internet. For example, here is a sample based on a recent Q sort activity completed by an undergraduate class on the topic of favorite vacation destinations using the prompt "Sort these vacation destinations from most to least favorite." Here is the list of vacation destinations they sorted:
Aspen, Colorado
Branson, Missouri
Cancun, Mexico
Myrtle Beach
New York City
San Francisco
I found a web site that listed 10 of the top vacation spots in the United States and I added the rest. (Yes, adding Pittsburgh was rather mischevious of me. But hey, I think it's a fantastic place to spend one's vacation. I also added "home," which I thought was rather insightful.) A lot of the students never heard of Branson, Missouri, which might be a good thing.

Each line of data is an individual's data separated by commas. Here are a few lines of the data to illustrate:

 vacation-abd,Tue, 1 Dec 2015 12:56:26 -0500,Favorite Vacation Destinations?,PERSON1,0,-2,+1,+3,0,+1,0,-1,-1,-3,+2,-1,+1,0,Summary Statement Results,2,Time (seconds),59  
 vacation-abd,Tue, 1 Dec 2015 12:56:31 -0500,Favorite Vacation Destinations?,PERSON2,+1,-1,0,+3,-3,+1,0,-1,0,-2,0,-1,+2,+1,Summary Statement Results,+3,Time (seconds),78  
 vacation-abd,Tue, 1 Dec 2015 12:57:51 -0500,Favorite Vacation Destinations?,PERSON3,+3,-3,+1,+2,0,+1,+1,-1,-1,-1,0,-2,0,0,Summary Statement Results,+3,Time (seconds),131  
 vacation-abd,Tue, 1 Dec 2015 12:57:51 -0500,Favorite Vacation Destinations?,PERSON4,0,-3,-1,+1,0,+3,0,-1,0,+1,+2,-2,+1,-1,Summary Statement Results,+3,Time (seconds),159  

Each line starts with the unique Q sort code, then a date/time stamp, then the name of the Q sort, then the name of the person (I obviously substituted PERSON for each name). This is all followed by the sorting data where each statement's rating is provided. I also ask participants to rate a "summary statement," something I explain briefly below, so those results are next. Finally, I collected the time it took the person to complete the Q sort (in seconds).

OK, great, I have collected data. How does one quickly make sense of these data quickly and accurately in order to trigger some class discussion?

My First Idea: Use Excel to Analyze the Data

First, it is important to recognize that the analysis I wanted to perform was very simple in comparison to a true Q sort analysis. An actual Q sort analysis is best thought of as an upside-down factor analysis, meaning that instead of reducing the number of measures (i.e. statements) down to a smaller number of common factors, the idea is to reduce the number of people into a smaller number of profiles comprising those people. This type of analysis can take weeks and requires a sophisticated statistical package to pull it off. No, what I wanted was a straightforward analysis that I could do quickly, with results that the students and I could understand and react to. The goal of the analysis is just to trigger discussion and more reflection. That is, this is a learning goal, not a research goal. So far, I've settled on computing the sum and standard deviation for each statement based on all of the participant scores. The sum gives an overall sense of the importance of each statement for this group of people in comparison to the others. The standard deviation gives a quick sense of how much the group "agrees" with the ranking of that statement.

My basic plan is for participants to have a short small-group discussion in class - about 10 minutes - about their Q sort results immediately after completing it while I quickly analyze the data and prepare some slides of the overall results. That's a tall order in 10 minutes. And yes, my first idea was to use Excel to analyze the data. This is actually a good idea, if there was more time. I was able to successfully create an excel file on the fly that would analyze the data, but it was a challenge to do so quickly without making any mistakes. It's hard to explain what it feels like to be in the "heat of the moment" when teaching, but it can be stressful to try to focus on a task such as this requiring attention to detail. Such an Excel file begins with the following:

It's easy to get the data into neat and tidy columns using the "Text to Columns" option in Excel. This option allows you to split a line of data into columns based on some delimiter, such as the handy dandy comma. As you can see, I deleted a bunch of the columns to just focus on the people and their statement ratings. I summed the results for each statement. I also computed the standard deviation for each statement (again, as a rough measure of "agreement"). This is all well and good, but I really need the data in this form:

Fortunately, Excel has a transpose option. You first copy the data cells, then you choose "Paste special." Transpose will be one of the options. I then added a column and pasted in the statement labels. Then I sorted the rows in order of sum (largest to smallest). I did the same thing for the standard deviation, though I sorted that data from smallest to largest so that the statements the participants were "most agreement with" are at the top.

So, sure, I could do this in Excel, but allow me to repeat: Doing this in about 10 minutes without making a mistake is quite a challenge. And, if I wanted to do several Q sorts in a single class session, I would really be struggling. I'm good, but I'm not that good. I played around with the idea of creating an Excel template to facilitate the process, but that didn't work so well. What I wanted was a one-click solution.

Creating a One-Click Q Sort Analysis Tool with LiveCode

To meet these challenges, I created yet another LiveCode project that takes the Q sort data and produces the results quickly as described above. Here's the main screen (which uses all of the undergraduate's responses):

All I need to do is paste in the raw data into the large field on the left and the statements on the right. The bottom two left fields give me to double-check the data to be sure everything is ready for the analysis. For example, if all is good, a person's ratings will sum to 0. (I have noticed that data is sometimes missing, such as one of a person's statement ratings. Other quirky errors in the data have happened too, such as a person's data being duplicated. I'm not really sure why this is happening. Fortunately, it's a relatively rare occurrence, and I can usually make the needed corrections. Still, it's a cause of some concern.) I also drew that long arrow from the right side of the screen to the left to remind me to check that the raw data is being parsed correctly. I built in an auto-refresh feature so that after the mouse leaves each field after pasting, the key data fields update automatically.

If all is good, I simply click the red button "Analyze" and voilĂ ! Here is the output:

The data are all nicely computed and sorted just the way I want.

If I now click on the button "Copy All to Clipboard" I can then paste the data into Excel and use the "Text to Columns" option to produce the following formatted spreadsheet:

This is more than adequate, but a few more edits within Excel will produce the following output:

I usually just project this final spreadsheet on the large screen for students to review. But, I could easily copy and paste this into a PowerPoint slide as well.

The main point to all of this is that I can go from raw data to PowerPoint in about 2 minutes without needing any intense concentration to do so. While this isn't exactly a one-click solution, it's pretty close!

I should explain the "Summary Statement Average" of 2.41. After participants complete the Q sort, I ask them to rate one more question. In this case, the statement was "It is important to go on a real vacation at least once a year." I used the same rating scale as that used in the Q sort, which in this case ranged from -3 to +3 (a seven point scale). A summary rating of 2.41 clearly indicates that this group believes in the idea of yearly vacations. This summary statement gives me a Likert-like item that I'm considering using as a kind of weight to modify the raw data. These results are shown above under the orange headings ("adjusted"). I have some research questions related to this. Perhaps I'll explain more about that later.

Are You Like Me?

You might have noticed another button just below the Analyze button titled "Are You Like Me?" This is rather interesting and I will explain it further in a follow-up blog posting. Basically, this button will produce an analysis where each person is compared to the rest of the group. I think this has much potential for both instruction and research. It leads to some very interesting results that really seem to grab the participants' attention and interest.

Final Thoughts

I really wasn't wanting to create this Q Sort Analysis Tool, but it was needed and I'm obviously now glad it's done. It makes the analysis process extremely easy for me. And, even though I compute some simple statistics (i.e. sum and standard deviation), it would be very easy to add other statistics as well, should they prove necessary.

So, how did these undergraduates sort this list of favorite vacation destinations? As you probably already noted, Hawaii topped the list with very little disagreement among the group. Somewhat surprisingly, London was next (I would have bet on Paris). I find it very interesting that "Home" garnered the most disagreement and this result could be used to initiate some interesting discussion. Alas, Pittsburgh was second to last and there seemed to be little disagreement about this among the students. I'll feel sorry for them the next time I'm enjoying a delicious fish sandwich with pierogies on the side at Cupka's Cafe on Pittsburgh's southside. But, Pittsburgh did rank higher than Branson, Missouri. Perhaps "Thank God for Branson, MO!" should become Pittsburgh's new motto.


  1. Its as if you had a great grasp on the subject matter, but you forgot to include your readers. Perhaps you should think about this from more than one angle.

    Digital marketing course mumbai

  2. Its as if you had a great grasp on the subject matter, but you forgot to include your readers. Perhaps you should think about this from more than one angle.

    artificial Intelligence course

    machine learning courses in mumbai

  3. After reading your article I was amazed. I know that you explain it very well. And I hope that other readers will also experience how I feel after reading your article.
    360Digitmg digital marketing training in hyderabad

  4. I wanted to leave a little comment to support you and wish you a good continuation. Wishing you the best of luck for all your blogging efforts. Best Data Science training in Mumbai

    Data Science training in Mumbai

  5. Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.salesforce training in chennai

    software testing training in chennai

    robotic process automation rpa training in chennai

    blockchain training in chennai

    devops training in chennai

  6. I like this post,And I figure that they making some incredible memories to scrutinize this post,they may take a good site to make an information,thanks for sharing it to me
    data science malaysia

  7. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me..
    data analytics courses delhi

  8. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me..

  9. Great blog,
    The demand for Digital Marketing is growing big every day. With more and more users coming online, the demand for digital marketing is expected to grow even further.
    Digital Marketing Course with Internship at digital brolly.

  10. very detailed informative article.keep up the good work.Angular training in Chennai

  11. Thanks for sharing this blog with us. Good content and informative content. Keep sharing more.
    Machine Learning Training with Placements

  12. Very nice job... Thanks for sharing this amazing and educative blog post!
    data science training in malaysia

  13. Thanks for the informative and helpful post, obviously in your blog everything is good.
    Digital Marketing Courses in Mumbai

  14. I want you to thank for your time of this wonderful read!!! I definitely enjoy every little bit of it and I have you bookmarked to check out new stuff of your blog a must read blog!
    data analytics course in hyderabad

  15. Informative blog and knowledgeable content. Keep sharing more blogs with us. Thank you. If you want to become a data scientist, then follow the below link.
    Data Science Training and Placements in Hyderabad

  16. You completed certain reliable points there. I did a search on the subject and found nearly all people will agree with your blog.
    data science course in hyderabad

  17. Actually I read it yesterday but I had some thoughts about it and today I wanted to read it again because it is very well written. 360DigiTMG data science course in jaipur

  18. I have read many data science posts online previously, but none has managed to captivate my attention like this one. This is truly a masterpiece, and a perfect guide for all data science aspirants. Thanks to the writer for spelling out the concepts clearly, and using just the right words and data science institute in nashik with placement

  19. This post demonstrates an impressive application of LiveCode for Q sort data analysis, simplifying a complex task into a user-friendly tool. The author's dedication to enhancing classroom engagement and research efficiency is commendable.
    Data Analytics Courses in Nashik

  20. This blog post likely provides valuable insights into creating a custom tool for analyzing Q Sort data, a task essential for researchers and analysts in various fields. Q Sort methodology involves assessing and categorizing subjective data, making custom analysis tools valuable for efficient and accurate processing. The post is likely a valuable resource for professionals looking to streamline their Q Sort data analysis processes, offering guidance and practical tips. A must-read for researchers seeking to enhance their data analysis capabilities in this specific context.
    Data Analytics Courses in Delhi

  21. I really enjoyed reading your blog post on creating a custom tool to analyze Q-sort! It's fascinating to see how programming can be used to gain valuable insights and data from complex sorting methods like Q-sort. Data Analytics Courses In Pune

  22. Your blog post on developing a unique tool to analyse Q-sort was incredibly interesting to me! It's amazing to observe how programming can be utilised to extract insightful information and data from sophisticated sorting techniques like Q-sort.
    Data Analytics Courses in Agra

  23. nice blog
    Data Analytics Courses In Vadodara

  24. Thanks for such informative and amazing Content
    Digital Marketing Course in Mumbai

  25. The blog post gives incredible explanation and tutorial on how to create interactive software for education.
    data analyst courses in limerick

  26. Excellent information. The concept that was explained is very useful and also ideas are awesome, I really love to read such a wonderful article. Thank you for the information.
    Investment Banking courses in bangalore

  27. Wow, your innovative Q Sort Analysis Tool streamlines data analysis effortlessly! Impressive work, making complex tasks quick and efficient. Thanks for sharing!

    Investment Banking Industry

  28. "Diving into the realm of Q Sort data analysis feels like crafting a bespoke lens to unravel the intricate patterns of participant perspectives. Crafting a custom tool is not just about code; it's sculpting a dynamic bridge between raw data and meaningful insights. Your blog provides a roadmap for those ready to embark on the exciting journey of personalized analysis – turning data into a canvas for understanding."
    Investment banking as a career in India

  29. Thank you for sharing informative tutorial on Creating a Custom Tool to Analyze Q Sort Data.
    Investment banking training Programs

  30. Thank you for sharing fantastic tutorial on Creating a Custom Tool to Analyze Q Sort Data.
    Digital marketing courses in woking

  31. This post is so helpfull and interactive.I simply needed to thank you very much again. I am not sure what I would've achieved without the type of tricks documented by you directly on that area of interest.

    here is the website for Best Datascience course in kochi 2024