Lloyd's Q Sort Tool is Now a Viable Option for Q Researchers

It is going on three years since I began creating a digital Q sort tool. Over the past few weeks, I've been on a very "deep dive" into this project thanks to having a research leave this semester - my first leave of any kind in 30 years. My goals this semester have been to finalize the app's initial development, create a web site for other teachers to use my tool at no cost, do a lot of reading about Q methodology, write about its development, and submit a grant proposal for some modest funding to keep the work going.

The three readers of this blog know that I actually released beta version 1.0 back in August. I used the term "beta" because this was a preliminary and cautious release. Anyone interested in getting an instructor account to use my tool needed my approval. It's not that I was expecting thousands of people to start using it, but I deliberately wanted to keep the number of users very small in order to have them help to spot problems and bugs, while also suggesting ideas for further development. As an unfunded project with me as chief designer,  programmer, web developer, statistician, PR department, and marketer, there are likely to be a lot of as yet undetected errors. I didn't want too many people cursing my name from afar as the errors were revealed.

As it turned out, very few instructors have signed up to be users, which has been disappointing, even though I've made three presentations about this project since the release of version 1.0, including one at the annual conference of the Association for Educational Communications and Technology (AECT) in November and another at the Conference on Higher Education Pedagogy (CHEP) at Virginia Tech in February. I've also sent out two announcements about my app to the Q-Method listserv run out of Kent State University. I actually had quite a few inquiries from the Q-Method folks, but only about whether my app could be used for research projects. I explained that my main goal and purpose in developing the app has been for instructional, not research, applications. These folks wished me well, but obviously had little interest other than the app's potential for conducting Q methodology research.

Time For Me to Sit Up and Pay Attention

So, for the last month or so I've started to rethink the goals of my Q sort project with an eye toward expanding it to include the needs of Q researchers. Obviously, that is the group who has shown the most interest and encouragement. So I decided to go for the best of both worlds by making my Q sort tool viable for Q researchers and people like me who want to tap the power of Q for enhancing instruction. To that end, I've made some important updates to the tool, many in just the past week (thanks to this being spring break here at the University of Georgia). This set of revisions mark an important milestone in the ideation and iterations of the design, especially as I write a draft design article about this project for presentation at the upcoming AERA conference in New York.

Here is a short summary of those updates.

Easy Exporting of the Data into Q Analysis Statistical Tools

Even though I was careful to make the raw data of a Q sort completed with my app readily accessible to a researcher, it was, well, really raw. With the help of one of my doctoral students, Tong Li, I added an export option in the instructor version of my app. This option now makes it very easy to export the data to the popular Q analysis applications, such as PQMethod an Ken-Q. I've become a big fan of the Ken-Q Analysis Tool, mainly because it's a web site, not a standalone app (and it's free), so I made a quick video on how to export data from my app into it:

I don't try to explain anything about the analysis, but only to demonstrate to a Q researcher how easy it is to export the data.

Two Other Features that Q Researchers Have Asked About, and One They Didn't

Embedded End-of-Q Sort Survey

Q researchers have repeatedly asked about whether my app can conduct and collect some survey data after the Q sort has been completed. Asking the user to explain their high and low sort choices along with any other comments that help to explain their thinking is a typical procedure in Q methodology. My response has been no, but also that it shouldn't be a big deal to collect these data through other means, such as through a Qualtrics survey. But, I also know that having the user jump out to yet another application at the critical moment when they are asked to do the hard work of reflection runs a huge risk of the user just quitting. So, I've now made asking three open-ended questions a feature in my app as well, along with giving the researcher control over the wording of the three questions. Given my hunch that instructors who use my Q sort tool for teaching will not want to ask these three questions, this feature can be turned off when the Q sort definition is created.

Making the Summary Survey Question Optional

Interestingly, another feature of my app that I've included from the very beginning is one that Q researchers don't like. After a user completes the Q sort, my app asks them a simple summary survey question in the style of a Likert question. For example, after completing a Q sort about ice cream, the user is asked to rate the statement "I like ice cream." from strongly agree to strongly disagree. Asking a question like this has been very important and useful in using Q sorts for instruction because it lets a class know how the entire group feels about the topic in a single data point. This helps to start the conversation. Well, Q researchers really don't seem to like this. Even though I explain that this question comes after the Q sort and therefore doesn't interfere to invalidate the Q data, they just don't want to ask a question of this sort. So, I've now made the asking of this summary survey question optional when a Q sort definition is created.

Creating these two features - and the option to use or not use either in a Q sort - have extended my understanding of how to have a standalone app created with LiveCode communicate with a mySQL database. Indeed, a great deal of my work over the past week has been exclusively on the mySQL database and the PHP code on the web site I created months ago to support the tool.

Kiosk Mode

Interestingly, one of my colleagues who is interested in using my Q sort tool in her teaching matter of factly mentioned how she planned on setting up a computer or two in a corner of her classroom for students to complete the Q sort at some point during the class. The reason is that she didn't expect students to bring a laptop with them for downloading the app. I thought this was a great idea, but it quickly dawned on me as she walked away that this wasn't possible because my app doesn't allow someone to complete a Q sort a second time, except for practice without any data being submitted to the database. I solved this problem by adding a feature to the instructor version of my app that I call Kiosk mode. As the name suggests, you can run my app like a kiosk on a dedicated computer somewhere and have lots of people come by to complete the Q sort. There is the risk of one person completing a Q sort multiple times, so it will be up to the instructor (or researcher) to police this. And, given that this feature is only available in the instructor version of the app, there is no chance some "rogue participant" could use the regular version of the app to stuff the ballot box.

Immediate Next Step: Lots of Testing

Any time a new version is rolled out, there is a good chance that bugs and errors remain. Testing is a time-consuming endeavor and, as I already mentioned, my time is divided between all of the roles needed on the project. So, before I release the version of the app with all of the updates discussed here, I'll first be turning my attention over the week or so to the mundane task of creating lots of sample Q sorts while playing the role of both interested and uninterested participant, all in the mission of uncovering the bugs. If anyone wants to join me, just let me know.

Eventual Next Step: Posting a Note to the ITFORUM Listserv

One place I haven't disseminated information about my Q sort project is the ITFORUM listserv. This listserv remains one of the best ways to communicate to professors, researchers, designers, and students in the field of learning, design, and technology (aka instructional technology). I'm not sure why I've been hesitating. I think one reason is that I wanted the app to be very solid before announcing it there. But, time may be almost ripe to post something to the ITFORUM listserv, especially because it might spur a few people in my field to seriously consider using Q sorts in their teaching or Q methodology in their research.