I've been very derelict in providing updates on my project to create a digital tool to allow instructors to use Q sorts in their teaching. I suppose my only excuse is that I've been focused these past three months on actually doing the design work and coding to make it all possible. Well, I'm happy to announce that version 1.0 of the my app - Lloyd's Q Sort Tool - is now available. [Cue the dropping of balloons and confetti.]
Part of the task has been the monumental effort to also create a web portal for instructors to use. Although I had designed the mySQL database carefully over a year ago to allow for instructor accounts, I had never actually created anything more than a crude administrator interface for me to do the behinds-the-scene database tasks to support the field tests I have been running. And yes, this was a very ugly site. So, I've been working on developing an attractive and (hopefully) intuitive web portal for instructors while also making the final revisions of the app. Oh yeah, I was also teaching two courses for the University of Georgia this summer at the same time.
But, I am now ready to make instructor accounts available on a very limited basis. [Cue the triumphant horns.] It is really only the beta of version 1.0 that has been released because I feel it is necessary to do much field testing of the app before I'm confident that I have finally reached a true version 1.0. If you are one of the few who read this blog and one of the very few who might be interested in using Q sorts in your instruction, please feel free to go to my Q sort project's web site to request an account:
http://www.nowhereroad.com/qsort/
Of course, if you are not an instructor, but would like to check out how my Q sort tool works, I invite you to download the app and try one of the public Q sorts I've included on my web site.
Here is a video demonstration of how the app works:
I need to provide separate blog postings to spotlight and explain each of the big revisions to the app, but here is a quick summary of the most significant. Some of these represent true breakthroughs in my design.
For the first two years of the project, my design limited all Q sorts to no more than about 20 statements. Although this proved sufficient for all of my field tests, I knew that this limitation had to be overcome at some point. So, this past spring I mounted a design effort to find another approach. The solution came in the form of dynamically collapsing and expanding statements as they were sorted on the grid. This allowed me to switch to a more standard design of the Q sort board which resembled an inverted normal curve. I did keep the sideways orientation to the format of this board, which helped to leave the majority of the screen open for the statements. However, this approach triggered other user experience challenges and it was only through feedback from people who participated in some critical field tests that I was able to find a way for this to work.
It is standard practice within Q methodology to ask participants to first group all statements into three general piles: most agree, least agree, and neutral. Although my first prototypes advised people to do this, it never really worked well given the way the screen was organized. I found a way to persuade users to do this preliminary grouping which provided a solution to the limited screen space. Along the way, I think I found a unique design approach to doing a preliminary grouping. I added three long strips at the top of the screen, marked respectively as high, neutral, and low. Participants are advised to take each statement and do a "gut sorting" of it into one of these three groups. The unique design is that the placement of each statement, from left to right, on each of the three strips, gives a priority ranking to the statements in that strip. I have never seen this added dimension in any of the Q sort designs - paper or digital - that have been developed. I then added a 1-click option for participants to move the statements in any of the three strips into the main area of the screen in an expanded form and in the same order in which they appeared in the strip. The important thing to note about this strategy is that the initial grouping effectively removes all of the statements from the main area of the screen, thus opening this space for the statements to appear when when the 1-click option is used. In this way, the limited screen space is effectively used even if a large number of total statements is used in the Q sort. I also programmed an auto-magnify feature to immediately show the full statement when the participant mouses over any of the statement numbers in the sandbox.
I also programmed the list of statements to be auto-numbered. This provides a secondary cue to which statement is which. I combined this with the option to have a pop-up window to list all of the statements. Furthermore, this list can be copied to the clipboard for pasting in another application, such as a word processing document, to allow the participant easy access to all of the statements along with the statement number. This helps participants to cross-reference statements to their numbers.
I made a key decision about a year ago to put the analysis directly into the app itself. Before then, the analysis was done with a separate app I built meant just for the instructor. This decision put the analysis data directly in the hands of students with real-time access to the data. That is, the student could check to see how many responses had been submitted and run an analysis on the responses submitted so far.
More recently, I extended the analysis by adopting a method very much aligned with Q methodology. Although I have long had an analysis option called "Are You Like Me?" which was based on difference scores calculated between each pairs of people who completed the Q sort, this never proved to be an effective means of getting people to engage with each other. This summer I programmed this option to also provide correlation coefficients along with noting which were statistically significant. Although I have yet to field test this revision, I'm optimistic that it will make people more likely to both talk to those who share their views and talk to those people who do not.
Creating the instructor web portal was a challenge, not so much in terms of programming it, but in designing it to sufficiently explain how everything works to an instructor who has never met me or has never heard of a Q sort. Fortunately, the instructors to whom I've already given our accounts are people who heard me speak at various conferences about this project and asked permission to use the app in their teaching. I had to kindly tell them that the project was not yet "ready for prime time" and that I could not yet honor their requests. But, I added their names to a list of interested folks.
I provide some extensive guidance to instructors on how to use the site. I also created the following video demonstration:
There are some key concepts that, if not well understood, will make everything else confusing for instructors. The main one is probably the difference between a Q sort definition and Q sort activities. I won't give the long answer here, but the difference has allowed me to take a smart approach to creating Q sorts in a way that should save instructors lots of time. In short, one first creates a definition of a Q sort that contains almost all of the information about the Q sort. Then, one creates one or more activities that are linked, or bound, to that one definition. In this way, one use and reuse a Q sort definition dozens or even hundreds of time without have to go through the time-consuming task of creating yet another Q sort.
It's been almost two and a half years since I first wrote about my decision to create a digital version of a Q sort in April 2015. After hundreds of hours of coding and more than a dozen field tests, it feels so good to finally put this project "out there." Yes, I have built it, but will anyone come. To be honest, I only want a small number to request an instructor account so that I can identify and fix errors and problems which are inevitably present. My hope is that eventually at least a small community of instructors will form who are interested in exploring ways to capitalize on students' subjective perspectives in teaching. I have charted an initial strategy for their use, but I know there are many other creative approaches yet to be identified.
I look forward to seeing how the project unfolds. I plan on presenting this project at upcoming conferences, beginning with the small (but wonderful) IDD@UGA conference on August 19, 2017. But, I will be presenting this project at the upcoming conference of the Association for Educational Communications and Technology (AECT) in Jacksonville, Florida in October. I also have submitted a proposal to present the project at the American Educational Research Association (AERA) in New York in April 2018. I also intend to submit a proposal to the Conference on Higher Education Pedagogy (CHEP) held each February 2018 at Virginia Tech to provide an update of the project.
I have spent a lot of time on this project. I hope it proves to be useful to teachers, instructors, and trainers.
Part of the task has been the monumental effort to also create a web portal for instructors to use. Although I had designed the mySQL database carefully over a year ago to allow for instructor accounts, I had never actually created anything more than a crude administrator interface for me to do the behinds-the-scene database tasks to support the field tests I have been running. And yes, this was a very ugly site. So, I've been working on developing an attractive and (hopefully) intuitive web portal for instructors while also making the final revisions of the app. Oh yeah, I was also teaching two courses for the University of Georgia this summer at the same time.
But, I am now ready to make instructor accounts available on a very limited basis. [Cue the triumphant horns.] It is really only the beta of version 1.0 that has been released because I feel it is necessary to do much field testing of the app before I'm confident that I have finally reached a true version 1.0. If you are one of the few who read this blog and one of the very few who might be interested in using Q sorts in your instruction, please feel free to go to my Q sort project's web site to request an account:
http://www.nowhereroad.com/qsort/
Of course, if you are not an instructor, but would like to check out how my Q sort tool works, I invite you to download the app and try one of the public Q sorts I've included on my web site.
Here is a video demonstration of how the app works:
Summary of the Most Significant Revisions to the App
I need to provide separate blog postings to spotlight and explain each of the big revisions to the app, but here is a quick summary of the most significant. Some of these represent true breakthroughs in my design.
Breaking Through the 20 Statement Limit
For the first two years of the project, my design limited all Q sorts to no more than about 20 statements. Although this proved sufficient for all of my field tests, I knew that this limitation had to be overcome at some point. So, this past spring I mounted a design effort to find another approach. The solution came in the form of dynamically collapsing and expanding statements as they were sorted on the grid. This allowed me to switch to a more standard design of the Q sort board which resembled an inverted normal curve. I did keep the sideways orientation to the format of this board, which helped to leave the majority of the screen open for the statements. However, this approach triggered other user experience challenges and it was only through feedback from people who participated in some critical field tests that I was able to find a way for this to work.
Creating a Sorting Sandbox to Encourage Initial Grouping of Statements
It is standard practice within Q methodology to ask participants to first group all statements into three general piles: most agree, least agree, and neutral. Although my first prototypes advised people to do this, it never really worked well given the way the screen was organized. I found a way to persuade users to do this preliminary grouping which provided a solution to the limited screen space. Along the way, I think I found a unique design approach to doing a preliminary grouping. I added three long strips at the top of the screen, marked respectively as high, neutral, and low. Participants are advised to take each statement and do a "gut sorting" of it into one of these three groups. The unique design is that the placement of each statement, from left to right, on each of the three strips, gives a priority ranking to the statements in that strip. I have never seen this added dimension in any of the Q sort designs - paper or digital - that have been developed. I then added a 1-click option for participants to move the statements in any of the three strips into the main area of the screen in an expanded form and in the same order in which they appeared in the strip. The important thing to note about this strategy is that the initial grouping effectively removes all of the statements from the main area of the screen, thus opening this space for the statements to appear when when the 1-click option is used. In this way, the limited screen space is effectively used even if a large number of total statements is used in the Q sort. I also programmed an auto-magnify feature to immediately show the full statement when the participant mouses over any of the statement numbers in the sandbox.
Auto-Numbering of the Statements
I also programmed the list of statements to be auto-numbered. This provides a secondary cue to which statement is which. I combined this with the option to have a pop-up window to list all of the statements. Furthermore, this list can be copied to the clipboard for pasting in another application, such as a word processing document, to allow the participant easy access to all of the statements along with the statement number. This helps participants to cross-reference statements to their numbers.
Improving and Extending the Q Sort Analysis
I made a key decision about a year ago to put the analysis directly into the app itself. Before then, the analysis was done with a separate app I built meant just for the instructor. This decision put the analysis data directly in the hands of students with real-time access to the data. That is, the student could check to see how many responses had been submitted and run an analysis on the responses submitted so far.
More recently, I extended the analysis by adopting a method very much aligned with Q methodology. Although I have long had an analysis option called "Are You Like Me?" which was based on difference scores calculated between each pairs of people who completed the Q sort, this never proved to be an effective means of getting people to engage with each other. This summer I programmed this option to also provide correlation coefficients along with noting which were statistically significant. Although I have yet to field test this revision, I'm optimistic that it will make people more likely to both talk to those who share their views and talk to those people who do not.
Overview of the Instructor Web Portal
Creating the instructor web portal was a challenge, not so much in terms of programming it, but in designing it to sufficiently explain how everything works to an instructor who has never met me or has never heard of a Q sort. Fortunately, the instructors to whom I've already given our accounts are people who heard me speak at various conferences about this project and asked permission to use the app in their teaching. I had to kindly tell them that the project was not yet "ready for prime time" and that I could not yet honor their requests. But, I added their names to a list of interested folks.
I provide some extensive guidance to instructors on how to use the site. I also created the following video demonstration:
The Difference Between Q Sort Definitions and Q Sort Activities
There are some key concepts that, if not well understood, will make everything else confusing for instructors. The main one is probably the difference between a Q sort definition and Q sort activities. I won't give the long answer here, but the difference has allowed me to take a smart approach to creating Q sorts in a way that should save instructors lots of time. In short, one first creates a definition of a Q sort that contains almost all of the information about the Q sort. Then, one creates one or more activities that are linked, or bound, to that one definition. In this way, one use and reuse a Q sort definition dozens or even hundreds of time without have to go through the time-consuming task of creating yet another Q sort.
Next Steps for the Project
It's been almost two and a half years since I first wrote about my decision to create a digital version of a Q sort in April 2015. After hundreds of hours of coding and more than a dozen field tests, it feels so good to finally put this project "out there." Yes, I have built it, but will anyone come. To be honest, I only want a small number to request an instructor account so that I can identify and fix errors and problems which are inevitably present. My hope is that eventually at least a small community of instructors will form who are interested in exploring ways to capitalize on students' subjective perspectives in teaching. I have charted an initial strategy for their use, but I know there are many other creative approaches yet to be identified.
I look forward to seeing how the project unfolds. I plan on presenting this project at upcoming conferences, beginning with the small (but wonderful) IDD@UGA conference on August 19, 2017. But, I will be presenting this project at the upcoming conference of the Association for Educational Communications and Technology (AECT) in Jacksonville, Florida in October. I also have submitted a proposal to present the project at the American Educational Research Association (AERA) in New York in April 2018. I also intend to submit a proposal to the Conference on Higher Education Pedagogy (CHEP) held each February 2018 at Virginia Tech to provide an update of the project.
I have spent a lot of time on this project. I hope it proves to be useful to teachers, instructors, and trainers.
No comments:
Post a Comment