LiveCode vs. Captivate and Storyline: Battle of the Multiple-Choice Question

Many of our students here at UGA learn Adobe Captivate or Articulate Storyline. Both are easy-to-learn tools that are good for what typically passes as eLearning. You know, present some information, then quiz people on it for practice, maybe provide some extra guidance, then test them, record the results, and move on. I know there is a place for this sort of thing and I realize there are those rare times when this is exactly the kind of instruction that is needed, but I find no inspiration in it. And, unfortunately, all too often this is the "default" instructional approach for so many designers. But, let us not fault the multiple-choice question itself for its overuse and abuse within eLearning. It can be a very effective design element when used creatively, an idea I'll revisit at the end of this blog posting.

One of the appealing things about Captivate or Articulate is the ease with which one can create quizzes. One can simply choose a question type and then fill in the various boxes. But, is it really that easy? I actually think this approach is very labor intensive, especially for multiple choice questions - the interactive darling of the eLearning world. After all, you have to go in and manually enter each part of the question - the stem and the all the choices - through the tool's quiz GUI. Although it's easy to do, I repeat that it's very labor intensive, especially when you want to go and modify the quiz questions. The ability or option to reuse quiz questions from one application to another is also an important consideration. Captivate allows for the creation of question pools that can be shared by multiple projects. Storyline also allows for copying of content between applications, though it looks cumbersome to do so to me.

A better approach is to put all the questions into a simple text file and then import the questions into the quiz engine. With Captivate, this supposedly can be done using the GIFT format (here is another article and a PDF that explains the GIFT format). QTI, or Question and Test Interoperability, is another standard for creating quiz questions that can be imported into a variety of learning management systems (LMSs). It was developed by IMS. Similarly, Respondus is popular tool with higher education faculty who want to create question banks for use with their university's LMSs.

Unfortunately, Articulate Storyline does not support GIFT or any other method to import questions from a text file into its quiz tool. So, designers must enter the questions one by one within the Storyline interface. And, frankly, I doubt many Captivate authors know about or use the GIFT format. (To be honest, I did not know about either GIFT or QTI before now.)

Let's take a moment to look a little closer at the GIFT format because it is similar to the approach that I invented for my quiz tool. It is just a simple text file with certain symbols used to define the question and the answers (correct and incorrect). Here is the basic format (explained in a Wikipedia article):

//Comment line
::Question title
:: Question {
     =A correct answer
     ~Wrong answer1
     #A response to wrong answer1
     ~Wrong answer2
     #A response to wrong answer2
     ~Wrong answer3
     #A response to wrong answer3
     ~Wrong answer4
     #A response to wrong answer4
}

You would type the questions using this format, one after the other, in a text file.

Lloyd's Multiple-Choice Quiz App

That's a long-winded introduction to the fact that a few months ago I decided I would take a shot at creating a multiple-choice quiz generator that reads all of the questions from a simple text field. Here's a five-minute video of how the app works:

 LiveCode file

[ Get the free LiveCode Community version. ]

I started this one morning and just jumped into the programming with nothing other than my thoughts to guide me, a prototyping approach I love to use, and one that works well with LiveCode. I spent about an hour and made good progress. After I got home from work that day, I spent maybe another two hours on the prototype.

I returned to the prototype just the other day and decided to make one significant improvement to the original prototype - the option to randomize the answers for a particular question. This is a relatively simple thing to do and took only about an hour to program, yet it vastly improves the quiz environment in many ways. In traditional eLearning designs, it preempts the possibility that students will share specific answers to questions. In other designs, such as gaming, this feature dramatically extends the shelf-life of the question bank. Of course, some questions need to have the answers presented in a certain order, so I provided the means to override the randomization feature for individual questions. I'm thinking about adding the feature of randomizing the order in which the questions are presented. This also would not be hard to do.

This question app can consist of as many questions you want with up to 5 answers per question. The app begins by counting and displaying the total number of questions. Each question is displayed in order. If there are fewer than 5 answer choices, then only the buttons with choices are shown.

Feedback consists of "Knowledge of Correct Results," that is, if wrong, the person is given the correct answer. The quiz results, including the percentage correct, are updated and displayed after each question.

As I mentioned above, I did not know about the GIFT format before creating this tool, but it would not be hard to create a new version that uses the GIFT format. It would be also easy to create an app to assist a designers or teacher to create quizzes in the GIFT format. But I doubt there would be much of a market for it.

A little sidenote... when I decided to revisit this app after many months, it took more time than I want to admit to "relearn" how it was programmed. I had not commented the code very well. So, this is a good "reminder to self" that the effort to comment the code thoroughly is a good investment of time that will be returned to you manyfold later on.

How to Enter and Format Questions in a Text File

The questions are entered using the following format:

1. On the first line enter the line number of the correct answer (using the line of the first answer as line number 1). If you want a particular question to override the "Randomize Answers" option, then add a comma and enter "off" (example: 3,off).
2. Enter the question stem on the next line.
3. Enter as many answers as you wish, up to five. Put each answer on a separate line.
4. After the last answer, on a separate line, write the word "end."

Here's an example:

4,off
What color is a stop sign?
green
yellow
blue
red
violet
end

Feel free to take, use, or adapt this LiveCode file any way you wish. In future posts, I'll explain how key features of this quiz app work.

A feature of LiveCode I really like is the ability to import text files from the Internet using this simple command:

put URL "http://mywebsite/mytextfile.txt" into field "data"

So, although the current prototype of my app requires you to type the questions into the app directly, it would be easy to have the app check a web site as soon as the app opens to retrieve the question bank for the app. I plan on creating a version of this question app that does this.

OK, You Can Create M/C Questions in LiveCode. So What?

Unlike Captivate or Storyline, the range of possibilities of what one can do with multiple-choice quizzing in LiveCode is limited only by the imagination of the designer. Again, this may seem unfair to Captivate or Storyline, but I don't think so. I argue that these tools significantly constrain the designer. With LiveCode, my thoughts about what to do with my questioning app immediately turn to gaming. Probably my most favorite family game is Trivial Pursuit, a fact that continues to surprise me because it's all about answering questions about, well, trivia. I find it fascinating that a game with such a premise is so compelling to me and many other people. But, it's fun to know what you know. (I once scored big for my team because I knew the middle name of former president Jimmy Carter. Do you?)

But, any game you can imagine that uses the question and answer format could be built. One design that I like is commonly used in an in-flight game on Delta airlines where a timer begins when presented with a question. More points are given for a correct answer the faster you make your selection. However, you are not allowed to change your answer once selected.

Another idea I like is where you have students be responsible for generating the questions. This is a very constructivist design strategy.

So, Who Invented the Multiple-Choice Quiz Anyway?

Finally, I must give a nod to the people who came up with the idea of multiple-choice tests. After an exhausting literature search (translation: I googled it), I found that, according to Wikipedia, the originator was the famous behavioral psychologist E.L. Thorndyke, but it seems most of the credit for popularizing them goes to Frederick J. Kelly, who "...was the first to use such items as part of a large scale assessment. While Director of the Training School at Kansas State Normal School (now Emporia State University) in 1915, he developed and administered the Kansas Silent Reading Test. Soon after, Kelly became the third Dean of the College of Education at the University of Kansas. The first all multiple choice, large scale assessment was the Army Alpha, used to assess the intelligence and more specifically the aptitudes of World War I military recruits."

It's a good reminder that although multiple-choice quizzes and tests are easy targets for criticism, we should all be so clever to come up with an interaction and assessment strategy that has stood the test of time so well. Yes, they are overused and abused, but that is the fault of the designer, teacher, or more likely the state, federal, or company bureaucrats, not the inventor.

Gee, I wonder who invented the number 2 pencil?

Lloyd's Video Analysis Tool: Windows Version Update (I think it's working now)

First, a Special Note to Some of My Facebook Friends...

I'm sharing this post with a new group of people, namely my professional friends on Facebook. I've been use a software tool called dlvr.it (short for "deliver it") to automatically share my blog postings on LinkedIn and Twitter. I've now set it up to also share my blog postings to Facebook, but only to my friends who are connected in some way to the instructional technology profession. 

I do this with some trepidation as I, frankly, don't trust Facebook. But, I thought I'd give it a try to see if it helps me to connect better with some of my professional friends. I realize this may be the perfect reason for some of you to unfriend me on Facebook. In fact, I can easily imagine that many of you now have your finger poised above that unfriend button ... well, don't worry, I promise to keep my posts down to a few dozen per day.

Also, Facebook friends, if you want a little more information about my video analysis tool project, check out my most previous blog posting.

Back to Lloyd's "Major Accomplishment"...

Yes, I think I fixed the problem. Adding this one line of code to the card script seems to have solved the problem:

   set the playRate of player "player" to 1

Not sure why this was needed as the LiveCode user manual says that the playRate is set to 1 by default, and I never had set it otherwise. (Heck, I didn't even know the playRate property existed!) As I was trying out the Windows version, it looked like the video was playing correctly, but just at a tremendously fast rate. For example, I would press the "Play button" and the video would play all the way to the end in about a second, with the "buzzing insect" sound of video in extreme fast forward. When I clicked on a clip, it correctly only played from the clip's start frame to the clip's end frame, but in about a millisecond. So, I hunted around in the LiveCode dictionary and found out about the playRate property.

So, if there are any Windows users out there that want to try out my video analysis tool prototype, now is your chance. I've updated the prototype's Web page with information about this new option.

Lloyd's Video Analysis Tool, Part 9: Final Thoughts (For Now)

This will be my final blog posting about this project. Well, at least the last one of any significance ... probably. The reason is that I feel I have met all of my basic design goals and have begun some user testing. I'm sure I'll post updates about the project as time unfolds, especially as I move closer to releasing version 1.0. However, I'll be moving on to other project ideas and writing about them here. So, in this "probably final" posting, I'll describe some of the latest updates to the tool. And, there have indeed been several significant updates. I'll also describe some frustrations I've had with LiveCode over the past few weeks, which I believe are the result of some unexpected incompatibilities between the latest updates to the LiveCode software and the functions of my tool. As a result, I've "retreated" to version "LiveCode 6.7 (dp 3)" for all of my recent programming.

And, to be clear, I fully expect a few more significant updates to the tool as a result of the feedback I expect to get from the people currently testing my tool as I begin a count down to the launching of version 1.0. However, I will now be reluctant to make major modifications. Instead, I will consign all new ideas and features to the planning of the inevitable updates after I publish version 1.0.

Newest Features

To begin, here is a list of the tool's new functions:

• Option to show time code in the list of clips.
• Option to search the clips using any tag actually used in a clip.
• Option to reorder how the clips are displayed.

The first two new options were repeatedly mentioned by people who only saw the tool demonstrated. The third option seemed like an obvious one to make the more I used the tool myself. As soon as one identifies about 10 or more clips, it becomes difficult to keep straight which clip is which. And, combining the option of searching with reordering is a potent combination. I plan on redoing my overview video in order to show all of the new functions, but for now, here is another link to the existing video:

(After I make the new video, I'll update this blog post with it.)

Features I'm Still Considering Adding for the Release of Version 1.0

I think more search features are needed beyond searching on a single tag. The obvious next step would be to search on multiple tags using simple Boolean logic (i.e. and/or). An even more useful search option would be a keyword search that would scan any part of the clip information, including the clip title/description, tags, comments, and transcription. The reason for this is that I have found parts of a clip's transcript to be the most memorable when manually searching.

Another feature, which I also think is obvious, is to have the reports reflect the current search status. That is, give the user the option to create a report based on all of the clips, or only those currently showing after a search.

Video Analysis Tool Support Web Site

I created a Web site to support the tool.

It's very simple at this point, but it already contains some very useful information. For example, Apple recently implemented the "Gatekeeper" function in OS X, which by default is set to only allow the opening of applications that were obtained through the Mac App store. When you try to open my app, you get a message saying something like it "can't be opened because it is from an unidentified developer." Fortunately, it is easy to change your security settings on a Macintosh  to allow the app to open, and I point to a couple of short videos I found in YouTube that demonstrate how.

Unexpected, Frustrating Problems with Recent Updates to LiveCode

Near the end of my LiveCode workshop at the AECT conference in Jacksonville, Florida two weeks ago, I proudly showed my video analysis tool to the workshop participants. However, just a few days prior to the workshop I updated both LiveCode and Xcode in order to be able to show some of the basics of how to create and test mobile applications on Apple's iOS simulator. My attention in those days leading up to the workshop was not on my video analysis tool, but only the structure and main activities of the workshop. The tool was working fine up to that point and it never entered my mind that using the latest update to LiveCode would cause a problem. So, I committed the presenter's sin of not actually trying out the video analysis tool "one more time" before showing it during the workshop. Imagine my irritation (horror is too strong a word given the real problems in the world these days) at having key features of the tool not work as expected. In particular, my little transcription tool failed miserably. I've now used some common sense and have created a standalone of the tool that I know works and I will only show that for demonstration purposes.

Hey, I Thought This Was Supposed to be an iPad App!

Uh, um, ahem, yes ... yes, that was the original plan. But hey, plans sometime change! Two things have persuaded me to focus on a Macintosh app. First, when I began this project I knew only 10% of the LiveCode functions and commands needed to make it work. So, my learning of those functions -- many of which were related to file management -- have been intricately tied to the Mac architecture. Second, this particular tool just seems well suited for a laptop or desktop computer.

This doesn't mean that I've given up on the idea of someday creating an iPad version of this app. If the tool turns out to be useful by people, then I'll revisit the iPad app idea. Also, I plan on working on a simple video project on the iPad in order to learn about the iPad architecture, particularly in terms of how videos are stored and then accessed by a third party app.

OK, How About a Windows Version?

Actually, given that one is supposed to be able to export LiveCode projects to Macintosh, Windows, and Linux, I thought this would be a no-brainer. The tool works about 97% correctly on a Windows computer with QuickTime installed, but some of the key functions are not working properly, which has me befuddled. For example, the "Play/Pause" button that is just below the video window does not work. I made a new stack and created a button with basically the same script and it worked just fine, so I'm not sure what is going on. Once Macintosh version 1.0 is finished, I plan on coming back to work on this.

Windows Version Update! I think I fixed the problem. Adding this one line of code to the card script seems to have solved the problem:

   set the playRate of player "player" to 1

Not sure why this was needed as the LiveCode reference information says that the playRate is set to 1 by default, and I never had set it otherwise. (Heck, I didn't even know the playRate property existed!) As I was trying the Windows version, it looked like the video was playing, but just at a tremendously fast rate. I would press the "Play button" and the video would play all the way to the end in about a second, with the "buzzing insect" sound of video in extreme fast forward. So, I hunted around the LiveCode dictionary and found out about the playRate property.

So, if there are any Windows users out there that want to try out my video analysis tool prototype, now is your chance.

Still Searching for the Perfect Name for the App

I have been racking my brain to give this tool a proper name. Interesting, a few weeks ago I spent a weekend brainstorming some possibilities, but none of them seemed right. Here were some of the leading contenders:

• Video Milkshake Maker: For a Rich and Thick Analysis
• Video Explorer: Go and Discover What's in Your Video
• Video P.I.: Find the Clues in Your Video

(Actually, I thought the "milkshake" idea was pretty clever for a day or so.)

Checking a thesaurus also led me to these poor possibilities:

• Video Detective
• Video Scientist
• Video Researcher
• Video Discovery
• Video Investigator
• Video Inspector
• Video Examiner
• Video Prober
• Video Prospector
• Video Reconnaissance

Frankly, the name "Video Prober" gave me the willies.

I then thought about acronyms, and for about an hour I seriously considered "Video CAT" - short for "Content Analysis Tool."  (Yes, imagine a cute cat photo inserted here.) 

Then, I thought about taking and rearranging some of the syllables to create one of those "hip" app names that are all the rage, but I was dismayed as to how many negative connotations emerged from "analysis" (anal, lies, sissy). However, "eyes" is also a derivative, so I thought about variations like "Video-ize."

When I discussed some of the best of these ideas with my wife, she just frowned, shook her head, and walked away. So, in the end, I remain inspired by the Pittsburgh documentary filmmaker Rick Sebak who titles his programs very simply -- "A Hot Dog Program," "An Ice Cream Show," etc..

So, for now, the title remains "Lloyd's Video Analysis Tool."

Any ideas?

LiveCode Workshop at AECT in Jacksonville on Tuesday

I'll be conducting a LiveCode workshop at the AECT conference in Jacksonville, Florida on Tuesday afternoon. Although the workshop officially filled awhile back, a few people emailed me over the weekend to say that their travel plans changed and now cannot participate -- so, a few seats remain in case anyone is interested.

I'll be updating my LiveCode Workshop site after the workshop, in case anyone who cannot join us in Jacksonville wants to check that out:

http://nowhereroad.com/livecode/

I plan on making one or two more video tutorials based on the feedback of the workshop attendees. (I really want to do a video for the "Basketball Camp" project, so I'll likely do that no matter what.)

Looking forward to "learning LiveCode" in Jacksonville!

Lloyd's Video Analysis Tool, Part 8: Prototype Ready for Some User Testing

I've been working slowly but surely on my video analysis tool. I've now finished the initial programming of all of the essential features, including the transcription tool. I've also spent some time on the graphic design of the project, which, while not great, now makes the tool somewhat more "presentable." I feel I now have a prototype that is ready to be shared and tested by some people who have been expressing interest in the tool. Getting feedback from people who are already analyzing video in their work, but who are also willing to put up with yet-to-be-discovered bugs and problems will be crucial.

I have prepared a short video to explain how the tool works. I intentionally did not mention LiveCode at all because I know the people with whom I'll share this video care deeply about video analysis, but don't care at all about how the tool was made.

Here's a list of the main features of the tool:

• Defining as many video scenes as you wish in the video file you are analyzing (I just call them video entries).
• Tagging the video scenes with as many tags as you wish.
• Creating a list of "quick tags" to more easily add the most common tags or codes to a video scene.
• Adding a comment or description to each video scene.
• Adding a transcription to each video scene, along with a transcription tool inspired by the YouTube transcription tool.
• The option to create reports of a video analysis project, including an "Excel ready" report option.
• A file management system that saves all the projects inside a special folder in the computer's document folder.
• The option to manually save (or export) back-ups of a video project (called "source files").
• The option to import source files; this is meant as a way to share the project with another user of the tool. (However, you also have to share the video file separately.)

I'm particularly proud of the transcription tool. Although it still needs some work, I find it very effective way to create transcriptions because the tool directly controls the video as you type. When the tool is activated, only the space bar plays the video and every other key stops the video. There is also an automatic rewinding effect. I also included a 5 second manual rewind option.

There are still a few features I still would like to add to the tool. The one at the top of the list is a search option, mainly for searching among the tags.

All of my testing so far has been on the Macintosh, but I will also begin testing a Windows version of the tool.

I want to publicly thank a few people who made some very helpful suggestions based on demonstrations of earlier prototypes back in July and August: Dr. AnnaMarie Conner, Dr. TJ Kopcha, and Dr. Mardi Schmeichel. Dr. Conner is on the faculty in the Department of Mathematics and Science Education, Dr. Kopcha is fellow member of the Learning, Design, and Technology faculty, and Dr. Schmeichel is on the faculty in the Department of Educational Theory and Practice.

Special thanks to AnnaMarie and Mardi for taking some extra time to show me the qualitative analysis tools they are using for video. Mardi uses Dedoose and AnnaMarie uses Transana. AnnaMarie's research involves very complex and expansive use of video.

One of the most beneficial things about seeing a tool like Transana, especially within the context of a sophisticated video-based research program, is that it reinforces and clarifies my initial thoughts that my tool is not meant in any way to compete with them. My video analysis tool is meant for small scale, niche applications, only some of which will be research.

Finally, special thanks also go to Daisyane Barreto, a doctoral student in our department who has been testing an earlier prototype of my tool for the past month or so in the analysis of video data she has collected for her dissertation research. She reports that the tool is working well for her and I made some modifications to the current prototype based on my conversations with her.

Reflections on the RunRevLive.14 Conference

I recently attended and presented at the RunRevLive.14 conference in San Diego, CA. As a result, I have come to two simple conclusions:

1. People at RunRev and the LiveCode community at large consist of very, very smart people who are also very helpful, supportive, and kind.
2. I know about 1% of LiveCode.

Even though I do feel now as if I know very little of what LiveCode can do, the fact that I am still able to do so much shows its power and potential. Consequently, the conference has greatly reinforced my decision to commit to learning LiveCode.

I was able to attend the conference because my proposal to make a presentation at the conference was accepted. This came with a complimentary conference registration, otherwise, I never could have afforded the $1599 registration fee. The title of my presentation was "From Concept to Prototype to the App Store" and it featured the story of my design of my iOS app "Lunar Hotel Shuttle." I've written about this app in previous posts, but in short, the app teaches about Newton’s Laws of Motion in a futuristic, imaginary context. The storyline is that NASA has built a hotel on the moon that is so large that elevators are not possible or practical. Instead, hotel guests are shuttled to their hotel floor by a small shuttle that the player has to learn how to fly.

I explained in my presentation that the main purpose of developing this app was not to create a great game that would make any money, but rather to test the appropriateness and functionality of LiveCode for action games on mobile devices, especially a game where the "feel" is very important. This was important to me in order to know whether LiveCode would be an appropriate tool for projects like this and to understand where LiveCode's limits were to be found for games requiring high responsiveness. I also wanted to test out a wide range of functionality within LiveCode.

I also contrasted LiveCode as a development tool for instructional design with Adobe Captivate and Articulate Storyline, probably the two most popular eLearning development tools currently on the market. I used a taxonomy of learner interactions from the military to based my comparison of LiveCode with these other tools:

---

Levels of Interactivity in eLearning

Level 1 - Passive
Watching a video, self-running slide show, or animation

Level 2 - Limited Participation
Simple interactions (e.g. M/C) with no branching

Level 3 - Complex Participation
Simple and complex interactions (e.g. drag and drop) that conditionally branch the lesson

Level 4 - Real-Time Participation
Games and simulations

Adapted from “Department of Defense Handbook: Development of Interactive Multimedia Instruction,” 1999, Report No. MIL-HDBK-29612-3

---

I like to refer to Level 4 as "creative interactions." Here were my simple conclusions based on this comparison:

• LiveCode is an important, but not exclusive, tool in the instructional design & development toolbox.
• LiveCode is excellent for creating Level 4 (Creative) Interactions.
• LiveCode is strong for mobile native app development for eLearning, but weak for Web-based eLearning.
• The option to export to HTML5 should make LiveCode a game changer for eLearning development.

Yes, you read right - RunRev is currently working on providing the option to export LiveCode applications to HTML5. I believe LiveCode will become a very serious contender in the eLearning design world when this export option becomes available. (Apparently, LiveCode will be converted into JavaScript for easy integration into existing HTML5 applications, therefore no plug-ins or Java will be required for it to run.)

My audience was small, but very engaged. I thank all who came to my session. A special thank you to Bernie Dodge, Professor of Learning Design and Technology at San Diego State University, for coming to my session. LiveCode is one of the tools Bernie is recommending to students in his gaming and simulation course this semester. Bernie and I also had some very good conversations at the conference about design and reflections on the recent history (past 30 years or so) of the instructional technology field.

On the plane ride to and from San Diego, I read a good deal from Paul Graham's book Hackers and painters: Big ideas from the computer age, published by O'Reilly Media, Inc. One quote from the book captures well the enthusiasm and commitment of the LiveCode community:

“Ordinary technology changes fast. But programming languages are different: programming languages are not just technology, but what programmers think in. They’re half technology and half religion.” p. 179

I am already looking forward to my next RunRevLive conference.

Using LiveCode to Stitch Together "Big Data": Yet Another Example of the Value of Craft Programming

I've been teaching a massive open online course (MOOC) for the past year - Statistics in Education for Mere Mortals - and I've been collecting research data all along. Time has come to work on writing up this research to share with my professional community. I tend to work best best when there is a deadline. "Lucky" for me, the deadline for proposals to the 2015 annual conference of the American Educational Research Association (AERA) in Chicago is just around the corner - July 22, 2014.

However, I've been slow in analyzing the data because I've been a little confounded as to how, exactly, to go about it. You see, the data have been collected by the learning management system (LMS), but not in a way that is conducive for research. In order to effectively analyze quantitative data in a research study, you need to have all of the raw data for each participant neatly organized into a single line of information. By raw data, I mean every single detail, such as individual answers to specific questions, not just totals. Put into spreadsheet terms, each person needs to be a row and all of the data for that person needs to be the columns. Then, I can import this into a statistical software tool, such as SPSS.

Collecting Data in a Learning Management System

Unfortunately, most LMSs do not organize course data in this way. It's true the LMS has a grade book that automatically keeps track of all course participants, but this only keeps track of aggregate data, such as subscores on individual quizzes, surveys, and the like. Again, I need the raw data. The LMS does keep track of the raw data, but only within individual activities. A good example for my needs is a series of surveys I ask participants to complete at the beginning, middle, and end of the course with questions such as the following:

Which of the following best describes your learning goal at this moment?

• I intend to fully participate in the course and complete it.
• I intend to fully participate in most of the course.
• I intend mainly to browse and participate in a few activities.
• I will probably just browse around with little actual participation.
• I really don't have a specific learning goal at this point.

I don't care about their overall "score" on these surveys, instead I need to focus on answers to specific questions. For example, I'm interested in comparing among those who actually did finish the course with those who declared at the start of the course their intent to do so versus those who said they were there just to browse with no intention to actually complete the course.

So, yes, I have access to raw data for each of these surveys, but these data are "trapped" within individual survey reports. No problem, you say, just copy and paste the data from each survey for each person into a master spreadsheet file. That would work fine if I only had 20 or 30 participants. But, I have had an average of over 800 participants in each of the four courses -- about 3400 overall! And, to complicate things further, participants vary tremendously in terms of what surveys and survey questions they choose to complete. So, I find overwhelming the magnitude and complexity of manually copying and pasting this amount of data together.

I had held out some hope that the LMS administrators could help me out and produce a master spreadsheet file for me, so I made a special request for this. Unfortunately, they reported being unable to comply as apparently the data are also located in separate files at the system level (which is a little surprising).

LiveCode to the Rescue

So, what to do? Fortunately, LiveCode's powerful list processing capabilities once again provided a solution. Most of the data reports contain a few common categories, such as the person's name and a course ID. I was able to use these common data points to "stitch" together the various data files.

The code is surprisingly simple, and is based on a algorithm I've used in many programs I've written about already. Before I get into that, let's see how I've structured the LiveCode stack. The stack consists of just one card. Here's a screen shot:

As you can see, there are three fields named (from left to right) "data1," "data2," and "finaldata." The idea is to copy and paste comma-delimited text of the data from an individual survey into the first two fields, with the first course survey going into data1. (The LMS is able to generate Excel files for each survey, so it is then easy to generate a .csv file from that.) Each line is an individual participant with each data point for that person separated with commas. My script starts with the first line in data1, then methodically scans each line in data2 to look for that same person. It if finds a match on the person's name, it then combines (i.e. "stitches") the two lines together and puts it into the next line of field "finaldata." If it doesn't find a match, it then just puts the line from data1 into finaldata.

A Very Short Script That Does Some Heavy-Duty Work!

The script for button "Cross-check with item 1" does this work:

on mouseUp

   put the number of lines in field "data1" into line 1 of field "data1lines"

   put the number of lines in field "data2" into line 1 of field "data2lines"

   

   put empty into field "finaldata"

   

   repeat with i=1 to the number of lines in field "data1"

      put false into varMatch

      put i into line 1 of field "progress"

      repeat with j=1 to the number of lines in field "data2"

         if item 1 of line i of field "data1" = item 1 of line j of field "data2" then

            put true into varMatch

            put line i of field "data1"&comma&line j of field "data2"&return after field "finaldata"

            exit repeat

         end if

      end repeat

      if varMatch = false then put line i of field "data1"&return after field "finaldata"

   end repeat

   

   beep 2

end mouseUp

Item 1 of field data1 and data2 contains the person's name. In the parlance of databases, this acts as the "primary key" for each of these two separate data files. The local variable "varMatch" keeps track of whether a match has been found or not for each line in data1. When a match is found, it becomes "true" and the lines from the two fields are combined. If, after scanning all lines in data2, no match is found, this remains false and the line from data1 is added to finaldata without anything added to it.

It's important to note that after I have all the data combined, I need to delete the individual student names. That is important in order to comply with all human subjects policies and procedures. 

Cascading Data, or Creating a Data Quilt

After I run this program once, I am far from done. I have several more surveys and other data files to stitch together in this master file. So, I built the option to first empty fields data1 and data2, then take the data from the finaldata field and move it into data1. Then, I can copy and paste the comma-delimited text file from the next survey into data2 and begin to stitch those together. This reminds me of a cascading waterfall of data. Another way of putting this is to say that I'm stitching together a large "quilt"from each patch of data from the individual surveys!

The Continued Value of Craft Programming

I consider this tool another example of craft programming. The program focuses on solving a unique and narrow problem. Without modifications, it is of benefit to no one else but me. But, it solved a very difficult problem for me very well and was well worth the time to create. It only took me about two hours to figure out how to build this stack. It then took about 30 minutes to run the program for the data for one of my MOOC course sections, stopping as I went to generate a new .csv file for each successive survey followed by copying and pasting the data into the data2 field. I hesitate to estimate what it would have taken to manually cut and paste the data to do the same job. Maybe 25 very tedious hours per MOOC? Maybe a lot more. Really hard to say. And, remember, I have a total of four course sections to analyze, with more sections to be offered next year. I'm also sure I would have made many errors doing this manually, whereas I can trust LiveCode to follow my algorithm perfectly.

There was no one else I could turn to for help on this. No "real" programmer within my university would have been inclined to take on this job as a service to a researcher like me and I had no funds to pay someone to do it. Creating this craft programming project with LiveCode has been a lifesaver.

Now on to the real work of analyzing the data.