Sunday, October 6, 2013

Creating a Classroom Observation Research Data Collection App

I'm teaching a doctoral seminar this semester titled "Developing Learning Analytics and Digital Assessment Tools for an Educational Research Program." We are demonstrating a bunch of ways to build research tools that collect data. One of the first tools I showed was LiveCode in the context of creating an app that would assist a researcher in systematically collecting data during classroom observations.

So, imagine you are sitting in a classroom and you want to make keen observations about what is going on. It's easy to imagine exchanges between the teacher and students.  Examples of teacher activities might include giving directions, lecturing, asking questions, leading a class discussion, and so on. Similarly, examples of student activities might include individual work, working in pairs, group work, making presentations, reading, and so on. And, lots more categories could be included such as tools that both teachers and students use, such as the Internet, PowerPoint, Smartboard, word processing, and so on.

Now, you might think it would be just fine to sit there with a notebook and pen and write down what you see. If you think this, then it's probably been a long time since you were in a classroom. There is no way you could keep up with what is going on. Plus, we humans are not very good are taking accurate notes when it comes to time and duration. You could try to look at your watch and note what time one activity started and ended and who was involved, but the number of errors you would make would likely make your notes not very usable.

Enter the computer. It does a marvelous job of noting what time it is, down to the millisecond if you wish. And wouldn't it be nice to simply click a button called "Teacher" or "Asking Question" for the computer to note who did what and at exactly what time. That's basically what this app does.

During week 2 of the seminar I showed a classroom observation app prototype I quickly built with LiveCode to give everyone a simple example what LiveCode could do, particularly in relation to coding time and duration. In the weeks that have gone by since, several of the students have considered using an adaptation of this first prototype as the basis of their course project. Just this past week I was conferring with Lindy Johnson, a very talented doctoral student in Language and Literary Education who is taking the course. We played around with her version of the app. In the day or two after talking with Lindy, I came up with an idea for an enhanced version of the app. This is the version I'm sharing here.

Here's a screen shot of the app:

[ Get the free LiveCode Community version. ]

Notice the bright red line going down the middle. This is a graphic object named "dividing line," and it figures prominently in the scripts to follow.

I built the app using the dimensions of the iPad (1024 by 768) because I think a mobile device such as this would be the definitive platform for conducting this type of research in a school setting.

Here is the script of the stack's second card titled "Researcher Interface":

global gActor, gActivity, gTimeStamp, gDataString, varStudentZoneX

on addData
   if gDataString is empty then
      put the date&","&the time into gTimeStamp
      put the seconds into varTimeStart
      put gTimeStamp&","&varTimeStart into gDataString
   end if   

   put gDataString&","&gActivity into gDataString
   put gDataString into line 1 of field "datastream"
end addData

on openCard
   put item 1 of the location of graphic "dividing line" into varStudentZoneX
   put empty into gActor
   put empty into gActivity
   put empty into gDataString
   put empty into field "datastream"
end openCard

A variety of global variables are declared at the start. I then define a new procedure (or command) called "AddData." We'll come back to this procedure and how it works later in this post.

Then, some script is executed when the card opens. The first line notes the location of the graphic "dividing line" and puts its horizontal, or x, coordinate into the variable "varStudentZoneX." (Recall that "the location" of an object will result in two items indicating the horizontal and vertical screen coordinates of the object. Think of these as the x,y coordinates of the object (and remember that the 0,0 origin is in the upper left-hand corner of the screen). All we care about here is the x coordinate of the dividing line.

The other code in the openCard script simply puts "empty" into a bunch of variables, thus initializing them (i.e. getting them ready) for use in the card.

Button Properties and Script

Let's take a look at one of the buttons. As it turns out, each is identical with two critical exceptions -- the name and label of each button. Here is a snapshot of the properties of the button "Gives Direction":

An interesting feature of a button is that its name and label do not have to match. The name of the button is what appears to the user, but the label can be something entirely different. I chose to make both the name and label the same, but I wanted to show how to use this feature in this app. So, the label could be something much shorter, such as a special code such as "GD" for "Give Directions," or whatever you like.

Here is the script for this button (again, all the buttons share this exact same code):

global gActivity, gActor, varStudentZoneX

on mouseUp
   if item 1 of the location of me > varStudentZoneX then
      put "student" into gActor
      put "teacher" into gActor
   end if
   put gActor&":"&the label of me into gActivity
end mouseUp

Notice the first block of code after "on MouseUp." We have a simple If-Then-Else structure that looks to see if item 1 of the location of "me" (that is, the x coordinator of this button) is greater than varStudentZoneX, which you will recall is the x coordinator of the red dividing line. If it is, then it must be a student-related button and "student" is put inside of the global variable gActor. If it is not, then it must be a teacher-related button and "teacher" is put inside that variable instead.

So, this is a cool way to have buttons on the screen self-determine if they are on the left or right of the screen, and hence are related to what the teacher or student is doing.

Let's look at the last two lines of the button script (just above the "end MouseUp"). Here's the first:

   put gActor&":"&the label of me into gActivity

This puts a concatenation of the following into the global variable "gActivity" (recall that the & symbol concatenates, or "adds together" a string of items):  gActor (either "student" or "teacher"), a colon, and the label of "me" (which again is the label of this particular button).

The last line executes the custom procedure "addData" defined above in the card script. So, let's take a closer look at that:

on addData
   if gDataString is empty then
      put the date&","&the time into gTimeStamp
      put the seconds into varTimeStart
      put gTimeStamp&","&varTimeStart into gDataString
   end if      

   put gDataString&","&gActivity into gDataString
   put gDataString into line 1 of field "datastream"
end addData

You'll notice a whole lot of concatenation going on in this, so look for those &s. It first looks to see if the variable "gDataString" is empty. If it is, this indicates a new event is about to be coded and it puts the date and time into gTimeStamp, each separated with a comma.

Next, it puts the seconds into varTimeStart. I've talked about "the seconds" in previous blog posts. It's basically the current time also, but given in seconds. This is particularly handy when one wants to compute how much time has passed, such as with a timer. (Revisit my blog posting on making of the game "Crack the Code" for a full explanation.) So, these various functions take note of the current time, down to the exact second, when the classroom event began.

The next lines concatenates gTimeStamp and varTimeStart (with a comma in-between) and puts it all into the global variable gDataString.

The next line concatenates gActivity -- the value of which came from the button just pressed -- with the contents of what was already in gDataString.

The contents of gDataString are then displayed in the field "datastream," which is the field located toward the bottom of the card window. So, as the researcher clicks on more buttons, the continually building stream of data will be shown on the screen.

Add a Button: Go Ahead and Try It

If you want to add a button to the list of events, just copy any of the buttons currently available on the card. Paste it anywhere you want, then just change the button's name and label. If you put the button to the left of the red dividing line, it will be counted as a teacher event. Put it on the right of the dividing line and it will be counted as a student event. Give it a try.

Yikes! I Clicked the Wrong Button

OK, this is good so far. The researcher is clicking away and seeing the data of events unfolding before his/her eyes. But, what happens if the researcher clicks the wrong button? Well, the button "Erase Last Data Point" acts as an undo button by removing the last data point from the data stream. 

Let's take a look at how this button works. Here the button's script:

global gActor, gActivity, gTimeStamp, gDataString

on mouseUp
   put empty into varTemp
   put line 1 of field "datastream" into varCurrentItems
   put the number of items in varCurrentItems into varItems
   put item 1 of varCurrentItems into varTemp
   repeat with i = 2 to varItems -1
      put varTemp&","&item i of varCurrentItems into varTemp
   end repeat
   put varTemp into gDataString
   put gDataString into line 1 of field "datastream"
end mouseUp

I'm not going to dissect this line by line, but basically it simply looks at how many items are in the current data stream, separates out each items, then reconstructs the data stream by stopping one short. The result is the same data stream, but minus the last item. The last line of code (just before the "end mouseUp" puts the reconstructed gDataString into line 1 of the field "datastream," thus replacing the line of data previously located there.

The button "Reset" simply empties out the current contents of "gDataString," the main variable holding the data, and removes anything that is in the field "datastream":

global gDataString

on mouseUp
   put empty into field "datastream"
   put empty into gDataString
end mouseUp

This of this as "Do Over" button.

Recording the Data

When the classroom event has ended, the researcher can click the "Record" button to record the data on the card "data." It also then resets everything for the next classroom event.

Let's take a look at the script in the "Record" button:

on mouseUp
   put line 1 of field "datastream" into line (the number of lines of field "data" on card "data" +1) of field "data" on card "data"
end mouseUp

There are only two lines of code here. The first puts the data from the field "datastream" into the next open line in the field "data" on the card "data." It does this by first counting the number of lines currently being used on the field "data", then adds one to this number. (Another way to do this, which is simpler, is to use the command "after.")

Here's a partial screen shot of the data resting comfortably in the field "data" on the card "data":


The simple one word script "opencard" does just that, it reopens the current card which you will recall initializes everything.

It's important to note that the current strategy for collecting the data only works while running the app directly in LiveCode. Before you publish this app as a stand-alone application, you have to save the data from this field to a text file that would accompany the app. (I show how to do this on my LiveCode workshop site.)

Analyzing the Data

One thing that this app does not do is analyze the data. But, why bother scripting that out when we have such fantastic tools such as Excel and SPSS? All of the data recorded in the field "data" on the card "data" can be copied and pasted into either application. For example, here is an example of some data copied and pasted into Excel:

After copying and pasting the data into the first cell of a new Excel file, I used Excel's "Data > Text to Columns" feature to separate each item of data into its own column (using the comma between items as the delimiter).

Now that we have the data in Excel, we can add a column that figures the duration of each event. I added a new column - Column D - and inserted this formula in the first row:


This subtracts the time the second classroom event started from the time the first classroom event started and puts the answer as the total time that the first event lasted. By using the "fill down" technique, I can quickly add this relative formula to all of the other events (one event per row).

I could do other quantitative analyses very easily, such as counting the number of event occurrences with the COUNTIF or COUNTA functions. Lindy is also interested in adding other tags besides "teacher" and "student" -- besides the "activity" and "tools" tags, her app has about six more. (Her specific items under each tag go far beyond what my version shows here.)

What About Qualitative Data?

Good question. Despite the apparent heavy quantitative elements to this app, recall that the app's main function is to code and record classroom events as they occur, a predominately qualitative activity. If the researcher is also recording the audio or video of the class (and simply synced the time that the class recording and app started, it would be very easy to then revisit the app's data stream to add the student and teacher voices to the narrative. It would also be easy to add a notepad to the app to let the researcher make short narrative notes while collecting the other data.

Using the Computer as a Partner in Research

Again, the whole purpose of this was to be creative in using the computer's processing abilities as a powerful tool to implement and collect data in an educational research project. I think this app shows an intriguing example of how to do this.

No comments:

Post a Comment