Lloyd's Video Analysis Tool, Part 3: Building a Log of Video Clips

OK, now that we have a command of some of the basics of using video within LiveCode, let's begin to explore how to build some custom features for the video analysis tool. A key element of the prototype of my video analysis tool is giving the user a way to log a series of video clips. Then, each clip can be played back, tagged, and commented on.

Here's a screen shot of the LiveCode file we will be building:

 LiveCode file

[ Get the free LiveCode Community version. ]

 

As you can see, I widened the stack (to 1024 pixels, the landscape width of an iPad) and I moved the elements from the previous version toward the left. On the right is a scrolling list field that contains an ever-growing list of video clips. The idea is to let the user define a starting and ending point for as many clips as they wish. Each can then be played back at will.

Let's start by adding a scrolling list field on the right hand side of the card and title it "log." Be sure it is the "scrolling list field" and not a "scrolling field." Hover and pause over the various list objects in the tool palette to see the object names appear. I'll explain later the difference

Next, add a button titled "New Entry" and add this code to it:

on mouseUp
   put the number of lines in field "log" into l
   //start of video snippet
   put the currentTime of player "player" into item 1 of line l+1 of field "log"
   //end of video snippet
   put the timescale of player "Player" into ts
   put the currentTime of player "player" +(ts*5)  into item 2 of line l+1 of field "log"
end mouseUp

The first line looks at how many lines in the field "log" have content and puts that value into a local variable "l" (short for log). When the field empty, the result will be 0. Our goal will be to add content for the next clip after that line.

Next we need to define a video clip. We will take what we learned in the previous blog post to build the starting and ending points of the clip. The third line notes the current frame of the video player and then puts this as the first item of the first empty line of field "log" (i.e. l+1). I want the default length of the video clip to be 5 seconds. To do this, I must first check to see what the timescale of the movie is, and line 5 accomplishes this and puts the value in a local variable "ts." Line 6 multiples ts by 5, adds this amount to the current position of the video playhead (as denoted by the property currentTime), then puts the result also into line l+1, but as item 2 (LiveCode will automatically add a comma to partition the two items).

Great! We now have a starting and ending point for the video clip. Try it out by moving the playhead to different locations and pressing the "New Entry" button. It should add a starting point at the position of the playhead and an ending point five seconds later. A new video snippet will be added to each successive line in the field.

I think it would be useful to add a button that clears out all the log entries. This will be particularly helpful as we test out the prototype. Add a button called "Clear Log" with this one line of code:

on mouseUp
   put empty into field "log"  
end mouseUp

Notice that I put this button in the upper right hand portion of the screen. I didn''t want it next to the other log buttons so as to avoid inadvertently pressing it and deleting all the entries by mistake.

Let's now create a button that plays that video snippet. Title it "Play Entry" and place it just below the "New Entry" button. Here is the code for it:

on mouseUp
   set the playSelection of player "player" to true
   set the startTime of player "player" to item 1 of the selectedtext of field "log"
   set the endTime of player "player" to item 2 of the selectedtext of field "log"
   start player "player"
end mouseUp

The first line is important because the property "playSelection" is set to false by default. So, what is this property? If it is true, then the player will start playing the video and not stop until the end of the entire movie has been reached. Of course, we want it to stop when the playhead reaches the final frame of the clip. Note that you can change this property also by going to the basic properties page of the player object's inspector (i.e. "Play selection only"), but I think it's preferable to code this directly into the button.

The next two lines set two important player properties. As their names suggest, each refers to the starting and ending position of a video (or audio) clip. Therefore, we set the property "startTime" to item 1 of the respective line of the log field, and "endTime" to item 2.

Take a few minutes now to add three or four log entries. Each will be five seconds in length. Click on the respective line in the log field and press the "Play Entry" button. That entry should play and end in five seconds.

Please also take a moment now to recognize the difference between a scrolling list field and a regular scrolling field. When the user clicks a line in a scrolling list field, the entire line is selected, as compared to placing the cursor at a certain location within that line. This difference allows us to use the function "selectedtext"to select the entire line. A subtle, but important and very useful, difference.

Giving Users the Option to Update an Entry

All is working great, but obviously a five second clip will unlikely be satisfactory to the user. So, the next step is to give the user the option to update the beginning or ending points of the clip. The design is straightforward. Have the user click on an entry and give them the option to move the playhead anywhere they want, then give them the buttons "Update Start" or "Update End" to update the clip. Let's add both of these buttons now.

Create a button named "Update Start" and add this script to it:

on mouseUp
   put the hilitedLine of field "log" into l
   put the currentTime of player "player" into ct
   //error trap - starting point must be before ending point
   if ct > item 2 of line l of field "log" then
      answer error "Not Allowed! Starting point must come before the ending point!" with "OK"
      exit mouseUp
   end if
   put ct into item 1 of line l of field "log"
   set the hilitedLine of field "log" to l
end mouseUp

Line 1 uses a cool property called "hilitedLine." As the name suggests, this property takes note of what line has just been highlighted in a text field. Line l puts this line number into a local variable "l" (this is a lower case "L" remember, not the number one).

Line 2 notes the current position of the video playhead and puts it into a local variable "ct." (Recall that the user would have moved the playhead before clicking this button.)

The next several lines do some error trapping, as we humans are very prone to errors. We'll come back to these in a moment.

Let's jump to line 8. This line puts the value of "ct" into the selected line of field "log," replacing whatever value was already there. Well, of course, the idea of this button is to update the entry after all.

The last line highlights the line in the log we just edited. This is important from a user design standpoint. Without this line, the user would have to again click on that line if they wanted to make another update. I think it's easy to imagine the user making lots of updates in a row until they get things "just right." (Tha'ts what I found myself doing as I tried out the prototype.) So, this line highlights the line for them.

OK, what about that error trapping? Think about it for a moment. We have to ensure that the starting point of the video clip is a number less than the ending point of the clip. So, line 4 starts an if/then statement to check this. If "ct" (the new position of the videohead) is greater than the ending point of the video clip, then that's a problem. So, LiveCode pops up with a message saying something to the effect that you can't do that, using the handy "answer" command. The next line - "exit mouseUp" terminates button's script, so nothing else happens. (Alas, error trapping is a very time-consuming, but necessary part of all software design. One has to be very cunning to determine where human error is likely to occur. This is very hard to predict and is one of the valuable outcomes of extensive field testing.)

Next, add another button and title it "Update End." As you might suspect, the script for this button is very similar to the button "Update Start":

on mouseUp
   put the hilitedLine of field "log" into l
   put the currentTime of player "player" into ct
   //error trap - ending point must be after ending point
   if ct < item 1 of line l of field "log" then
      answer error "Not Allowed! Ending point must come after the starting point!" with "OK"
      exit mouseUp
   end if
   put ct into item 2 of line l of field "log"
   set the hilitedLine of field "log" to l
end mouseUp

In fact, you could just copy and paste the "Update Start" button and make just a few, small (but critical) revisions. For example, in line 8, the only revision is changing item 1 to item 2.

The only other changes are in the error trapping. In this case, we have to be sure that the ending point comes after the starting point. Here is a quick summary of the changes to these lines of code:

• Change the "greater than" sign to a "lesser than" sign;
• Change "item 2" to "item 1"; and
• Change the text of the message as needed.

Finally, I added a blue rectangle to more appropriately show this "editing zone." To move the rectangle "to the back" go to the "size and position" page of the property inspector and change its layer.

Revising the +10 and -10 Second Editor

Let's do one more thing. Recall from my previous blog posting that we created two buttons that advanced or rewound the playhead by 10 seconds. An increment of 10 seconds was a good choice there to demonstrate this feature, but it really is not a useful increment for "fine tuning." So, let's revise those two buttons so that the increment is only one second. (One could argue that even a smaller increment, such a half or tenth of a second, would be even better.) Making this revision is a snap.

Let's take a look at the existing script for the "-10 seconds" button:

on mouseUp
   put the timescale of player "Player" into ts
   put the currentTime of player "Player" into t
   put t-(ts*10) into t
   set the currentTime of player "Player" to t
end mouseUp

All we need to do is change the 10 to a 1 in line 3. (You might also argue that we could just delete the "*10" altogether, but I don't recommend that because we might find ourselves coming back and changing this later to a different value, such as .5.

Do the same thing for the "+10 seconds" button and, of course, don't forget to change the button titles.

Final Thought

My final thought is simply "Hey, pretty cool!" We have a good start to an interesting video analysis tool.

In my next post, I'll focus solely on how to convert or transform LiveCode's time code intervals to something we humans can better understand, such as hours:minutes:seconds.

Lloyd's Video Analysis Tool, Part 2: Some Video Basics

In this post I'll explore a few of the basics of bringing video into a LiveCode project. Here is screen shot of the stack we'll build:

 LiveCode file

[ Get the free LiveCode Community version. ]

 

You'll obviously need your own movie to use in this project.

The first step couldn't be easier. Just drag a player object onto a card:

The player object's name defaults to "Player," but feel free to rename this as you wish. (But, note that we will be referring to the player object's name in the scripting below, so decide on the name now.) I'll leave it named as "Player." A good next step is to explore the properties of the player object by browsing the various pages of the object's inspector window.

Source video files can either be stored on your computer, or you can enter the URL of the source video. There are some limitations on what video file types can be used. According to the LiveCode manual, video is limited to QuickTime, AVI, or MPEG file types. So, unfortunately and much to my disappointment, flash video is not a supported format (and hence, one cannot use YouTube videos.)

For many projects, this is all you need to know! The player object will now play your video and it includes all of the expected video options, such as play, pause, step forward and backward, audio control, and user control of the playhead.

Note: If you choose to resize your video, be sure to check the box beside "lock size and position" inside the size and position properties page. Otherwise, it will appear full size the next time you open the file.

Video-Related Scripting

To create a more sophisticated video application, such as the video analysis tool I have in mind, we need to explore how to script various video functions. Let's explore just a few of the most basic here. A good place to start is scripting our own play and stop buttons. The scripts for each button are very simple:

on mouseUp
   start player "Player"
end mouseUp

on mouseUp
   stop player "Player"
end mouseUp

In my earliest Video Analysis Tool prototype, I've already included the option to move the video playhead forward or back by one second. Let's explore how to do that as our next script. Instead of one second, let's use an increment of 10 seconds in order to really notice the playhead skipping ahead or back.

To do this, we first have to note where the playhead is currently in the video. Then, we'll just add or subtract 10 seconds worth of video and jump to that new spot. Let's say we stopped the video one minute and 15 seconds into the video. LiveCode does not "see" the video in seconds or minutes. Instead, it will play a particular video in a certain number of intervals per second. This interval will vary from video to video. Fortunately, we can use the timeScale property to learn what this interval is. Let's open the Message box (an option under "Tools") and type in this script:

   put the timescale of player "Player"

After you press Return, a number will appear in the message box. For the video I'm using, the number is 90,000. Yours will most likely be different. So, this means that for my video, the time scale of my video is 90,000 intervals per second. (An amazingly large number, in my opinion, and I freely admit I'm not sure why it is not something more standard like 30 or 24. I'm sure I'll learn about this eventually.) Let's use this information to script out a button titled "+10 seconds" that will move the playhead forward 10 seconds:

on mouseUp
   put the timeScale of player "Player" into ts
   put the currentTime of player "Player" into t
   put t+(ts*10) into t
   set the currentTime of player "Player" to t
end mouseUp

I'm using two local variables here: ts (short for timeScale) and t (short for time).

The first line puts the timeScale property of the movie into ts. In my case, it will be 90,000. So, each second will last for 90,000 intervals. The second line notes what is the current position of the playhead in the video using the "currentTime" property and puts that value into t. Line 3 does a little bit of arithmetic. First, it multiplies ts by 10, so that we will have an interval of 10 seconds. Next, it adds that interval to t. Finally, we set the currentTime of the player to t, which of course is 10 seconds ahead.

To create a button that will skip backwards 10 seconds, let's start by copying and pasting the "+10 seconds" button and title it "-10 seconds," and then change the plus sign in line 3 to a minus sign:

   put t-(ts*10) into t

This is a good start to learning how to control video with LiveCode scripts. Don't worry, much more is to come.

If you are ready to learn more about working with video in LiveCode, here is a good LiveCode tutorial to check out. I found it very helpful to begin to learn how to script video options and techniques for me, even though I didn't actually built the project shown in the tutorial. Basically, it helped push me in the right direction.

Lloyd's Video Analysis Tool, Part 1

One of my favorite sayings these days when I talk about teaching and technology is "video is the new PowerPoint." That is, the ability to create and use video for teaching is about as easy as it was to first start using PowerPoint when it became available back in 1990. In fact, given how long PowerPoint has been around, it's hard to remember that it too had a learning curve like any other software tool. Similarly, the ability to create, edit, and -- most important -- distribute video has become very routine for many teachers. I do think that the ease of distribution, thanks mostly to YouTube, has been the true game changer. After all, we've had the ability to easily create and edit homemade videos for many years, but the tough nut to crack was how to share the video with others.

Video has also become an increasingly popular tool and approach over the past 20 years or so for educational researchers. It's very difficult to observe and notice the rich and complex information within a classroom or other learning environment in real time. Recording the event to review later when there is enough time -- and the opportunity to rewind -- makes a lot of sense. Video has also become an excellent way for teachers to reflect and get feedback on ways to improve their teaching. Video recordings of one's teaching are also becoming a requirement or condition for certification in many states, especially with the advent of edTPA. These teacher certification requirements have been the source of many conversations between myself and teacher education faculty at the University of Georgia. Many of these faculty not only want to learn how to meet the requirements of edTPA, but also how to use video as the means of providing formative feedback to pre-service teachers. These conversations inevitably end with the question "Is there a video analysis tool out there we can use to give our candidates quick and useful feedback on their teaching?"

Fortunately, there is a commercial tool available that is also very affordable. The product is called VideoWurks and it is produced by a company named Instructive Insight. The tool was originally created at the University of Georgia under the leadership of Dr. Art Recesso. I highly recommend taking a look at this tool. I think many teacher educators will find it meets their needs quite well. I also just learned from the Chronicle of Higher Education's Profhacker blog of a new online tool called VidBolt that allows you to annotate videos on the fly with some very cool community features.

But, I have found few other video analysis tools for educational purposes available (my google search did find some for analyzing sports-related activities, such as baseball and golf swing analyzers). So, I have begun to create my own video analysis tool with LiveCode. Before starting this project, I had only explored a minimal amount of LiveCode's ability to work with video. So, I think this is a perfect project for learning more.

I have invested about seven hours in the prototype so far, with -- believe it or not -- three of these devoted to graphic design and interface design. (Despite what the design literature says and my own experience, it always surprises me how much time is devoted to user design issues.) Here is a YouTube demonstration of the current prototype:

I'm sure you are thinking, "Man, this prototype has a long way to go!" Well yes, yes it does! You might also doubt that I've spent three hours on its graphic/interface design. So, for comparison purposes,  here is a screen shot of the very first version of this prototype:

See what I mean?

I'm particularly interested in designing this tool to be very simple to use, to the point of using technologies or strategies that might be considered out-of-date. For example, I have this idea that it would be great to have the output of the video analysis be a simple text file that could be easily shared between candidate and professor, something each could even send to the other within the body of an email. (I don't know about you, but I have found myself organizing and archiving my professional "life" with email, particularly in my ability to search for past emails of important events and activities.) If you've read any of my other posts, you know I'm also very fond of off-loading difficult tasks to other applications whenever possible, such as Excel. So, my design here is again based on creating a comma-delimited text field with all of the time code, tags, and comments for each of the video log entries for each sharing and importing within tools like Excel.

I hope this post gave you a good overview of the project at its earliest point. I'm deliberately not giving any attention to LiveCode here, but instead I will be writing many more posts, each with a focus on some interesting LiveCode programming detail, all in the spirit of helping others begin their own video projects.

Postscript: People have asked me to explain a bit about the video I used in my prototype. This video is a short "mash-up" of video snippets from my MOOC "Statistics in Education for Mere Mortals" offered by Canvas.net. The current course section is about to end, but I plan to offer it again with a starting date tentatively set for July 7, 2014. And, you can access all of the course videos on YouTube - here is a link to the playlist (the mash-up is the first in the list).