Lloyd's Video Analysis Tool, Part 7: Generating Reports

I added the feature of generating some simple reports to the project. I am deliberating not including a video in this posting because I will reporting in my next post on my first public demonstration of the project prototype to my college held earlier today. That posting will include a short video introducing the current prototype from start to finish. I'll include a good overview of the the reporting features.

Two Report Options

As I've written elsewhere in this blog, I'm a big fan of using Excel whenever possible to help organize and analyze data from projects. So, I created an "Excel Ready" report option that creates and saves a comma-separated value (.csv) file to the user's hard drive. Here is an example of this "table style" report already opened in Excel:

From here, one can use Excel to sort, organize, or search the data.

I also created a second report option that produces a simple text document using a "label style" format. Here's an example already opened in the Mac's TextEdit application:

This could obviously be opened instead with Microsoft Word and then edited anyway the user wishes.

Other Report Options to Consider

There is obviously much more that can be done here. For example, I plan on providing the option to produce reports that only focus on certain tags or combinations of tags. Likewise, I plan on giving users the option to include or exclude categories of information. For example, the user may just want the comments, but not the tags or the time code.

Tag Summary Report

At the bottom of both reports, you can see I added a short "Tag Summary Report." This seemed like a very obvious and useful summary of some key information. The list processing strengths of LiveCode make these sorts of summaries easy to produce. I again use the strategy of having two "shadow fields" (fields the user can't see) in combination with two scripts:

The first script scans the entire video analysis data and builds a running log of all tags in the field "tags found." After this field is generated, a second script scans it to produce the field "unique tags" on the right. The contents of this second field is then copied to the bottom of the report using yet another field not shown here titled "report bin."

Here's the code for producing the Excel-ready version (the text only version only differs in the way it outputs the data):

//Create list of unique tags

   repeat with i = 1 to the number of lines in field "tags found"

      put false into varTagFound

      repeat with i2 = 1 to the number of lines in field "unique tags"

         if line i of field "tags found" = line i2 of field "unique tags" then 

            put true into varTagFound

            exit repeat

         end if

      end repeat

      if varTagFound = false then put line i of field "tags found" into line (the number of lines in field "unique tags"+1) of field "unique tags"

   end repeat

   

   //Count the use of each unique tag

   repeat with i = 1 to the number of lines in field "unique tags"

      put 0 into varCountTags

      repeat with j = 1 to the number of lines in field "tags found"

         if line i of field "unique tags" = line j of field "tags found" then add 1 to varCountTags

      end repeat

      put varCountTags into item 2 of line i of field "unique tags"

   end repeat

   

   put the number of lines in field "report bin" into l

   put "Tags used in this analysis:"&comma&"Frequency"&return into line l+2 of field "report bin"

   put field "unique tags" after field "report bin"

   

Yes, there is a lot of nested repeating going on here! But LiveCode somehow makes it easy to conceptualize these repeat structures in ways that other programming languages don't, at least for me.

Final Thoughts

These simple reports were the last major piece of the puzzle to creating a tool that would actually do something useful. Even if I stopped working on the project right now, I have a tool (albeit crude) that does some useful work. This is a very important milestone. Heck, I might almost be ready to declare that the project has moved from beta version 2 to beta version 3!

Lloyd's Video Analysis Tool, Part 6: File Management

I have a brother-in-law in the construction business. He talks about working with clients who get very happy and excited when they see wall studs go up or an old roof being rapidly torn off.  In contrast, they can get impatient when he is working on plumbing or electrical jobs because they don't see much visual progress, or VP. This is frustrating to him because he is working just as hard and putting in just as many long hours. It's just that you can't see at a glance the progress he's made. Well, I've been working on some of the "plumbing" for my video analysis tool project lately and though I've been working hard on important stuff, there's not much VP.

Up to this point, I've been using the same movie for all of my project development - my mash-up video of my MOOC on statistics. The movie file was "hard wired" into the project, that is, the file name and path were manually entered and saved into the video player tool. This meant that this movie, and only this movie, would play. So, I've finally updated the project (now up to version 0.2.3) to allow any movie to be used and -- drum roll please -- to allow multiple projects to be saved and edited. This is like having hot and cold water on the construction site! Here is a brief movie demonstrating this latest version:

Obviously, I continue to be largely ignoring the graphic design of the tool. However, I think the user interface design works pretty well.

Saving Project Files

Although I have worked with scripts that save data in other LiveCode projects, I have not worked on a LiveCode project where one has the option to create and then subsequently edit new project files. Before moving forward, I had to decide on the design for this capability. I saw two possibilities. The first is what I consider the standard approach, common in applications such as Word or PowerPoint, where the user creates a new file, then saves it wherever they wish on their computer. It is up to the user to remember where the file is kept (though the "Open Recent" options help the user to quickly recall and find the most recently worked on files). The second approach is more common in mobile apps where the app itself manages all of the stored files. Evernote is a good example of this approach. I decided to go with this second approach, at least for now, particularly because I envision this tool being used on an iPad.

Project Workflow

The ability to create and then work on as many projects as one wishes required a major new design feature related to project workflow. I envisioned three possibilities:

1. Continue working on the same project as before;
2. Start a new project;
3. Select a previously saved project to work on.

I created a new card, which is now the first (or home) card of the stack, to provide these options:

Managing the Project Files

Actually, all of the project files are just text files. For now, I save all these project files into the "Documents" folder of the computer. Macintosh, Windows, iOS, and Android operating systems all have a special documents folder that can easily be accessed using this LiveCode command:

set the defaultFolder to specialFolderPath("Documents")

The defaultFolder property specifies the folder that all files and folder options used by the LiveCode file will point to. The specialFolderPath function lets you quickly point to one of several system folders, such as the documents folder.

I think the documents folder is the best choice for saving project files like these on mobile devices. However, on laptops or desktops it would probably be better to just have the files be saved into the same folder as the application itself. So, I may change the folder path later just for versions of the project meant for laptop/desktop computers.

I also needed a way to guarantee that the file names would be unique, so I created a counter for the number of projects created to date using a file name structure as follows: "VATproject"+{project number}

Contents of a Project File

Here is an example of the contents of a project file titled "VATproject43.txt":

Project Number,43
Project Name,Introduction to z Scores
/Users/lloydrieber/Documents/MOOCs/Presentations/Descriptive Statistics Module/03-z Scores/DescriptiveStatistics-z-Scores.mp4
VATLOG
1,Using Sunny in Example,8070000,8520000,dog,example
2,z Score Formula,32529000,32979000,formulas
3,Girl Scout Example,44334000,44784000,cookies,example
4,Comparing z Scores,78900000,79350000,example
VATCOMMENTS
VATComment,4
An interesting example of how z scores can be used to compare two different distributions.
VATComment,2
The formula is not complicated.
VATComment,3
This is a good context for an example.
VATComment,1
A cute example, especially if you like dogs.
VATTAGS
computation
cookies
dog
example
formulas

Commas are used throughout to separate items on each line. Here are some examples of important, specific information:

• Item 2 of line 1 contains the project number;
• Item 2 of line 2 contains the project name given to it by the user;
• Line 3 contains the path to the video.

Notice the sections that come next, each with a special header: VATLOG, VATCOMMENTS, and VATTAGS. I'm sure you can easily decipher the information that each contains. The number after each comment links the comment to that same numbered video entry. The tags for each video entry are contained at the end of each entry.

Parsing the Project File into the Project Elements

Because there is no limit to how many video entries the user identifies I had to build this text file in such a way as to allow it expand or contract over time. Basically, the script asks LiveCode to scan each line until that header is found, then it knows that all information from the next line down to the next header contains all of the information for that section. For example, here is the code that looks for comments:

   repeat with i = 1 to the number of lines in field "project data"

      if line i of field "project data" = "VATCOMMENTS" then 

         put i+1 into starti

         exit repeat

      end if

   end repeat

   put empty into tempdata

   put 0 into ii

   repeat with i = starti to the number of lines in field "project data"

      add 1 to ii

      if line i of field "project data" = VATTAGS then exit repeat

      put line i of field "project data" into line ii of tempdata

   end repeat

   put tempdata into URL ("file:currentprojectcomments.txt")

   

The first repeat loop looks for the text string "VATCOMMENTS." The second repeat loops takes each line thereafter, up to the point of finding the text string "VATTAGS," and stores it in a temporary local variable called "tempdata." When done, the contents of tempdata are transferred into a text file called "currentprojectcomments.txt." While working on a project, I save the video log, comments, tags, and basic project data (e.g. project name, file path to the movie) in separate text files all with the prefix "currentproject...". Then, I combine these files into one long file and save it with a unique project name. When the user chooses to work on an existing project, this single file is then broken apart and saved into the separate "currentproject" files. I split and recombine this information continuously as the user works on the project.

I debated whether to include some sort of auto-saving, but I wasn't confident I could do this in a way that would be reliable. So, I decided to give the user the option to save manually while working on the project and also save the project automatically when the user chooses to go back to the home card.

I will admit to having some difficulty early on in writing this script. My earlier drafts of this script used the keyword "after" instead of "into" when putting data into a particular line of tempdata. I couldn't get the data stored as expected and if the data gets slightly "out of kilter," nothing will work. So, I gave up on that more elegant and simple (potentially) script in favor of a script that made sure I knew where each line was being saved.

Things to Work on Next

These file management properties of the current version are important improvements, but there are some obvious things that need to be done. First, there are several minor, yet important features that I have not yet tackled:

• Allow the user to edit the tags assigned to a clip;
• Allow the user to edit the project name;
• Error trapping: Catch for the possibility that the user has moved the movie and give the user the opportunity to "find" the movie and re-establish the path.

Also, I need to begin work on the feature of creating reports for each video analysis.

So, I'll focus on these in the next version or two.

Lloyd's Video Analysis Tool, Part 5: Major Improvements in Version 0.2.0

I have made good progress on my video analysis tool. So much so that I feel upgrading its status to "version 0.2.0" is warranted. Here are the major tasks I've tackled:

• Users can now add as many tags as they wish for each video clip;
• Users can now add a comment that is as long as they wish, extending over as many lines (i.e. paragraphs) as they wish.
• Users have the option of adding a brief description for each clip.
• I created a video clip dashboard to display all important important about the selected video clip.

I'll address each of these improvements in this blog post. First, here is a short movie of me demonstrating this prototype:

Please note that I won't be providing any LiveCode files to download as I did in previous posts for this project. For each of those posts, I had created a second and unique version of the project to make clearer the various LiveCode skills and principles I was demonstrating. The project has now progressed to the point where continuing to update and share this second LiveCode file is too time prohibitive. However, I'll continue to share important code where appropriate. And, I think those previous files will help anyone just getting into the use of video within LiveCode.

Attaching Multiple Tags for Each Video Clip

In the previous versions, the user could only add one tag per clip. I knew all along this would be insufficient. Of course, any one video clip could have many tags or no tags, so I needed to account for an uneven number in the design. I debated on the best way to approach this, deciding in the end to just have the tags continue to be added as separate items in the field "log." Doing so created a dilemma about where to put the user comments as they had been also part of this field. I decided to create a separate field for "project comments." This opened up other possibilities for comments, which I'll discuss later in this blog posting.

In previous posts I described the basics of how the tagging system works. Recall that users can create a list of unique tags. Each tag is just a copy and paste of a template button found on the "Library" card where the name and label of the button is changed to whatever tag name the user entered. I revised the script of that template button:

global gActivity
on mouseUp
   //Check to make sure that a clip has been chosen
   if the selectedtext of field "log2" is empty then
      answer "You first have to select an entry from the log."
      exit mouseUp
   end if
   put the label of me into gActivity
   put the hilitedLine of field "log2" into l // read as "line lower case L"; assume this in remaining lines
  
   //count the number of items in the line
   put the number of items of line l of field "log" into varItemsCount
  
   //check to see if tag is already there
   repeat with i = 5 to varItemsCount
      if gActivity = item i of line l of field "log" then
         set the hilitedLine of field "log2" to l
         exit mouseUp
      end if
   end repeat
   put the label of me into item varItemsCount+1 of line l of field "log"
  
   //Update the display of tags
   put "Tags: " into varShowTags
   put the number of items in line l of field "log" into varItemsCount
   if varItemsCount<5 then put "Tags: No tags for this clip" into varShowTags
   //all tags begin with item 5
   repeat with i = 5 to varItemsCount
      put varShowTags&item i of line l of field "log" into varShowTags
      if i<varItemsCount then put varShowTags&", " into varShowTags
   end repeat
   put varShowTags into line 1 of field "Tags"
  
   set the hilitedLine of field "log2" to l
  
end mouseUp

The commented lines show the main parts of this script. It begins by checking to see if the user has chosen a clip and if not, it reminds the user to do so, followed by the command "exit mouseUp." This command tells LiveCode to do just that, exit the script triggered by the mouseUp without executing any other of the lines that follow. I highlighted this line in red. (Notice that I use this command again later in the script.)

The next task is to see if the user has already chosen that tag because we obviously don't want duplicate tags listed for a clip. First,  I check to see how many items are in the highlighted line of field "log" and store this number in the local variable "varItemsCount." That number will be used for the upper limit of a repeat loop coming up, plus we will use it to determine where to put the next unique tag.

Next is the repeat loop, triggered by this line:
repeat with i = 5 to varItemsCount

The local variable i increments by one in each loop, starting with 5 and ending with the value in varItemsCount. I start with 5 because I know the first four item slots are already taken. Item 1 now contains a clip number and item 2 now contains a short description of the clip -- two things I address briefly below. Items 3 and 4 contain the starting and ending time code of the clip in user friendly terms (this was the subject of my most previous post). This repeat loop then checks to see if the tag button just pressed matches any of the tags listed in items 5 and onward. If it finds a match, it exits the mouseUp script. If it doesn't, it adds this tag as the very next item (i.e. varItemsCount + 1).

The next block of script updates the display of the tags in the video clip dashboard. This dashboard is a new feature. Let's address that next.

Video Clip Dashboard

One of the big improvements in version 0.2.0 is the video clip dashboard. Recall that I included some screen placeholders for this video dashboard in the previous versions of the prototype. Well, I finally tackled the implementation of the dashboard here.

I created four fields for the dashboard titled "time code," "Description," "tags," and "display comment." These four fields are populated with the respective information. To populate the tags field, I use another repeat loop to create a string variable called "varShowTags." This contains a list of all the tags with a comma and space in-between, all concatenated as the repeat loops works its magic.

The script ends by highlighting the video clip line in the field "log2."

Updating the Video Clip Dashboard When the User Selects a New Clip

Let's continue to look at the video clip dashboard. When the user clicks on a video entry in the field "log2," the dashboard immediately updates with the information for that clip using the following script attached to the field itself:

on mouseDown
   if the number of lines in me is 0 then exit mouseDown
   stop player "player"
   set the playSelection of player "player" to true
   put the hilitedLine of me into l
   set the currentTime of player "player" to item 3 of line l of field "log"
  
   displayClipInformation l
   displayComment l
end mouseDown

Yes, that's right, you can treat a field like a button and have events triggered if it is clicked. This short script does a lot, thanks to two new commands I created titled "displayClipInformation" and "displayComment." Commands are close relatives to functions. The idea is the same -- you are inventing your own vocabulary and adding them to LiveCode's dictionary. The difference is that a command simply does something without returning any value like a function would (review my description of functions in my previous post). The command displayClipInformation does some magic to populate the fields "time code," "Description," and "tags." The function displayComment is used to populate the field "display comment." Both functions are shown below in their entirety, but I won't try to describe them in any depth here.

Improvement to User Comments

Let's address the design behind the improvement to user comments. I knew I needed to give the user the opportunity to make in-depth comments about any of the clips. This would have to include a basic narrative structure using sentences and paragraphs, along with standard narrative symbols, such as commas and periods. It was clear that the only way to do this was by creating yet another field titled "Project Comments" to store the comments. The tricky part is connecting the right comment to the correct video clip. Since any one comment would potentially contains multiple lines, perhaps dozens, some sort of indexing system would be needed beyond just using the line number of the clip in the field "log2."

So, I used an approach that is adapted from how relational database work. I merely created an indexing system where each clip gets a unique number. I then put the term "VATComment" as item 1 on a line and the index number as item 2 on the very same line. That way, every time the user clicks on a video entry LiveCode searches the "Project Comments" field for that term as item 1 on a line. If it finds it, it then notes the index number. All lines after that and up to the next time the term VATComment is found (or the end of the field is reached), is assumed to be the entire comment for that video clip. Pretty clever if I say so myself.

I also had to design a way for a user to update a particular comment. The best strategy I came up with was to create yet another command to first "seek out and destroy"all lines associated with a particular comment when the user updates a comment, followed immediately by creating a new set of lines for the updated comment. In a sense, every update is really a new comment, as far as LiveCode is concerned. This command is titled "deleteComment" and I've also listed the entire script below. 

Final Thoughts

I also wrote a bunch of other small scripts here and there to have everything run smoothly and to catch user errors. It's really amazing the number of situations one must address when trying to anticipate a user's interaction.

One thing I have not yet done is program a way for the user to delete tags. That will come in the next version of the program. I'm currently mulling over a few ways of doing this, so you'll have to check back later to see how I decide to tackle this issue. Also, I think it is high time I address the need to let the user select any video for analysis. Yes, it is time to put my placeholder video to rest. But, you can always view it here if you miss it.

---

Appendix: Scripts for the Commands "displayClipInformation" and "displayComment"

Both scripts are defined in the card script. Both scripts take the number corresponding to the selected video clip as input.

command displayClipInformation videoEntryLine
   //Display entry information
   put videoEntryLine into l
   put item 2 of line l of field "log" into field "Description"

   //Construct time code string
   put the timescale of player "Player" into ts
   put item 3 of line l of field "log" into tcStart
   put item 4 of line l of field "log" into tcEnd
   put (tcEnd - tcStart)/ts into varCliplength
   put round(varCliplength,1) into varCliplength
   put convertTimeCode (tcStart) into tcStart2
   put convertTimeCode (tcEnd) into tcEnd2
   put "Clip length: "&tcStart2&" to "&tcEnd2&" ("&varCliplength&" sec)" into line 1 of field "Time Code"
  
   //Determine tags for display
   put "Tags: " into varShowTags
   put the number of items in line l of field "log" into varItemsCount
   if varItemsCount<5 then put "Tags: No tags for this clip" into varShowTags
   //all tags begin with item 5
   repeat with i = 5 to varItemsCount
      put varShowTags&item i of line l of field "log" into varShowTags
      if i<varItemsCount then put varShowTags&", " into varShowTags
   end repeat
   put varShowTags into line 1 of field "Tags"
end displayClipInformation

---

command displayComment videoEntryLine
   put empty into field "display comment"
   put videoEntryLine into l
   put item 1 of line videoEntryLine of field "log" into videoEntryNumber
  
   repeat with i = 1 to the number of lines in field "project comments"
      if item 1 of line i of field "project comments" is "VATComment" and item 2 of line i of field "project comments" is videoEntryNumber then
         put line i+1 of field "project comments" into line 1 of field "display comment"
         repeat with j = i+2 to the number of lines in field "project comments"
            if item 1 of line j of field "project comments" <> "VATComment" then
               put line j of field "project comments" into line (number of lines in field "display comment"+1) of field "display comment"
            else
               exit displayComment
            end if
         end repeat
         exit displayComment
      end if
   end repeat
   put "No comments yet for this video entry" into line 1 of field "display comment"
end displayComment

---

command deleteComment videoEntryLine
   //put empty into field "display comment"
   put videoEntryLine into l
   put item 1 of line videoEntryLine of field "log" into videoEntryNumber
  
   repeat with i = 1 to the number of lines in field "project comments"
      if item 1 of line i of field "project comments" is "VATComment" and item 2 of line i of field "project comments" is videoEntryNumber then
         //put line i+1 of field "project comments" into line 1 of field "display comment"
         delete line i of field "project comments"
        
         repeat with j = i to the number of lines in field "project comments"
            if item 1 of line i of field "project comments" <> "VATComment" then
               delete line i of field "project comments"
            else
               exit deleteComment
            end if
         end repeat
         exit deleteComment
      end if
   end repeat
   //put "No comments yet for this video entry" into line 1 of field "display comment"
end deleteComment

Lloyd's Video Analysis Tool, Part 4: Displaying Time Code

As we have seen so far in this series on developing a video analysis tool, LiveCode has its own unique way of tracking intervals per seconds of a video. Each video can have a different number of intervals per second and this can be determined by using the timeScale property. For the video I've been using for testing purposes in my prototype, that number has been 90,000 intervals per second.

So, using this scale, the numbers 37,800,000 to 49,950,000 would mean playing the video from the seven minute mark to the nine minute 15 second mark. The way most of us are used to seeing time code expressed is in this form: hours:minutes:seconds, or...

00:07:00 to 00:09:15

Unfortunately, LiveCode does not have a function to show the time code in this way. But, there is nothing stopping us from inventing our own function to do this. Writing custom functions is a very powerful feature of most programming languages and LiveCode is no exception. Think of it as adding to LiveCode's dictionary. Just as any human language naturally grows with use over time, so too can a computer programming language. So, we'll use this problem as an opportunity to explore how to create functions in LiveCode. I'll name this new function as "convertTimeCode." You can name functions anything you want, assuming you don't pick the name of an already existing function (some programmers add a special prefix or suffix, such as their initials, to each unique function they create to ensure no existing functions names are inadvertently used).

So, what does a function look like in use? Well, we've been using LiveCode built-in functions all along in this blog. The absolute value function is a good example. Consider the following:

   abs(-109)

This returns the number 109. Or, you could say it converts -109 to its absolute value.

Consider these two examples:

abs(598-601)
abs(601-598)

Both return a value of 3. Now consider this block of code:

put 434 into varFirst
put 822 into varSecond
abs(varFirst-varSecond)

This will return a value of 388.

Similarly, let's imagine I've already created the new function "convertTimeCode" and entered the following as part of some scripts:

   convertTimeCode(49950000)
   convertTimeCode(324990000)

These each return values, respectively, of "00:09:15" and "03:00:11".

(Of course, these two examples assume a timeScale value of 90,000; remember, the timeScale value can be different depending on the video. So, we just need to always check for the timeScale when writing our scripts.)

Creating Our Own LiveCode Function

The complete script of the function I created is at the bottom of this post. I'll explain briefly how it works in a moment, but here is its basic construction:

function convertTimeCode ct
     //all of the script that makes the function work goes here
     return myTimeCode
end convertTimeCode

The number that is entered in parentheses when the function is called is put into "ct," a local variable that I get to name. (This was meant to be short for "time code," but I guess I got the letters backwards when I wrote the function!) The script that makes the function work eventually put the final value of the time code into a variable called "myTimeCode." Its value must look like the time code described above (e.g. 01:34:14).

I don't think a full explanation of how this function actually works is needed here. In short, the function takes the number of intervals and converts it to seconds. This is then divided by 3600 -- the number of seconds in an hour -- to figure out how many hours there are. The remainder is then divided by 60 -- the number of seconds in a minute -- to figure out how many remaining minutes there are. The remainder is the number of seconds. I relied on LiveCode's trunc function (short for truncate) for these steps. When you have numbers like 29.35 or 31.78, the trunc function strips away everything to the right of the decimal point, leaving just the integer, such as 29 and 31. No rounding occurs.

Along the way, the script adds extra zeros if and when they are necessary. For example, if there are only 9 minutes for the middle part of the time code, an extra zero is added in front of the 9 (e.g. "09"). Finally, colons are added where needed.

Figuring all this out was just a fun little logic puzzle for me. I'm sure there are probably more elegant ways to do this, but hey, it works.

Displaying the Time Code to the User

Great, we have a function that converts LiveCode's intervals into time code that a person can interpret. Now, what should we do with it? We need to display this time code and have the user interact with it. Of course, this nicely formatted time code isn't what LiveCode uses to actually play video snippets. So, we will continue to need the original video interval information. As in many of my previous LiveCode projects, the strategy here once again points to the use of what I call "Shadow Fields." These are fields that are actually hidden from the user's view, but do work behind the scenes.

I've copied and pasted the field "log" and named this new field simply "log2." I'll put all of the time code data inside log2 in parallel with the interval data in the original log. For now, I'm displaying both so you can see the parallel output, though I'll eventually hide the field "log" (the field "log2" is on the left and the field "log" is on the right):

 LiveCode file

[ Get the free LiveCode Community version. ]

As you can see, the two lines of the two fields match, or correspond, to each other. All I need to do now is focus the user's interaction on log2 instead of log. But, I have to be careful to use the original log when sending instructions to the movie player. This is easily done because I can refer to the same line number in each field. I just have to be careful when I reference log2 versus log. As an example, here is the revised script for the "Update Start" button:

on mouseUp
   put the hilitedLine of field "log2" into l //note: this is a lower-case L
   put the currentTime of player "player" into ct
   //error trap - starting point must be before ending point
   if ct > item 2 of line l of field "log" then //note: this is a lower-case L
      answer error "Not Allowed! Starting point must come before the ending point!" with "OK"
      exit mouseUp
   end if
   put ct into item 1 of line l of field "log" //note: read this as "into item one of line [lower-case L]"
   put convertTimeCode(ct) into item 1 of line l of field "log2" //ditto
   set the hilitedLine of field "log2" to l //note: this is a lower-case L
end mouseUp

First, notice my use of the convertTimeCode function in the second-to-last line of the script (not counting the "on mouseUp" and "end mouseUp" lines). (Note: Sorry the differences between the number one and the lower-case L are not more readily apparent. I hope my notes in red make this clearer.)

Second, notice how I switch back and forth between referencing log2 versus log. I do the same in the other buttons in the blue editing zone.

One more quick enhancement: Adding an Alert Box

Every time a new entry is added to the log, the previously selected entry is un-highlighted. This causes a problem if the user then immediately clicks on the play button in the editing zone. So, we need to communicate a reminder to the user that they first need to highlight one of the video entries first. We could dim out the play button if nothing is highlighted (which, frankly, sounds like the way to go), but for now I've just added a little alert box in the button "Play entry." Here's the script I added to that button:

   if the selectedtext of field "log2" is empty then
      answer "You first have to select an entry from the log."
      exit mouseUp
   end if

Things are coming along very nicely for this video analysis tool project.

---

Appendix

Here's the script for my "convertTimeCode" function - impressive, isn't it?

function convertTimeCode ct
   put the timescale of player "Player" into ts
   //Convert intervals into seconds
   put round((ct)/ts) into s
   //Determine number of hours and subtract that number of seconds from total
   if s<3600 then put "00" into myTimeCode
   if s>=3600 then
      put trunc(s/3600) into h
      put s-(h*3600) into s
      if h<10 then
         put "0"&h into myTimeCode
      else
         put h into myTimeCode
      end if
   end if
   //Determine number of minutes
   if s<60 then put myTimeCode&":00" into myTimeCode
   if s>=60 then
      put trunc(s/60) into m
      put s-(m*60) into s
      if m<10 then
         put myTimeCode&":0"&m into myTimeCode
      else
         put myTimeCode&":"&m into myTimeCode
      end if
   end if
   //Determine number of seconds
   if s>=10 then put myTimeCode&":"&s into myTimeCode
   if s<10 then put myTimeCode&":0"&s into myTimeCode
   return myTimeCode
end convertTimeCode

Lloyd's Video Analysis Tool, Part 3: Building a Log of Video Clips

OK, now that we have a command of some of the basics of using video within LiveCode, let's begin to explore how to build some custom features for the video analysis tool. A key element of the prototype of my video analysis tool is giving the user a way to log a series of video clips. Then, each clip can be played back, tagged, and commented on.

Here's a screen shot of the LiveCode file we will be building:

 LiveCode file

[ Get the free LiveCode Community version. ]

 

As you can see, I widened the stack (to 1024 pixels, the landscape width of an iPad) and I moved the elements from the previous version toward the left. On the right is a scrolling list field that contains an ever-growing list of video clips. The idea is to let the user define a starting and ending point for as many clips as they wish. Each can then be played back at will.

Let's start by adding a scrolling list field on the right hand side of the card and title it "log." Be sure it is the "scrolling list field" and not a "scrolling field." Hover and pause over the various list objects in the tool palette to see the object names appear. I'll explain later the difference

Next, add a button titled "New Entry" and add this code to it:

on mouseUp
   put the number of lines in field "log" into l
   //start of video snippet
   put the currentTime of player "player" into item 1 of line l+1 of field "log"
   //end of video snippet
   put the timescale of player "Player" into ts
   put the currentTime of player "player" +(ts*5)  into item 2 of line l+1 of field "log"
end mouseUp

The first line looks at how many lines in the field "log" have content and puts that value into a local variable "l" (short for log). When the field empty, the result will be 0. Our goal will be to add content for the next clip after that line.

Next we need to define a video clip. We will take what we learned in the previous blog post to build the starting and ending points of the clip. The third line notes the current frame of the video player and then puts this as the first item of the first empty line of field "log" (i.e. l+1). I want the default length of the video clip to be 5 seconds. To do this, I must first check to see what the timescale of the movie is, and line 5 accomplishes this and puts the value in a local variable "ts." Line 6 multiples ts by 5, adds this amount to the current position of the video playhead (as denoted by the property currentTime), then puts the result also into line l+1, but as item 2 (LiveCode will automatically add a comma to partition the two items).

Great! We now have a starting and ending point for the video clip. Try it out by moving the playhead to different locations and pressing the "New Entry" button. It should add a starting point at the position of the playhead and an ending point five seconds later. A new video snippet will be added to each successive line in the field.

I think it would be useful to add a button that clears out all the log entries. This will be particularly helpful as we test out the prototype. Add a button called "Clear Log" with this one line of code:

on mouseUp
   put empty into field "log"  
end mouseUp

Notice that I put this button in the upper right hand portion of the screen. I didn''t want it next to the other log buttons so as to avoid inadvertently pressing it and deleting all the entries by mistake.

Let's now create a button that plays that video snippet. Title it "Play Entry" and place it just below the "New Entry" button. Here is the code for it:

on mouseUp
   set the playSelection of player "player" to true
   set the startTime of player "player" to item 1 of the selectedtext of field "log"
   set the endTime of player "player" to item 2 of the selectedtext of field "log"
   start player "player"
end mouseUp

The first line is important because the property "playSelection" is set to false by default. So, what is this property? If it is true, then the player will start playing the video and not stop until the end of the entire movie has been reached. Of course, we want it to stop when the playhead reaches the final frame of the clip. Note that you can change this property also by going to the basic properties page of the player object's inspector (i.e. "Play selection only"), but I think it's preferable to code this directly into the button.

The next two lines set two important player properties. As their names suggest, each refers to the starting and ending position of a video (or audio) clip. Therefore, we set the property "startTime" to item 1 of the respective line of the log field, and "endTime" to item 2.

Take a few minutes now to add three or four log entries. Each will be five seconds in length. Click on the respective line in the log field and press the "Play Entry" button. That entry should play and end in five seconds.

Please also take a moment now to recognize the difference between a scrolling list field and a regular scrolling field. When the user clicks a line in a scrolling list field, the entire line is selected, as compared to placing the cursor at a certain location within that line. This difference allows us to use the function "selectedtext"to select the entire line. A subtle, but important and very useful, difference.

Giving Users the Option to Update an Entry

All is working great, but obviously a five second clip will unlikely be satisfactory to the user. So, the next step is to give the user the option to update the beginning or ending points of the clip. The design is straightforward. Have the user click on an entry and give them the option to move the playhead anywhere they want, then give them the buttons "Update Start" or "Update End" to update the clip. Let's add both of these buttons now.

Create a button named "Update Start" and add this script to it:

on mouseUp
   put the hilitedLine of field "log" into l
   put the currentTime of player "player" into ct
   //error trap - starting point must be before ending point
   if ct > item 2 of line l of field "log" then
      answer error "Not Allowed! Starting point must come before the ending point!" with "OK"
      exit mouseUp
   end if
   put ct into item 1 of line l of field "log"
   set the hilitedLine of field "log" to l
end mouseUp

Line 1 uses a cool property called "hilitedLine." As the name suggests, this property takes note of what line has just been highlighted in a text field. Line l puts this line number into a local variable "l" (this is a lower case "L" remember, not the number one).

Line 2 notes the current position of the video playhead and puts it into a local variable "ct." (Recall that the user would have moved the playhead before clicking this button.)

The next several lines do some error trapping, as we humans are very prone to errors. We'll come back to these in a moment.

Let's jump to line 8. This line puts the value of "ct" into the selected line of field "log," replacing whatever value was already there. Well, of course, the idea of this button is to update the entry after all.

The last line highlights the line in the log we just edited. This is important from a user design standpoint. Without this line, the user would have to again click on that line if they wanted to make another update. I think it's easy to imagine the user making lots of updates in a row until they get things "just right." (Tha'ts what I found myself doing as I tried out the prototype.) So, this line highlights the line for them.

OK, what about that error trapping? Think about it for a moment. We have to ensure that the starting point of the video clip is a number less than the ending point of the clip. So, line 4 starts an if/then statement to check this. If "ct" (the new position of the videohead) is greater than the ending point of the video clip, then that's a problem. So, LiveCode pops up with a message saying something to the effect that you can't do that, using the handy "answer" command. The next line - "exit mouseUp" terminates button's script, so nothing else happens. (Alas, error trapping is a very time-consuming, but necessary part of all software design. One has to be very cunning to determine where human error is likely to occur. This is very hard to predict and is one of the valuable outcomes of extensive field testing.)

Next, add another button and title it "Update End." As you might suspect, the script for this button is very similar to the button "Update Start":

on mouseUp
   put the hilitedLine of field "log" into l
   put the currentTime of player "player" into ct
   //error trap - ending point must be after ending point
   if ct < item 1 of line l of field "log" then
      answer error "Not Allowed! Ending point must come after the starting point!" with "OK"
      exit mouseUp
   end if
   put ct into item 2 of line l of field "log"
   set the hilitedLine of field "log" to l
end mouseUp

In fact, you could just copy and paste the "Update Start" button and make just a few, small (but critical) revisions. For example, in line 8, the only revision is changing item 1 to item 2.

The only other changes are in the error trapping. In this case, we have to be sure that the ending point comes after the starting point. Here is a quick summary of the changes to these lines of code:

• Change the "greater than" sign to a "lesser than" sign;
• Change "item 2" to "item 1"; and
• Change the text of the message as needed.

Finally, I added a blue rectangle to more appropriately show this "editing zone." To move the rectangle "to the back" go to the "size and position" page of the property inspector and change its layer.

Revising the +10 and -10 Second Editor

Let's do one more thing. Recall from my previous blog posting that we created two buttons that advanced or rewound the playhead by 10 seconds. An increment of 10 seconds was a good choice there to demonstrate this feature, but it really is not a useful increment for "fine tuning." So, let's revise those two buttons so that the increment is only one second. (One could argue that even a smaller increment, such a half or tenth of a second, would be even better.) Making this revision is a snap.

Let's take a look at the existing script for the "-10 seconds" button:

on mouseUp
   put the timescale of player "Player" into ts
   put the currentTime of player "Player" into t
   put t-(ts*10) into t
   set the currentTime of player "Player" to t
end mouseUp

All we need to do is change the 10 to a 1 in line 3. (You might also argue that we could just delete the "*10" altogether, but I don't recommend that because we might find ourselves coming back and changing this later to a different value, such as .5.

Do the same thing for the "+10 seconds" button and, of course, don't forget to change the button titles.

Final Thought

My final thought is simply "Hey, pretty cool!" We have a good start to an interesting video analysis tool.

In my next post, I'll focus solely on how to convert or transform LiveCode's time code intervals to something we humans can better understand, such as hours:minutes:seconds.

Lloyd's Video Analysis Tool, Part 2: Some Video Basics

In this post I'll explore a few of the basics of bringing video into a LiveCode project. Here is screen shot of the stack we'll build:

 LiveCode file

[ Get the free LiveCode Community version. ]

 

You'll obviously need your own movie to use in this project.

The first step couldn't be easier. Just drag a player object onto a card:

The player object's name defaults to "Player," but feel free to rename this as you wish. (But, note that we will be referring to the player object's name in the scripting below, so decide on the name now.) I'll leave it named as "Player." A good next step is to explore the properties of the player object by browsing the various pages of the object's inspector window.

Source video files can either be stored on your computer, or you can enter the URL of the source video. There are some limitations on what video file types can be used. According to the LiveCode manual, video is limited to QuickTime, AVI, or MPEG file types. So, unfortunately and much to my disappointment, flash video is not a supported format (and hence, one cannot use YouTube videos.)

For many projects, this is all you need to know! The player object will now play your video and it includes all of the expected video options, such as play, pause, step forward and backward, audio control, and user control of the playhead.

Note: If you choose to resize your video, be sure to check the box beside "lock size and position" inside the size and position properties page. Otherwise, it will appear full size the next time you open the file.

Video-Related Scripting

To create a more sophisticated video application, such as the video analysis tool I have in mind, we need to explore how to script various video functions. Let's explore just a few of the most basic here. A good place to start is scripting our own play and stop buttons. The scripts for each button are very simple:

on mouseUp
   start player "Player"
end mouseUp

on mouseUp
   stop player "Player"
end mouseUp

In my earliest Video Analysis Tool prototype, I've already included the option to move the video playhead forward or back by one second. Let's explore how to do that as our next script. Instead of one second, let's use an increment of 10 seconds in order to really notice the playhead skipping ahead or back.

To do this, we first have to note where the playhead is currently in the video. Then, we'll just add or subtract 10 seconds worth of video and jump to that new spot. Let's say we stopped the video one minute and 15 seconds into the video. LiveCode does not "see" the video in seconds or minutes. Instead, it will play a particular video in a certain number of intervals per second. This interval will vary from video to video. Fortunately, we can use the timeScale property to learn what this interval is. Let's open the Message box (an option under "Tools") and type in this script:

   put the timescale of player "Player"

After you press Return, a number will appear in the message box. For the video I'm using, the number is 90,000. Yours will most likely be different. So, this means that for my video, the time scale of my video is 90,000 intervals per second. (An amazingly large number, in my opinion, and I freely admit I'm not sure why it is not something more standard like 30 or 24. I'm sure I'll learn about this eventually.) Let's use this information to script out a button titled "+10 seconds" that will move the playhead forward 10 seconds:

on mouseUp
   put the timeScale of player "Player" into ts
   put the currentTime of player "Player" into t
   put t+(ts*10) into t
   set the currentTime of player "Player" to t
end mouseUp

I'm using two local variables here: ts (short for timeScale) and t (short for time).

The first line puts the timeScale property of the movie into ts. In my case, it will be 90,000. So, each second will last for 90,000 intervals. The second line notes what is the current position of the playhead in the video using the "currentTime" property and puts that value into t. Line 3 does a little bit of arithmetic. First, it multiplies ts by 10, so that we will have an interval of 10 seconds. Next, it adds that interval to t. Finally, we set the currentTime of the player to t, which of course is 10 seconds ahead.

To create a button that will skip backwards 10 seconds, let's start by copying and pasting the "+10 seconds" button and title it "-10 seconds," and then change the plus sign in line 3 to a minus sign:

   put t-(ts*10) into t

This is a good start to learning how to control video with LiveCode scripts. Don't worry, much more is to come.

If you are ready to learn more about working with video in LiveCode, here is a good LiveCode tutorial to check out. I found it very helpful to begin to learn how to script video options and techniques for me, even though I didn't actually built the project shown in the tutorial. Basically, it helped push me in the right direction.

Lloyd's Video Analysis Tool, Part 1

One of my favorite sayings these days when I talk about teaching and technology is "video is the new PowerPoint." That is, the ability to create and use video for teaching is about as easy as it was to first start using PowerPoint when it became available back in 1990. In fact, given how long PowerPoint has been around, it's hard to remember that it too had a learning curve like any other software tool. Similarly, the ability to create, edit, and -- most important -- distribute video has become very routine for many teachers. I do think that the ease of distribution, thanks mostly to YouTube, has been the true game changer. After all, we've had the ability to easily create and edit homemade videos for many years, but the tough nut to crack was how to share the video with others.

Video has also become an increasingly popular tool and approach over the past 20 years or so for educational researchers. It's very difficult to observe and notice the rich and complex information within a classroom or other learning environment in real time. Recording the event to review later when there is enough time -- and the opportunity to rewind -- makes a lot of sense. Video has also become an excellent way for teachers to reflect and get feedback on ways to improve their teaching. Video recordings of one's teaching are also becoming a requirement or condition for certification in many states, especially with the advent of edTPA. These teacher certification requirements have been the source of many conversations between myself and teacher education faculty at the University of Georgia. Many of these faculty not only want to learn how to meet the requirements of edTPA, but also how to use video as the means of providing formative feedback to pre-service teachers. These conversations inevitably end with the question "Is there a video analysis tool out there we can use to give our candidates quick and useful feedback on their teaching?"

Fortunately, there is a commercial tool available that is also very affordable. The product is called VideoWurks and it is produced by a company named Instructive Insight. The tool was originally created at the University of Georgia under the leadership of Dr. Art Recesso. I highly recommend taking a look at this tool. I think many teacher educators will find it meets their needs quite well. I also just learned from the Chronicle of Higher Education's Profhacker blog of a new online tool called VidBolt that allows you to annotate videos on the fly with some very cool community features.

But, I have found few other video analysis tools for educational purposes available (my google search did find some for analyzing sports-related activities, such as baseball and golf swing analyzers). So, I have begun to create my own video analysis tool with LiveCode. Before starting this project, I had only explored a minimal amount of LiveCode's ability to work with video. So, I think this is a perfect project for learning more.

I have invested about seven hours in the prototype so far, with -- believe it or not -- three of these devoted to graphic design and interface design. (Despite what the design literature says and my own experience, it always surprises me how much time is devoted to user design issues.) Here is a YouTube demonstration of the current prototype:

I'm sure you are thinking, "Man, this prototype has a long way to go!" Well yes, yes it does! You might also doubt that I've spent three hours on its graphic/interface design. So, for comparison purposes,  here is a screen shot of the very first version of this prototype:

See what I mean?

I'm particularly interested in designing this tool to be very simple to use, to the point of using technologies or strategies that might be considered out-of-date. For example, I have this idea that it would be great to have the output of the video analysis be a simple text file that could be easily shared between candidate and professor, something each could even send to the other within the body of an email. (I don't know about you, but I have found myself organizing and archiving my professional "life" with email, particularly in my ability to search for past emails of important events and activities.) If you've read any of my other posts, you know I'm also very fond of off-loading difficult tasks to other applications whenever possible, such as Excel. So, my design here is again based on creating a comma-delimited text field with all of the time code, tags, and comments for each of the video log entries for each sharing and importing within tools like Excel.

I hope this post gave you a good overview of the project at its earliest point. I'm deliberately not giving any attention to LiveCode here, but instead I will be writing many more posts, each with a focus on some interesting LiveCode programming detail, all in the spirit of helping others begin their own video projects.

Postscript: People have asked me to explain a bit about the video I used in my prototype. This video is a short "mash-up" of video snippets from my MOOC "Statistics in Education for Mere Mortals" offered by Canvas.net. The current course section is about to end, but I plan to offer it again with a starting date tentatively set for July 7, 2014. And, you can access all of the course videos on YouTube - here is a link to the playlist (the mash-up is the first in the list).