OK, now that we have a command of some of the basics of using video within LiveCode, let's begin to explore how to build some custom features for the video analysis tool. A key element of the prototype of my video analysis tool is giving
the user a way to log a series of video clips. Then, each clip can be
played back, tagged, and commented on.
Here's a screen shot of the LiveCode file we will be building:
As you can see, I widened the stack (to 1024 pixels, the landscape width of an iPad) and I moved the elements from the previous version toward the left. On the right is a scrolling list field that contains an ever-growing list of video clips. The idea is to let the user define a starting and ending point for as many clips as they wish. Each can then be played back at will.
Let's start by adding a scrolling list field on the right hand side of the card and title it "log." Be sure it is the "scrolling list field" and not a "scrolling field." Hover and pause over the various list objects in the tool palette to see the object names appear. I'll explain later the difference
Next, add a button titled "New Entry" and add this code to it:
on mouseUp
put the number of lines in field "log" into l
//start of video snippet
put the currentTime of player "player" into item 1 of line l+1 of field "log"
//end of video snippet
put the timescale of player "Player" into ts
put the currentTime of player "player" +(ts*5) into item 2 of line l+1 of field "log"
end mouseUp
The first line looks at how many lines in the field "log" have content and puts that value into a local variable "l" (short for log). When the field empty, the result will be 0. Our goal will be to add content for the next clip after that line.
Next we need to define a video clip. We will take what we learned in the previous blog post to build the starting and ending points of the clip. The third line notes the current frame of the video player and then puts this as the first item of the first empty line of field "log" (i.e. l+1). I want the default length of the video clip to be 5 seconds. To do this, I must first check to see what the timescale of the movie is, and line 5 accomplishes this and puts the value in a local variable "ts." Line 6 multiples ts by 5, adds this amount to the current position of the video playhead (as denoted by the property currentTime), then puts the result also into line l+1, but as item 2 (LiveCode will automatically add a comma to partition the two items).
Great! We now have a starting and ending point for the video clip. Try it out by moving the playhead to different locations and pressing the "New Entry" button. It should add a starting point at the position of the playhead and an ending point five seconds later. A new video snippet will be added to each successive line in the field.
I think it would be useful to add a button that clears out all the log entries. This will be particularly helpful as we test out the prototype. Add a button called "Clear Log" with this one line of code:
on mouseUp
put empty into field "log"
end mouseUp
Notice that I put this button in the upper right hand portion of the screen. I didn''t want it next to the other log buttons so as to avoid inadvertently pressing it and deleting all the entries by mistake.
Let's now create a button that plays that video snippet. Title it "Play Entry" and place it just below the "New Entry" button. Here is the code for it:
on mouseUp
set the playSelection of player "player" to true
set the startTime of player "player" to item 1 of the selectedtext of field "log"
set the endTime of player "player" to item 2 of the selectedtext of field "log"
start player "player"
end mouseUp
The first line is important because the property "playSelection" is set to false by default. So, what is this property? If it is true, then the player will start playing the video and not stop until the end of the entire movie has been reached. Of course, we want it to stop when the playhead reaches the final frame of the clip. Note that you can change this property also by going to the basic properties page of the player object's inspector (i.e. "Play selection only"), but I think it's preferable to code this directly into the button.
The next two lines set two important player properties. As their names suggest, each refers to the starting and ending position of a video (or audio) clip. Therefore, we set the property "startTime" to item 1 of the respective line of the log field, and "endTime" to item 2.
Take a few minutes now to add three or four log entries. Each will be five seconds in length. Click on the respective line in the log field and press the "Play Entry" button. That entry should play and end in five seconds.
Please also take a moment now to recognize the difference between a scrolling list field and a regular scrolling field. When the user clicks a line in a scrolling list field, the entire line is selected, as compared to placing the cursor at a certain location within that line. This difference allows us to use the function "selectedtext"to select the entire line. A subtle, but important and very useful, difference.
All is working great, but obviously a five second clip will unlikely be satisfactory to the user. So, the next step is to give the user the option to update the beginning or ending points of the clip. The design is straightforward. Have the user click on an entry and give them the option to move the playhead anywhere they want, then give them the buttons "Update Start" or "Update End" to update the clip. Let's add both of these buttons now.
Create a button named "Update Start" and add this script to it:
on mouseUp
put the hilitedLine of field "log" into l
put the currentTime of player "player" into ct
//error trap - starting point must be before ending point
if ct > item 2 of line l of field "log" then
answer error "Not Allowed! Starting point must come before the ending point!" with "OK"
exit mouseUp
end if
put ct into item 1 of line l of field "log"
set the hilitedLine of field "log" to l
end mouseUp
Line 1 uses a cool property called "hilitedLine." As the name suggests, this property takes note of what line has just been highlighted in a text field. Line l puts this line number into a local variable "l" (this is a lower case "L" remember, not the number one).
Line 2 notes the current position of the video playhead and puts it into a local variable "ct." (Recall that the user would have moved the playhead before clicking this button.)
The next several lines do some error trapping, as we humans are very prone to errors. We'll come back to these in a moment.
Let's jump to line 8. This line puts the value of "ct" into the selected line of field "log," replacing whatever value was already there. Well, of course, the idea of this button is to update the entry after all.
The last line highlights the line in the log we just edited. This is important from a user design standpoint. Without this line, the user would have to again click on that line if they wanted to make another update. I think it's easy to imagine the user making lots of updates in a row until they get things "just right." (Tha'ts what I found myself doing as I tried out the prototype.) So, this line highlights the line for them.
OK, what about that error trapping? Think about it for a moment. We have to ensure that the starting point of the video clip is a number less than the ending point of the clip. So, line 4 starts an if/then statement to check this. If "ct" (the new position of the videohead) is greater than the ending point of the video clip, then that's a problem. So, LiveCode pops up with a message saying something to the effect that you can't do that, using the handy "answer" command. The next line - "exit mouseUp" terminates button's script, so nothing else happens. (Alas, error trapping is a very time-consuming, but necessary part of all software design. One has to be very cunning to determine where human error is likely to occur. This is very hard to predict and is one of the valuable outcomes of extensive field testing.)
Next, add another button and title it "Update End." As you might suspect, the script for this button is very similar to the button "Update Start":
on mouseUp
put the hilitedLine of field "log" into l
put the currentTime of player "player" into ct
//error trap - ending point must be after ending point
if ct < item 1 of line l of field "log" then
answer error "Not Allowed! Ending point must come after the starting point!" with "OK"
exit mouseUp
end if
put ct into item 2 of line l of field "log"
set the hilitedLine of field "log" to l
end mouseUp
In fact, you could just copy and paste the "Update Start" button and make just a few, small (but critical) revisions. For example, in line 8, the only revision is changing item 1 to item 2.
The only other changes are in the error trapping. In this case, we have to be sure that the ending point comes after the starting point. Here is a quick summary of the changes to these lines of code:
Here's a screen shot of the LiveCode file we will be building:
[ Get the free LiveCode Community version. ]
As you can see, I widened the stack (to 1024 pixels, the landscape width of an iPad) and I moved the elements from the previous version toward the left. On the right is a scrolling list field that contains an ever-growing list of video clips. The idea is to let the user define a starting and ending point for as many clips as they wish. Each can then be played back at will.
Let's start by adding a scrolling list field on the right hand side of the card and title it "log." Be sure it is the "scrolling list field" and not a "scrolling field." Hover and pause over the various list objects in the tool palette to see the object names appear. I'll explain later the difference
Next, add a button titled "New Entry" and add this code to it:
on mouseUp
put the number of lines in field "log" into l
//start of video snippet
put the currentTime of player "player" into item 1 of line l+1 of field "log"
//end of video snippet
put the timescale of player "Player" into ts
put the currentTime of player "player" +(ts*5) into item 2 of line l+1 of field "log"
end mouseUp
The first line looks at how many lines in the field "log" have content and puts that value into a local variable "l" (short for log). When the field empty, the result will be 0. Our goal will be to add content for the next clip after that line.
Next we need to define a video clip. We will take what we learned in the previous blog post to build the starting and ending points of the clip. The third line notes the current frame of the video player and then puts this as the first item of the first empty line of field "log" (i.e. l+1). I want the default length of the video clip to be 5 seconds. To do this, I must first check to see what the timescale of the movie is, and line 5 accomplishes this and puts the value in a local variable "ts." Line 6 multiples ts by 5, adds this amount to the current position of the video playhead (as denoted by the property currentTime), then puts the result also into line l+1, but as item 2 (LiveCode will automatically add a comma to partition the two items).
Great! We now have a starting and ending point for the video clip. Try it out by moving the playhead to different locations and pressing the "New Entry" button. It should add a starting point at the position of the playhead and an ending point five seconds later. A new video snippet will be added to each successive line in the field.
I think it would be useful to add a button that clears out all the log entries. This will be particularly helpful as we test out the prototype. Add a button called "Clear Log" with this one line of code:
on mouseUp
put empty into field "log"
end mouseUp
Notice that I put this button in the upper right hand portion of the screen. I didn''t want it next to the other log buttons so as to avoid inadvertently pressing it and deleting all the entries by mistake.
Let's now create a button that plays that video snippet. Title it "Play Entry" and place it just below the "New Entry" button. Here is the code for it:
on mouseUp
set the playSelection of player "player" to true
set the startTime of player "player" to item 1 of the selectedtext of field "log"
set the endTime of player "player" to item 2 of the selectedtext of field "log"
start player "player"
end mouseUp
The first line is important because the property "playSelection" is set to false by default. So, what is this property? If it is true, then the player will start playing the video and not stop until the end of the entire movie has been reached. Of course, we want it to stop when the playhead reaches the final frame of the clip. Note that you can change this property also by going to the basic properties page of the player object's inspector (i.e. "Play selection only"), but I think it's preferable to code this directly into the button.
The next two lines set two important player properties. As their names suggest, each refers to the starting and ending position of a video (or audio) clip. Therefore, we set the property "startTime" to item 1 of the respective line of the log field, and "endTime" to item 2.
Take a few minutes now to add three or four log entries. Each will be five seconds in length. Click on the respective line in the log field and press the "Play Entry" button. That entry should play and end in five seconds.
Please also take a moment now to recognize the difference between a scrolling list field and a regular scrolling field. When the user clicks a line in a scrolling list field, the entire line is selected, as compared to placing the cursor at a certain location within that line. This difference allows us to use the function "selectedtext"to select the entire line. A subtle, but important and very useful, difference.
Giving Users the Option to Update an Entry
All is working great, but obviously a five second clip will unlikely be satisfactory to the user. So, the next step is to give the user the option to update the beginning or ending points of the clip. The design is straightforward. Have the user click on an entry and give them the option to move the playhead anywhere they want, then give them the buttons "Update Start" or "Update End" to update the clip. Let's add both of these buttons now.
Create a button named "Update Start" and add this script to it:
on mouseUp
put the hilitedLine of field "log" into l
put the currentTime of player "player" into ct
//error trap - starting point must be before ending point
if ct > item 2 of line l of field "log" then
answer error "Not Allowed! Starting point must come before the ending point!" with "OK"
exit mouseUp
end if
put ct into item 1 of line l of field "log"
set the hilitedLine of field "log" to l
end mouseUp
Line 1 uses a cool property called "hilitedLine." As the name suggests, this property takes note of what line has just been highlighted in a text field. Line l puts this line number into a local variable "l" (this is a lower case "L" remember, not the number one).
Line 2 notes the current position of the video playhead and puts it into a local variable "ct." (Recall that the user would have moved the playhead before clicking this button.)
The next several lines do some error trapping, as we humans are very prone to errors. We'll come back to these in a moment.
Let's jump to line 8. This line puts the value of "ct" into the selected line of field "log," replacing whatever value was already there. Well, of course, the idea of this button is to update the entry after all.
The last line highlights the line in the log we just edited. This is important from a user design standpoint. Without this line, the user would have to again click on that line if they wanted to make another update. I think it's easy to imagine the user making lots of updates in a row until they get things "just right." (Tha'ts what I found myself doing as I tried out the prototype.) So, this line highlights the line for them.
OK, what about that error trapping? Think about it for a moment. We have to ensure that the starting point of the video clip is a number less than the ending point of the clip. So, line 4 starts an if/then statement to check this. If "ct" (the new position of the videohead) is greater than the ending point of the video clip, then that's a problem. So, LiveCode pops up with a message saying something to the effect that you can't do that, using the handy "answer" command. The next line - "exit mouseUp" terminates button's script, so nothing else happens. (Alas, error trapping is a very time-consuming, but necessary part of all software design. One has to be very cunning to determine where human error is likely to occur. This is very hard to predict and is one of the valuable outcomes of extensive field testing.)
Next, add another button and title it "Update End." As you might suspect, the script for this button is very similar to the button "Update Start":
on mouseUp
put the hilitedLine of field "log" into l
put the currentTime of player "player" into ct
//error trap - ending point must be after ending point
if ct < item 1 of line l of field "log" then
answer error "Not Allowed! Ending point must come after the starting point!" with "OK"
exit mouseUp
end if
put ct into item 2 of line l of field "log"
set the hilitedLine of field "log" to l
end mouseUp
In fact, you could just copy and paste the "Update Start" button and make just a few, small (but critical) revisions. For example, in line 8, the only revision is changing item 1 to item 2.
The only other changes are in the error trapping. In this case, we have to be sure that the ending point comes after the starting point. Here is a quick summary of the changes to these lines of code:
- Change the "greater than" sign to a "lesser than" sign;
- Change "item 2" to "item 1"; and
- Change the text of the message as needed.
Revising the +10 and -10 Second Editor
Let's do one more thing. Recall from my previous blog posting that we created two buttons that advanced or rewound the playhead by 10 seconds. An increment of 10 seconds was a good choice there to demonstrate this feature, but it really is not a useful increment for "fine tuning." So, let's revise those two buttons so that the increment is only one second. (One could argue that even a smaller increment, such a half or tenth of a second, would be even better.) Making this revision is a snap.
Let's take a look at the existing script for the "-10 seconds" button:
on mouseUp
put the timescale of player "Player" into ts
put the currentTime of player "Player" into t
put t-(ts*10) into t
set the currentTime of player "Player" to t
end mouseUp
put the timescale of player "Player" into ts
put the currentTime of player "Player" into t
put t-(ts*10) into t
set the currentTime of player "Player" to t
end mouseUp
All we need to do is change the 10 to a 1 in line 3. (You might also argue that we could just delete the "*10" altogether, but I don't recommend that because we might find ourselves coming back and changing this later to a different value, such as .5.
Do the same thing for the "+10 seconds" button and, of course, don't forget to change the button titles.
Final Thought
My final thought is simply "Hey, pretty cool!" We have a good start to an interesting video analysis tool.
In my next post, I'll focus solely on how to convert or transform LiveCode's time code intervals to something we humans can better understand, such as hours:minutes:seconds.
No comments:
Post a Comment