Sunday, June 22, 2014

Lloyd's Video Analysis Tool, Part 6: File Management

I have a brother-in-law in the construction business. He talks about working with clients who get very happy and excited when they see wall studs go up or an old roof being rapidly torn off.  In contrast, they can get impatient when he is working on plumbing or electrical jobs because they don't see much visual progress, or VP. This is frustrating to him because he is working just as hard and putting in just as many long hours. It's just that you can't see at a glance the progress he's made. Well, I've been working on some of the "plumbing" for my video analysis tool project lately and though I've been working hard on important stuff, there's not much VP.

Up to this point, I've been using the same movie for all of my project development - my mash-up video of my MOOC on statistics. The movie file was "hard wired" into the project, that is, the file name and path were manually entered and saved into the video player tool. This meant that this movie, and only this movie, would play. So, I've finally updated the project (now up to version 0.2.3) to allow any movie to be used and -- drum roll please -- to allow multiple projects to be saved and edited. This is like having hot and cold water on the construction site! Here is a brief movie demonstrating this latest version:


Obviously, I continue to be largely ignoring the graphic design of the tool. However, I think the user interface design works pretty well.

Saving Project Files


Although I have worked with scripts that save data in other LiveCode projects, I have not worked on a LiveCode project where one has the option to create and then subsequently edit new project files. Before moving forward, I had to decide on the design for this capability. I saw two possibilities. The first is what I consider the standard approach, common in applications such as Word or PowerPoint, where the user creates a new file, then saves it wherever they wish on their computer. It is up to the user to remember where the file is kept (though the "Open Recent" options help the user to quickly recall and find the most recently worked on files). The second approach is more common in mobile apps where the app itself manages all of the stored files. Evernote is a good example of this approach. I decided to go with this second approach, at least for now, particularly because I envision this tool being used on an iPad.

Project Workflow


The ability to create and then work on as many projects as one wishes required a major new design feature related to project workflow. I envisioned three possibilities:
  1. Continue working on the same project as before;
  2. Start a new project;
  3. Select a previously saved project to work on.
I created a new card, which is now the first (or home) card of the stack, to provide these options:


Managing the Project Files


Actually, all of the project files are just text files. For now, I save all these project files into the "Documents" folder of the computer. Macintosh, Windows, iOS, and Android operating systems all have a special documents folder that can easily be accessed using this LiveCode command:

set the defaultFolder to specialFolderPath("Documents")

The defaultFolder property specifies the folder that all files and folder options used by the LiveCode file will point to. The specialFolderPath function lets you quickly point to one of several system folders, such as the documents folder.

I think the documents folder is the best choice for saving project files like these on mobile devices. However, on laptops or desktops it would probably be better to just have the files be saved into the same folder as the application itself. So, I may change the folder path later just for versions of the project meant for laptop/desktop computers.

I also needed a way to guarantee that the file names would be unique, so I created a counter for the number of projects created to date using a file name structure as follows: "VATproject"+{project number}

Contents of a Project File


Here is an example of the contents of a project file titled "VATproject43.txt":

Project Number,43
Project Name,Introduction to z Scores
/Users/lloydrieber/Documents/MOOCs/Presentations/Descriptive Statistics Module/03-z Scores/DescriptiveStatistics-z-Scores.mp4
VATLOG
1,Using Sunny in Example,8070000,8520000,dog,example
2,z Score Formula,32529000,32979000,formulas
3,Girl Scout Example,44334000,44784000,cookies,example
4,Comparing z Scores,78900000,79350000,example
VATCOMMENTS
VATComment,4
An interesting example of how z scores can be used to compare two different distributions.
VATComment,2
The formula is not complicated.
VATComment,3
This is a good context for an example.
VATComment,1
A cute example, especially if you like dogs.
VATTAGS
computation
cookies
dog
example
formulas

Commas are used throughout to separate items on each line. Here are some examples of important, specific information:
  • Item 2 of line 1 contains the project number;
  • Item 2 of line 2 contains the project name given to it by the user;
  • Line 3 contains the path to the video.

Notice the sections that come next, each with a special header: VATLOG, VATCOMMENTS, and VATTAGS. I'm sure you can easily decipher the information that each contains. The number after each comment links the comment to that same numbered video entry. The tags for each video entry are contained at the end of each entry.

Parsing the Project File into the Project Elements


Because there is no limit to how many video entries the user identifies I had to build this text file in such a way as to allow it expand or contract over time. Basically, the script asks LiveCode to scan each line until that header is found, then it knows that all information from the next line down to the next header contains all of the information for that section. For example, here is the code that looks for comments:

   repeat with i = 1 to the number of lines in field "project data"
      if line i of field "project data" = "VATCOMMENTS" then 
         put i+1 into starti
         exit repeat
      end if
   end repeat
   put empty into tempdata
   put 0 into ii
   repeat with i = starti to the number of lines in field "project data"
      add 1 to ii
      if line i of field "project data" = VATTAGS then exit repeat
      put line i of field "project data" into line ii of tempdata
   end repeat
   put tempdata into URL ("file:currentprojectcomments.txt")
   
The first repeat loop looks for the text string "VATCOMMENTS." The second repeat loops takes each line thereafter, up to the point of finding the text string "VATTAGS," and stores it in a temporary local variable called "tempdata." When done, the contents of tempdata are transferred into a text file called "currentprojectcomments.txt." While working on a project, I save the video log, comments, tags, and basic project data (e.g. project name, file path to the movie) in separate text files all with the prefix "currentproject...". Then, I combine these files into one long file and save it with a unique project name. When the user chooses to work on an existing project, this single file is then broken apart and saved into the separate "currentproject" files. I split and recombine this information continuously as the user works on the project.

I debated whether to include some sort of auto-saving, but I wasn't confident I could do this in a way that would be reliable. So, I decided to give the user the option to save manually while working on the project and also save the project automatically when the user chooses to go back to the home card.

I will admit to having some difficulty early on in writing this script. My earlier drafts of this script used the keyword "after" instead of "into" when putting data into a particular line of tempdata. I couldn't get the data stored as expected and if the data gets slightly "out of kilter," nothing will work. So, I gave up on that more elegant and simple (potentially) script in favor of a script that made sure I knew where each line was being saved.

Things to Work on Next


These file management properties of the current version are important improvements, but there are some obvious things that need to be done. First, there are several minor, yet important features that I have not yet tackled:
  • Allow the user to edit the tags assigned to a clip;
  • Allow the user to edit the project name;
  • Error trapping: Catch for the possibility that the user has moved the movie and give the user the opportunity to "find" the movie and re-establish the path.
Also, I need to begin work on the feature of creating reports for each video analysis.

So, I'll focus on these in the next version or two.