Wednesday, April 8, 2015

Desperately Seeking Middleware: Connecting LiveCode to a mySQL Database with PHP

I think I have finally figured something out after about two years of false starts and false hopes. When I first started using LiveCode, one of the features that interested me most was its ability to connect to databases stored on Web servers. I quickly found the following tutorial which led me to believe that doing so would almost be child's play:

http://lessons.runrev.com/m/4071/l/7003-connecting-to-a-mysql-database

But, I quickly ran into a problem and submitted a request for help to the LiveCode's user forum. I quickly learned from other LiveCode users that this "easy way" to have LiveCode connect to a database was only possible if LiveCode has a "direct connection" to the database. However, for security reasons, few web servers allow direct connections. The solution was to use "middleware" code -- code that resides on the web server that acts as an intermediary between the database (stored also on the server) and my standalone LiveCode app (residing on my computer, my iPhone, etc.). I learned that I could write this middleware in PHP. Another approach I was encouraged to consider was installing a special version of LiveCode on the web server with special server functionality - called (not surprisingly) LiveCode Server - with which I could easily write this middleware. Unfortunately, the server and the database I wanted to connect to was administered by a commercial hosting company and so I did not have the ability to install LiveCode Server on it. Plus, I already had been working a little with PHP on other projects, so the PHP route seemed the way to go.

But, I really didn't have a firm conceptual understanding of what this middleware was supposed to do. Every now and then over the next two years I came back to this problem and each time I got stuck again. I usually chalked it up to just not doing a thorough enough search on Google or the LiveCode user forums. Finally, a few weeks ago, I redoubled my efforts and found a few intriguing leads. I also submitted a much more detailed and somewhat emotionally-laden request for help to the LiveCode user forum. Based on some very helpful replies, I realized that I was looking at the problem all wrong. As it turns out, I (more or less) knew all along how to solve this problem, but just didn't know it at the time. That realization opened the door for me to solve the problem.

So, the purpose of this blog post, and perhaps a few more to come, is to share what I've learned with some concrete examples, code and all. Put another way, the purpose of this post is to provide to others exactly what I needed two years ago.

But I must alert anyone who stumbles upon this posting that understanding the solution that follows requires not only some understanding of LiveCode, but at least a cursory understanding of PHP, a touch of SQL and HTML, and a very good understanding of how to set up and manage a mySQL database on a typical server installation (such as with the use of an interface such as phpMyAdmin).

Setting the Context: Imagine the World's Most Boring Game


I have created a very simple sample project to demonstrate how all this works. Imagine a game that generates a final score, except that we'll skip the game part and let users choose their own score. Next, imagine that you want the people playing your game to have the option to upload their score along with a user name of their choice. That way, we will have a record of all game scores tied to players. This would allow (later) the option to give a single user feedback on how well they are doing in relation to all other players. Similarly, although I don't do it here, it would also be straightforward to generate a list of the ten top high scores for the players to aim for. But, let's keep the task as simple as possible for right now.

So, yes, I created the world's most boring game for demonstration purposes here. Basically, you just enter whatever game score you would like along with a user name of your choice, then click a button to submit that information to be added to the database on the server. We'll also see how to retrieve and view all of the game scores and user names.

Assembling the Software Puzzle Pieces: mySQL Database, PHP, and LiveCode


To do this, you need to first create a database on the server. I'm using mySQL on a web site maintained by a commercial web hosting company. I created a simple database with just one table called "players." The table has the following columns:

  • ID_players
  • username
  • score_recent

Writing the LiveCode Scripts


Here is a screenshot of the boring game I built with LiveCode:



After entering your user name and game score -- in fields titled "username" and "score_recent" --  you click the button "Submit to the Game Center." Here is the script for this button:

on mouseUp
   put "username="&field "username" & "&score_recent="&field "score_recent" into varSendData
   post varSendData to URL "http://yourwebsite.com/gamecenter/insert_newscore.php"
   put it into varResults
   answer "Response from database:"&return&varResults
end mouseUp

The first line assembles a string containing the information the player entered into the two fields. Let's image that the person enter "Lloyd" for user name and "99" for the game score. The resulting string, put into the variable varSendData, would look like this:

"username=Lloyd&score_recent=99"

I used variable names that matched the column labels in my database, but that was not necessary.

This is information is then posted to the URL shown - obviously you would enter the URL of the PHP file on your web site. POST and GET are the two standard ways to send data to a URL. The GET method will attach the information to the end of the URL and hence will be visible in the user's browser (not recommended for sensitive information, such as passwords), whereas the POST method keeps the information hidden from view.

The PHP file "insert_newscore.php" takes that information and performs the database function of inserting the information into the database as a new record, then returns the message "Information entered into database." This message is put into the variable "varResults" which is displayed in a pop-up message.

OK, let's take a quick look at that PHP file.

Writing the PHP File "insert_newscore.php"


Here is the entire code for the PHP file "insert_newscore.php" that has been waiting on the web server:


<?php
//1. Create a database connection

# FileName="Connection_php_mysql.htm"
# Type="MYSQL"
# HTTP="true"
$hostname_connLivecode = "yourhostname";
$database_connLivecode = "demo_high_score";
$username_connLivecode = "yourusername";
$password_connLivecode = "yourpassword";
$connLivecode = mysql_pconnect($hostname_connLivecode, $username_connLivecode, $password_connLivecode) or trigger_error(mysql_error(),E_USER_ERROR);

//2. Perform database query

$username = $_POST["username"];
$score_recent = $_POST["score_recent"];

mysql_select_db($database_connLivecode, $connLivecode);
$query = "INSERT INTO players (";
$query .= " username, score_recent ";
$query .= ") VALUES (";
$query .= " '{$username}', '{$score_recent}'";
$query .= ")";

$result = mysql_query($query, $connLivecode) or die(mysql_error());

//3. Return feedback
echo "Information entered into database";
?>


This file is pure PHP. In fact, it is a single PHP code block. No HTML is included or needed. Adding HTML tags of any sort would just get in the way, as I'll explain later. The PHP code performs three steps. First, it establishes a database connection. You obviously need to enter your own hostname, database name, user name, and password for the database.

Second, it takes the two bits of information we posted (i.e. sent) to the page (shown in red) and puts each into PHP variables with the same names (again, not necessary, but a good habit to get into), followed by a SQL script to insert this information into the players table in the database.

Third, it returns the feedback "Information entered into database" to the LiveCode app using the PHP echo command (which I equate to a print-to-screen command). Indeed, if this PHP file were viewed in a browser at this moment, all you would see on the screen is the sentence "Information entered into database."

If you've never worked with PHP, please don't beat yourself for not understanding the syntax or command structure (and just be thankful that LiveCode isn't written this way). But, I do hope you can at least get the gist of what's going on here.

Using LiveCode to Trigger Database Queries


OK, what if I want to know all the games scores saved so far. The two buttons "Show All Users (sort by username)" and "Show All Users (no sort)" are used to initiate a process that will trigger the database to perform these queries then return this information from the database and put it in the field at the bottom of the card. The scripts for both buttons are identical except for one thing. Let's look at the script of the first button:

on mouseUp
   put empty into field "players"
   put URL "http://yourwebsite.com/gamecenter/show_all_scores_username.php" into varResults
   put varResults into field "players"
   if line 1 of field "players" is empty then delete line 1 of field "players" 

end mouseUp

The first line just empties out the field "players," which is the name of the field at the bottom of the card. The second line goes to the PHP file on the Internet and puts whatever it finds into the variable "varResults," which in turn is put into the field "players." (The last line is a hack -- for some reason, I get a blank line as the first line every time, so this just deletes it.)

The code for the other button is exactly the same, except that it goes to this URL:

http://yourwebsite.com/gamecenter/show_all_scores.php

OK, let's take a look at each of these two PHP files

Writing PHP Files to Query the Database and Return the Results to LiveCode


Here's the code for the PHP file "show_all_scores_username.php":

<?php
//1. Create a database connection
# FileName="Connection_php_mysql.htm"
# Type="MYSQL"
# HTTP="true"
$hostname_connLivecode = "yourhostname";
$database_connLivecode = "demo_high_score";
$username_connLivecode = "yourusername";
$password_connLivecode = "yourpassword";
$connLivecode = mysql_pconnect($hostname_connLivecode, $username_connLivecode, $password_connLivecode) or trigger_error(mysql_error(),E_USER_ERROR); 

//2. Perform database query

mysql_select_db($database_connLivecode, $connLivecode);
$query_rsUser = ("SELECT * FROM players ORDER BY username ASC");

$rsUser = mysql_query($query_rsUser, $connLivecode) or die(mysql_error());
$row_rsUser = mysql_fetch_assoc($rsUser);
$totalRows_rsUser = mysql_num_rows($rsUser);

if ($totalRows_rsUser > 0) {
    echo "#Data begins here"."\n";
if ($totalRows_rsUser == 0) {
    echo "no data found";


do {
  echo $row_rsUser['username'].",".$row_rsUser['score_recent']."\n";
   } while ($row_rsUser = mysql_fetch_assoc($rsUser)); 
  
?>



As you can see, this script has some similarities to that in the other PHP file. The main difference is the following SQL statement:

SELECT * FROM players ORDER BY username ASC

This says to select all of the records found in the table "players" and return and display them in alphabetical order of the column "username."

The other key command is the "do" command at the very end which also includes an "echo" command (show in red). This creates a loop where the database finds and shows each record in the database. The loop repeats for as many records in the database. As before, if we were to enter this URL in a browser, you would see the information appear on the screen. All LiveCode does is take this ouput and put it into the field "players."

It also merits pointing out that this PHP file does not need any input to work, so there is no need to POST or GET any information from the LiveCode file.

The script in the PHP file "show_all_scores.php" is identical except for the SQL statement:

SELECT * FROM players ORDER BY ID_players ASC

This just says to show the records in order of ID_players. ID_players is an integer that auto-increments by 1 every time a new record is created. It guarantees that every record will have a unique number assigned to it. Indeed, even if a record is deleted the number for that deleted record will not be reused. By ordering the output of the records by this number in ascending order, it will show the records in the order of which they were created.

Again, if you don't know PHP, then don't get stressed out over not understanding the ins and outs of every line. However, just to show that it really is just another programming language like LiveCode with the same sorts of functions and options, let's take a look at the line I highlighted above in red:

echo $row_rsUser['username'].",".$row_rsUser['score_recent']."\n";

Notice in the middle these characters: .",".

The period is the concatenation symbol in PHP, whereas we use the & symbol in LiveCode. So, I've merely joined the two variables with a comma in-between. Notice also at the end these characters: ."\n"

"Backslash n" is the PHP code for a line return, so I've simply concatenated this to the end of the string that is being "echoed" (printed to the screen). This ensures that each record in the database will be shown on a separate line.

So, let's imagine that this is the list of all scores in the database which gets returned to the LiveCode app:

#Data begins here
Mark,88
William,91
Lloyd,99

First, you might wonder why it starts with "#Data begins here." It's a little something I added. You will find this sentence in the script above. It's part of an if/then statement and is triggered only if there is data found in the database. Otherwise, the statement "no data found" is return." (I thought it would be a good idea to include the # sign, as I often use that symbol to aid in my list processing scripts because it's a unique and easy symbol to search for.)

Next comes the username and score for each record in the database separated by a comma, with each record on a new line. I decided to display it this way because this is a convenient form for LiveCode to handle given that the comma is the default itemDelimiter. So, item 1 of each line is the user name and item 2 of each line is the game score. If this were a "real" game, I would probably just keep the field hidden from view or alternatively store this list of data in a variable. Then I would display the information in a more friendly form to the user.

Why No HTML?


I'll bet you know some HTML and you are just dying to add a <p> here and a <br> there. Don't do it. The reason is that these tags will appear in their original text form in the field "player" along with the raw data, which is obviously something we don't want.

Short Summary of What I Learned


In the end, the PHP files above are really nothing special for LiveCode. If you were designing a Web site with a database backend, these are exactly the sort of pages you would be writing. Instead of having LiveCode send a user name with a game score, or asking for a list of scores to be shown, you would build an HTML page to do pretty much the same thing. That, in essence, was my conceptual breakthrough. I thought, incorrectly, that I needed to write a custom PHP script -- specialized for the needs of LiveCode -- that would allow LiveCode to "talk" to the database. This misconception blocked my ability to see the solution for two years.

Just so you know, I'm not convinced any of the above is the "best way" for LiveCode to communicate with a database, but it is definitely "one way" to do it. As always, I look forward to the more experienced LiveCode programmers out there to give feedback and guidance to me.

The best thing I can say is that I'm glad I persevered. I now have a whole new LiveCode world open to me.



Wednesday, March 18, 2015

March LiveCode Madness: Building a Clickable Bracket

For the first time ever, I have filled out my own bracket for the upcoming NCAA basketball tournament. For many people here in the USA, this is an important annual ritual. One's bracket simply predicts every winning team in every match-up in the six rounds of the tournament. The tournament starts with a field of 64 teams with half being eliminated after each of the first five rounds. The sixth round pits the two remaining teams in the championship game to crown the national champion. I'm finally participating at the encouragement of my brother, Bill, who happens to teach economics at Butler University in Indianapolis, Indiana. Bill is our family's sports historian with particular expertise in baseball. But this is basketball season and basketball is very big in Indiana. The significance of basketball in Indiana is depicted very well in the movie "Hoosiers." (Trivia: The final championship game in the movie was filmed at Butler Field House where the actual state high school championship game had long been played.)

Basketball is also very big at Butler University. Butler is probably the smallest school ever to appear in the NCAA championship game, having done so in both 2010 and 2011. Unfortunately, Butler lost both times. The loss in 2010 was quite dramatic with Butler, down two points, taking the last shot of the game. Alas, the ball bounced off the rim and Duke won 61-59. The University of Georgia is also playing in the tournament this year and it is conceivable that they could face Butler in the championship game. More on that later.

Like a lot of work places, Bill's colleagues at Butler run a little competition each year where you can enter your bracket. The person with the best prediction wins the competition. Bill enters every year and this year, again with Bill's encouragement, I'm participating too. Our other brother, Mark, is also participating.

To fill out my bracket, I found a PDF online at printyourbrackets.com. For those not familiar with what a bracket looks like, here is a quick snapshot:



I find the visual design very appealing. Teams from the midwest and west are on the left-hand side and teams from the east and south are on the right. You can easily see how teams are quickly whittled down until there is one national champion in the center-most box. But, to fill out this PDF, I had to do a whole lot of copying and pasting. The first version I sent to Bill had a serious copy and paste error, which I'm glad I caught soon after I sent my PDF to him. I had Wisconsin winning the game between Kentucky and Buffalo. (Wisconsin is good, but they are not that good.) I showed Wisconsin continuing to win in Kentucky's spot until the championship game, when Kentucky "reappeared." I fixed the PDF and sent my brother an updated version.

But, I found myself wishing that I had found a "clickable" version of the bracket online instead of a PDF. Then, I could have just clicked my way from team to team with each click quickly pasting in the name of my pick in the slot to the right or left. (It's likely there is such a clickable version out there, but I didn't find one in the few minutes I allocated to my search.)

So, I created a very simple and very incomplete version of a basketball bracket using LiveCode. And, I learned something new by creating this little project. In order for the "on mouseUp" handler to work within a field field, the text in the field has to be set to locked. This makes sense because otherwise every time you clicked in the field to type something into it -- such as the names of the basketball teams in round 1 of the tournament -- the on mouseUp handler would take over. But, to enter the first set of team names, the fields need to be unlocked. So, I needed to have an easy way to lock and unlock the text fields, or at least the text fields to the far left or right of the bracket.

I created two buttons - "Lockdown" and "Unlock" - that, well, lock and unlock the text fields. Now, in my little example, I only have 4 of the 64 teams represented. This results in a total of seven text fields (or three open brackets). If I had all 64 teams, that would require 126 text fields (with 62 open brackets). That's a lot of fields to lock or unlock. But, I learned a good trick while creating a previous project based on the fact that LiveCode automatically numbers all fields on a card. With that knowledge, I used the following script in the button "Lockdown" to lock all the fields on the card:

on mouseUp
   put the number of fields on this card into L
   repeat with i = 1 to L
      set the locktext of field i to true
      set the backgroundcolor of field i to red
   end repeat

end mouseUp

The first line just counts up the total number of fields on the card and puts this number into the local variable L.

Then, I create a loop that repeats L times, starting by putting 1 into the local variable i. The next line sets the locktext property of field i (that is, field 1 or field 2 or field 50, depending on what loop it is) to true, thus locking down the field. Each loop automatically adds one to i.

I also went ahead and set the backgroundcolor of locked fields to red. I thought it would be nice to use some color coding to signal to the user whether the fields were in a locked or unlocked state.

In the button "Unlock" I simply changed true to false in the "set the locktext" line. I also changed the background color to yellow. Here's a snapshot of the screen in the unlocked state:

[ Get the free LiveCode Community version. ]


Here's a snapshot of the screen after the fields have been locked down, thereby allowing the user to click on the fields to make selections, going from left to right:



I then added a script to each field that put a copy of whatever text is in that field to the next field to the right. Here's an example:

on mouseup
   put me into field "r2g1t1"

end mouseup

I labeled all the fields with a scheme that would be helpful if I later choose to continue building this project with a complete bracket. This scheme starts with the tournament round number, then the game number, then the team number. So, "r2g1t1" means round 2, game 1, team 1. Using this scheme, I think I could automate the script for the fields so that each would know what field to send their contents to. For now, I just hard coded the destination.

Looking Forward to the Day When LiveCode Exports to HTML5


As I write this, many of us are waiting anxiously for news on the RunRev company's efforts to enhance LiveCode to allow projects to be exported to HTML5. This project would be a perfect candidate for this feature because I could then embed this project within a web page. Then, I would have legions of basketball fans flocking to my clickable bracket maker site. As I understand it, exporting to HTML5 will essentially convert LiveCode scripts to Javascript. The company began the effort to add this feature to LiveCode about eight months ago with a timeline to completion of one year. So, I keep my fingers crossed everyday that the company will send out an announcement that this feature is finally ready.

One Last Bit of Basketball Trivia While Pondering Who Might Win This Year's College Championship


Here's a final piece of basketball trivia. As I understand it, dribbling evolved from a loophole in the original rules of basketball. The game was invented in the late 1800s by James Naismith, a Massachusetts teacher who wanted a safe, indoor game. The idea was to have players continually pass the ball to teammates to move the ball down the court with the goal of getting the ball into a peach basket mounted up high at the end of the opposing team's side of the court. It was against the rules to run while holding the ball. At some point, someone figured out that if you bounce the ball you are not technically holding it, so this was a way for the player to move about without breaking the rules. Ah, American ingenuity.

So, who do I predict will win the tournament this year? Yes, it is possible for my school - The University of Georgia - to meet my brother's school - Butler University - in the NCAA Basketball Championship game. But, even with the little I know about college basketball, I know the odds of this happening are very, very, small. So, despite the fact that I show above Butler beating Kentucky in round 4 of the tournament (also known as the "Elite 8"), I'm predicting Kentucky will go all the way this. But, then again, stranger things have happened.

Oh, one more thing ... what exactly is a hoosier, anyway?


Thursday, March 12, 2015

Creating a Reference Letter Writing App with LiveCode: New and Improved Design

Yes, I just couldn't help myself. After creating the initial prototype of a reference letter writing app "just for fun," and then realizing there was a much better design strategy just as I was finishing it, I just had to follow through on building the new design. Recall from my previous posting that it occurred to me - late - that a much better approach would be to use "piped" or replaceable text to build a letter template within a simple text field. This allows the user to simply paste in a letter template, or write a new template from scratch. Categories can then be embedded with the use of open and close braces, such as {myRating}. (The survey tool Qualtrics calls this feature "piped" text, hence the origin of my use of the term, but I think "replaceable text" is clearer, so I'll use that term from here on out.)

The resulting app works very well. I even added the feature where the app automatically saves the most recent letter template, addressee information, candidate information, and all the categories created up to that point.

I have also changed the name of the app to "Report and Letter Writing App" to reflect the extended range of uses I think the app could be used for. For example, I see myself using it to quickly generate student feedback on low stakes assignments. But, I can easily see uses in fields other than education, such as a doctor who needs to write a short patient report.

Here's a video demonstration of the app:


How the App Works


Using the app involves a four-step process. In step 1, the user is given a blank text field within which to write a letter template. At any point, the user can create a category for replaceable text. Adding the category to the letter template is done simply by double-clicking on the category name. The category can be updated as often as one wishes with as many choices for the replaceable text added as desired.

In step 2, the user enters basic information for the candidate - name and gender - and the name and address to whom the letter is addressed.

In step 3, the user simply makes the choices from all of the replaceable text categories that are appropriate for that particular candidate.

In the final step, the letter is generated and the user simply copies the resulting letter to the clipboard for pasting into a word processor or online form.

This app version still has many limitations - hey, I created it very quickly after all! For example, only the most recent letter template is saved along with all categories created to date. An obvious design feature would be to let the user save any number of letter templates with only the categories relevant to that letter.

Try It Yourself


I made a standalone version for both Macintosh and Windows - download and uzip it if you would like to try it for yourself. Be aware that both copies will make a folder titled "RLWA Program Files" in your documents folder that contain the app's management files. If you delete that folder or its contents, you will lose all of your work.


Macintosh Note: The first time you try to open it, you will likely get a message saying that it "can't be opened because it is from an unidentified developer." This is due to the Gatekeeper function that has been recently implemented in Macintosh OS X, which by default is set to only allow the opening of applications that were obtained through the Mac App store. Fortunately, it is easy to change your security settings to allow your Mac to open the app. Here are two short videos that show how to do this:
https://www.youtube.com/watch?v=gZYGzjbkmBg
https://www.youtube.com/watch?v=rmk6kiB4sw4 (this explanation is better, but the audio quality is poor)


Windows Note: This is an untested version, but it seemed to work just fine using Windows 7.

Programming Challenges


There were all sorts of interesting challenges to building this app and I learned many new LiveCode skills as a result. Below are two examples.

Identifying the Location of the Category Insertion Point


One challenge was learning how to note the location of the focus within a text field. I used the following script and attached it to the field "letter" on the card where one builds the letter template to accomplish this:

global varInsertion

on mouseleave
   put the selectedChunk of field "letter" into varInsertion
   set the itemDelimiter to space
   put item 2 of varInsertion into varInsertion
end mouseleave

What's somewhat cool about this script is that it only triggers as the mouse leaves the text field. The idea being that one could click at the point where you want to insert a category, followed by moving the mouse to the spot on the screen where the categories are listed. This movement of the mouse guarantees that the last insertion point would be noted.

Here's how it works. Consider the following sentence: "The purpose of this letter is to support the application of George Bailey."

If the word "George" is selected, the following would be put into the variable varInsertion:
char 61 to 66 of field 1

When the focus (i.e. blinking I-beam) is immediately before the "G" of George, the following is given:
char 61 to 60 of field 1

All I really want is the first number - 61 - which is the second item in the string if one uses the space character as the delimiter between items. The variable varInsertion is then used by the field containing the list of categories on the same card:

global varInsertion

on mousedoubleup
   put "{"&the selectedText of field "dynamic text"&"}" after char varInsertion-1 of field "letter"
   focus on field "letter"

end mousedoubleup

This script is only triggered when the user double-clicks on one of the available categories. The category name is then entered after the varInsertion point with braces added to each side. You might notice that I had to subtract one from varInsertion to get it to work properly.

The only small problem is that if you double-click on two categories in succession, the second always goes in front of the first, which is counter-intuitive to me. I've not been able to solve that problem, but it is a relatively minor annoyance. Otherwise, all works great.

Allowing for Cutting, Copying, and Pasting


One unexpected challenge was figuring out how to get the simplest of text functions to work within the standalone version of the app, namely cutting, copying, and pasting. These text functions worked just fine when I was building the app, but they simply didn't work in the standalone version (using the typical shortcut keys). I found this very puzzling, but luckily I came across an article in one of the LiveCode forums that addressed the issue. To resolve it, I simply had to add the following code to the stack script:

on copyKey
   copy the selection
end copyKey

on cutKey
   copy the selection
   delete the selection
end cutKey

on undoKey
   undo
end undoKey

on pasteKey
   paste
end pastekey

I've noticed this issue in other projects I've built, so it's good to finally get to the root of the problem and solve it.

Final Thoughts


Yes, what started out as a fun project just to prove I could do it to some of my colleagues has turned out to be a very nifty little app. Similar to the first version, I created this second version in about a week working on it here and there. So, not much time was needed, thanks to the wonderful programming environment of LiveCode and especially LiveCode's excellent text processing features. I learned some great new LiveCode commands and developed a few new programming skills along the way that will come in very handy I'm sure for future projects. I definitely see myself coming back and working on the app to develop it into a full-fledged app worthy of sharing.

Monday, February 23, 2015

Creating a Reference Letter Writing App with LiveCode

Somewhat on a dare, I've created a prototype of a reference letter writing app with LiveCode. I say "on a dare" because I was eavesdropping on a conversation of a few of my faculty colleagues at the University of Georgia before a faculty meeting last week. They were talking about the number of reference letters they were having to write. One colleague remarked that she had promised letters to about 30 very deserving undergraduate students who needed the letters as part of their application to graduate school. As happy as she was to write these letters for them, the task was really taking its tolI in terms of the time she needed to do it. I casually remarked that I thought it wouldn't be too hard to create a simple app with LiveCode that would construct at least a respectable first draft. Everyone thought this would be a fantastic idea for an app though I thought I sensed a hint of doubt in their enthusiastic voices that I could pull something like this off.

So, naturally I dropped everything else I was doing and created a prototype of just such an app. (As I like to say, I'm not a competitive guy, but "I'm more competitive than you.") Seriously, though, it did sound like an interesting and worthwhile project and one that I sincerely thought wouldn't be that hard to build. I even mentioned to my colleagues that it should be very easy to create a simple student rating system, say by using a scale such as from 1 to 3 (outstanding, very good, fair) which could be used to automatically generate a suitable draft letter.

I've spent about 3 hours on this project and it was about as challenging as I suspected. However, I don't think the prototype I built is the way to approach this problem. But, I think that is an important principle of design -- one cannot fully anticipate the design specifications, or even the best approach to identifying those specifications, for a project until one begins building it. I'll return later to discuss what I think would be a better way, but let's take a look at the current prototype:


Allow me to emphasize that I and my colleagues at the University of Georgia take the task of writing reference letters very seriously. All letters require much care and attention. So, don't get the impression that I'm making this app with the intent to trivialize or cheapen the reference letter writing process. My goal was only only to explore how to make an app that would help someone write at least a simple first draft. The letter should then be customized as needed for the particular student.

As mentioned in the video, it would be very easy to have LiveCode choose a phrase from the menu choice fields at random, though this would require that the user code each phrase as being acceptable within the excellent, very good, or fair rating categories. This would allow the app to create a full draft of the letter with just two clicks (one to select the rating and the other to generate the letter). That feature obviously has some appeal.

A Better Design Approach


To be honest, I feel there are all sorts of problems with the design approach I took with this prototype, with the main one being that the user is "locked in" to the basic format of the letter. The only way to change the format is by me going in and changing it within LiveCode. For the app to be useful, the user would need to have complete freedom to change the format and contents of the letter at will. Another obvious feature that is needed is the ability to save as many letter templates as one wishes.

In hindsight, I think a better approach would be to use something called "piped text" within a simple text letter template. The idea is that as one writes the letter, one could decide on those locations in the text where dynamic text should be used. The user could then insert the dynamic text by denoting it with special characters, such as start and end braces. The dynamic elements could likewise be edited at will also. If I continue developing this app in any serious way, I'll most likely redesign it completely with this strategy in mind.

Saturday, February 21, 2015

Creating a Function in LiveCode to Capitalize a Given Word

I'm working on a little project that automatically constructs a narrative - a letter, to be precise - based on some user choices. As I started to build it I immediately found myself needing a way to occasionally capitalize a given word. For example, consider the pronouns he and she, or his and her. My program correctly identifies the gender of the subject of the letter, but sometimes a sentence begins with one of these pronouns and other times the pronoun just occurs somewhere in the sentence. I didn't want to create a variable to hold an all-lowercase version of the pronoun and another to contain a capitalized version. There will certainly lots of other times when a word will need to be capitalized.

I was hoping that LiveCode already had this function, but alas I can't find it. But, it does have the function "upper," which will convert a given text to all upper case. So, I used this to quickly build my own function to capitalize a given word:

function capitalizeWord w
   put char 1 of w into x
   put upper(x) into xupper
   put the number of chars of w into L
   put xupper&char 2 to L of w into y
   return y

end capitalizeWord

The function takes a single word as input and puts it into the local variable w. So, let's imagine that the word entered was "nowhere."

The function begins by dissecting the given word into two parts: the first letter and the rest of the word. The first line takes the first character of w and puts it into another local variable x. So, in the case of "nowhere," x is equal to "n." The next line converts x to uppercase and puts the result in yet another local variable xupper. Great, we now have the first letter of the word capitalized. We now need to join this with the rest of the word

The next line takes note of the total number of characters in the word and puts this number in the variable L. Nowhere has seven characters, so L equals 7. The next line joins xupper ("N") to characters 2 to 7 of nowhere -- N+owhere -- resulting in "Nowhere" and puts this into y, which is then returned to whatever line of code called the function.

To demonstrate how to use this, I created a small stack with two fields and one button:

[ Get the free LiveCode Community version. ]


The left field is labeled "word1" and the field on the right is labeled "word2". The idea is that you type a word into the left field, press the button, and the word is capitalized and shown in the right field. Here's the script of the button "Capitalize":

on mouseUp
   put empty into field "word2"
   
   put line 1 of field "word1" into w1
   put capitalizeWord (w1) into w2
   put w2 into line 1 of field "word2"
   
   put empty into field "word1"
   focus on field "word1"

end mouseUp

The first line simply empties out field "word2." The second line takes whatever is in line 1 of the feld on the left and puts it into the local variable w1 (think of this as word 1).

The next line calls the capitalize function, puts the result into w2, which the next line puts into line 1 of the field on the right.

I added two more lines of code to tidy things up a little. I empty the field on the left and then I make sure that the focus (the flashing vertical text bar) is put into this field.

This function will be pretty darn handy in this and other LiveCode projects!




Sunday, January 25, 2015

LiveCode at UGA's Digital Learning Conference

The second annual Digital Learning Conference was held Friday and Saturday here in Athens at the University of Georgia. I did my best to have LiveCode well represented. It was a wonderful conference, thanks to all the organizers and volunteers, especially my colleagues Dr. Christa Deissler and Dr. Theodore (TJ) Kopcha. There were some amazing, high quality sessions, such as those by innovative folks like Cat Flippen and Gary Shattuck. Very impressive stuff.

I did a LiveCode workshop and also presented my video analysis tool project. Not too many people attended my workshop, but several people asked me about it during the conference. I also used this opportunity to create another LiveCode "first project" (i.e. a fun first project designed especially for people who have never programmed before). I called it "Travel the USA!" and I think it worked well.

The conference had a participatory theme. Presenters were encouraged to propose sessions that emphasized interaction and hands-on activities. This motivated me to create a beta tester web site for people interested in trying out my video analysis tool. I barely pulled off the development of this web site in time for my session, but it all worked well. I designed my one-hour session as a mini-workshop about my video analysis tool. And, I now have a way to easily give people access to the tool while also gathering some important usability data from people with real interest in doing video analysis. I used a delightful video called "A Toy Train in Space" to demonstrate my video analysis tool - check it out:


As always, I enjoy introducing people who have never programmed before to LiveCode and what you can do with it.



Saturday, January 17, 2015

Comparing Programming Approaches in LiveCode: An Order of Magnitude Difference?


I would like to say that this will be my most exciting, riveting, thrilling, hold-on-to-your-seat blog post yet. But, truth be told -- especially to the casual reader with little or no experience with LiveCode -- it may be the most boring one I've written. Of course, if you are a math nerd, perhaps the title piqued your interest a little with the mention of "order of magnitude." And I promise I will use this concept as the cornerstone of this posting. But, if that is the only thing that interests you, feel free to skip to the very end for a wonderful video about the powers of ten to make your decision to click your mouse or tap your screen to come here worthwhile.

Still here? OK, some quick background... In my previous post I mentioned how I received some helpful advice from Richard Gaskin a few months back suggesting that I could improve the performance of one of my projects if I were to take a different programming approach. Let me make it very clear at the start how grateful I am to Richard for taking the time to make this comment. This was a very kind gesture on his part. He was obviously under no obligation to respond, but took significant time to read through what I was working on and posted a very thoughtful comment. And, as I have googled other LiveCode questions and problems over the past two years, it's quite remarkable how many times I find an answer or suggestion given by Richard. He is a exemplary LiveCode citizen who is very willing to share his extensive expertise and who also seems to be constantly scanning the LiveCode world to try to help people out. I had the good fortune to meet him at RunRevLive 2014 in San Diego and he also seemed like a regular guy who was sincerely interested in whatever you were working on.

To briefly recap, I had been using text fields to handle list or data processing tasks and Richard wrote to say that I could dramatically speed things up if I would use variables instead. Here again is what Richard's wrote:
"...by using the "repeat for each" loop method combined with moving data out of the complex field structures into variables for use within the loop, this should bring your processing time down by at least an order of magnitude."
So yes, I've been thinking about this "order of magnitude" thing. I know Richard knows exactly what an "order of magnitude" means. In contrast, many people who throw this phrase around just kind of vaguely know it means "much more." I think a lot of other people think it means "twice as much." But, actually, an order of magnitude is used to compare things on a logarithmic scale, which most commonly uses base 10. The best example I can think is the Richter Scale, which is used to compare the size of earthquakes. An order of magnitude is a step along the scale that uses the powers of 10. For example, consider the difference between 10^2, 10^3, and 10^4. (I can't easily show exponents in this blog, so I'm using the character ^ to mean "raised to the power of.") If written out, these numbers are 100, 1000, and 10000, respectively. The difference between these three values is not equal, but instead increases exponentially. In fact, I think the best way to think of something measured on a logarithmic scale is to go in descending order, that is, from 10000 to 1000 to 10. At each step, you are going 90% the remaining distance! If you take a really huge number, such as 10^100 (known as a googol, by the way), you eliminate 90% when you go to 10^99. Going to 10^98 likewise removes 90% of that remaining amount, and so on.

So, for Richard to make the assertion that changing my programming approach would improve performance by at least an order of magnitude is really saying something - he's saying that the speed will increase tenfold. But the more I thought about it, I really wondered if this were true. So, I decided to test this hypothesis.

A LiveCode Programming Experiment


To test Richard's hypothesis, I quickly built a LiveCode app that performed a very simple operation: It takes a long list of words and transfers it from one "container" to another. In one version, the container is a text field. In a second version, the container is a variable. The idea is to see how long it takes for LiveCode to complete the process using both types of containers: 1) from one text field to the other text field; and 2) from one variable to the other variable.

And while that sounds simple enough, there are actually several other decisions we have to make. For example, should I take a word from the top of the first list and add it to the top of the second? Or, how about taking a word from the bottom of the first list and moving to the bottom of the second? And, there are quite a few more variations I could throw in. Let's start with what I think would be the most grueling for LiveCode to perform: taking a word from the top of the first list and moving it to the top of the second (thus deleting it from the first list). I'll call this experiment 1.

Here is the code for this loop:

   put the number of lines in field "field1" into L
   repeat with i = 1 to L
      put line 1 of field "field1"&return before field "field2"
      delete line 1 of field "field1"
      put the number of lines in field "field1" into field "field1 total"
      put the number of lines in field "field2" into field "field2 total"

   end repeat

I titled the two fields "field1" and "field2," with the words being transferred from the first to the second. This loop repeats for as many words (one word per line in the field) as there are in the field. I always take the first word (on line 1) from field1, so you will see "line 1" listed twice above: first to "put" the word into field2, then to delete the word.

The following line above puts that word into the first line of field2:

      put line 1 of field "field1"&return before field "field2"

As you can see, I also take advantage of the "before" keyword. This keyword very conveniently lets you put something at the beginning of a list of items without having to worry about how to move every other item. (A similar keyword is "after," which I talk about below.) It's also important to concatenate a "return" so that the word solely occupies the first line in field2.

The reason why I call this the most "grueling" approach for LiveCode to accomplish is that the words shown in the two fields are constantly shiftly, as shown in the video below.

The final two lines (before the "end repeat") are used to show a running tally of the number of words in each field as the loop executes. Depending on the experiment being performed, this can be the only feedback given to us that something is actually "working." So, I hope you can see that there is not many lines of code here.

Now, let's take a look at the related code for moving the words from one variable to another.

   put the number of lines in varField1 into L
   repeat with i = 1 to L
      put line 1 of varField1&return before varField2
      delete line 1 of varField1
      put the number of lines in varField1 into field "field1 total"
      put the number of lines in varField2 into field "field2 total"

   end repeat

Your first reaction should be "Whoa, isn't this the same code?" Well, it's almost the same. Instead of referring to the two fields, I simply substituted the local variables "varField1" and "varField2". The important point is that this code processes the words exactly the same between the two examples, except for what containers are used to store the words.

If you are still confused as to what is going on here, watch a short movie I made demonstrating the app:

[ Get the free LiveCode Community version. ]



OK, let's take a look at the results of this first experiment.

Experiment 1 Results


Here are the results from the first experiment (time given in seconds):

Fields
Variables
move before and delete
141.72 54.59


As these data show, using variables instead of fields for this task is 2.6 times faster when the words are moved to the top of the second field the AND the words from the first field are deleted as they are moved. Yes, that is a big difference, but it is not an order of magnitude difference (i.e. 10 times faster). So, a preliminary conclusion is that Richard is correct in saying that the use of variables will be processed faster, but he overestimated the magnitude.

Experiment 2 Results


OK, let's do another experiment. What if we move the words from the first container to the second, but not delete them from the first. Accomplishing this difference is simply a matter of not including line 4 that begins with the word "delete." This does not seem to me to require as much processing power because you are doing less with the first field. But, who knows, perhaps some other dynamic is in play here that I'm not aware of. It's definitely worth exploring. Here are the results:

Fields
Variables
move before
73.59 27.30

In this case, using variables instead of fields for this task is 2.7 times faster when the words are moved to the top of the second field, but not deleted from the first field. This is a tad faster, but I'll call it a tie. I am a little surprised that the difference is not lower.


Two More Experiments: 1) Move After and Delete; and 2) Move After, But Don't Delete


I performed two other experiments where I used the keyword "after" instead of "before." As the word implies, this means that each word was added to the second field at the end of the field instead of at the beginning. From a processing standpoint, after the words fill the field in the second field you don't actually see the words being added to the field as they are "below" the bottom edge of the field. So, this would imply that LiveCode can process this more quickly than putting them before. I'll share the data in a moment, but there were no big surprises here. But, I did yet another experiment which did surprise me.

One Last Experiment: What Happens When You Hide the Fields


Watching the program execute this list processing task in all of the previous experiments, particularly when the words are moved to the top of the second field and deleted from the first, it seems clear that moving the words from field to field takes quite a bit of processing power. You can just see the difference. (Actually, I find it quite mesmerizing.) I wondered, though, what would happen if the fields were hidden from view? So, one more experiment was in order.

This last experiment was pretty simple. It's the same as the first experiment, but with both fields hidden. The data from this second experiment were very surprising. There was virtually no difference in processing time between using fields and using variables!

Here is a table with all of the data from all of the experiments:

Fields
Fields
Variables
Visible
Hidden
move before
73.59 27.29 27.30
move before and delete
141.72 54.60 54.59
move after
27.94 27.30 27.29
move after and delete
100.78 54.59 54.57


So, In Conclusion...


My main conclusion is that it doesn't seem to matter if you use fields or variables as the containers for list processing tasks such as the one used here, given the very important condition that the fields are not visible.

But on the Other Hand...


Of course, I might have completely misunderstood Richard's advice and therefore set up a perfectly useless experiment. That's OK, I've wasted time in much more uninteresting ways. But, I am open to any feedback that Richard or any other LiveCode expert may want to give me here. I'm even willing to revise my programming experiment to test other approaches, assuming I can figure out how to code it. Or, better yet, you could download the file yourself and revise it to test your own hypotheses - be sure to let me know what you find.

Many thanks again to Richard Haskin for making me think and for motivating me to explore this programming question. This is exactly the kind of feedback I was hoping for from experts in the LiveCode community when I began writing this blog. 

Powers of 10 Video


I end this blog posting (most of which I'm sure many of you wisely skipped) by sharing what might be one of my most favorite science videos of all time. It's called simply "Powers of 10," and it is narrated Phillip Morrison, a famous physicist and science educator who taught at MIT and worked on the Manhattan project. This will truly give you an idea of what is meant by an "order of magnitude."