Dynamic Web Development
Fall, 2009
Christopher Sung
Interactive Telecommunications Program, NYU
Class 7 - Advanced PHP Tricks and Site Architecture Basics

Contents:


Scripts from the Command-Line and Using Cron   [top]
Running PHP scripts over the web via a browser is just one way of using PHP. You can also run it from the command line if end-user output is not the goal. You might have a script that creates thumbnails of images, or checks for image attachments in emails, or makes changes to your database. In this case, you'd want to execute this script from the command-line on the web server and at regular intervals (every day, every 10 min, etc).

Scripts from the Command-Line
Any script that you can run from the web can also be run from the command line. Simply SSH into your Stage account, change directory to your dwd directory and issue the following command:

php [name of any PHP script]
such as:
php hello.php
You should see the contents of the web page that normally would have been sent to the browser. This is the script executing but sending its output to the shell window and not to the browser. Using this concept, we can create scripts that don't actually interact with the browser. They can insert into or modify data in your database without printing anything to the user.

Scheduling Script Execution Using Cron
Cron is the built-in task scheduler inherent to UNIX-based systems. Using this functionality, any user with an account can schedule a certain process or job to start at a single or repeating intervals up to one minute apart. One easy way to schedule jobs to run using cron is to create a text file (let's call it cron.txt) that has entries like the following:

1 * * * * php /home/[netId]/public_html/dwd/hourly.php >/dev/null
45 4 1 * * php /home/[netId]/public_html/dwd/monthly.php >/dev/null
You may have noticed that besides the numbers and asterisks to the left, we've added " >/dev/null" to the end of our cmd. What is this and why?

In UNIX, if your cron job generates any output text, this text is automatically saved as mail for you on the system and if the scripts executes often and the text is lengthy, can clog up the system. To suppress these messages, we send it to the directory /dev/null which exists specifically for this purpose. The ">" character means pipe or send. Our PHP scripts always generate "Context-type: text/html\n\n", so this needs to be suppressed.

OK, so the six fields for the above format are:

  1. Minutes (0-59)
  2. Hours (0-23)
  3. Day of Month (1-31)
  4. Month (1-12)
  5. Day of Week (0-7) where Sunday is 0 or 7
  6. The process to run
The "*" is a wildcard meaning "run it" for all values of that field (i.e. every day of the month, or every hour).

So for the two example above, the hourly script will run at 1 minute past every hour, while the monthly script corresponding to the second entry will run at 4:45am on the 1st day of every month.

Within these fields, there's some other ways to change the range:

Single number: shown above, it denotes a single instance

Sequential range: use the dash delimiter (1-5) and no spaces

0-3 6 * * * php /home/[netId]/public_html/dwd/hourly.php >/dev/null
The above runs the hourly.php script at 6:00am, 6:01am, 6:02am, and 6:03am every morning.

Random range: use the comma delimiter (1,3,5) and no spaces

1,3,5 9 * * * php /home/[netId]/public_html/dwd/hourly.php >/dev/null
The above runs the hourly.php script at 9:01am, 9:03am, and 9:05am every morning.

Repeating: use the slash delimiter (*/2) and no spaces

*/2 * * * * php /home/[netId]/public_html/dwd/hourly.php >/dev/null
The above runs the hourly.php script every 2 minutes.

Before we install the task file, let's make sure of several things in our cron text file:

Once you are done, upload the text file to your cgi-bin dir in ASCII mode. You can then use the "crontab" function to install it like so:

crontab cron.txt

To see your current listing of jobs, simply issue the command:

crontab -l

In general, the commands are:

crontab filename
Install filename as your crontab file.

crontab -l
Display your crontab file.

crontab -r
Remove your crontab file.

crontab -e
Edit your crontab file, or create one if it doesn't already exist. Note that this will invoke whatever default editor is defined for you on Stage, most likely "vi". If this sounds unfamiliar, simply edit your cron.txt job file locally, re-upload, and re-install.

Nancy has been kind enough to set up a help page about using cron jobs on Stage:

  • http://itp.nyu.edu/help/pmwiki.php/Help/CronJobs


    GD and Image Processing   [top]
    GD is an open source code library for the dynamic creation of images by programmers. GD is written in C, and "wrappers" are available for Perl, PHP and other languages. GD creates PNG, JPEG and GIF images, among other formats. Most stock installations of PHP come with GD library support enabled.

    Let's consider the following two examples. In both of them, we will use the GD library to create an image with a random text string (think "captcha" security codes for site login) using the following code:

    # Create a random six letter alpha-numeric string
    $string_a = array("A","B","C","D","E","F","G","H","J","K",
      "L","M","N","P","R","S","T","U","V","W","X","Y","Z",
      "2","3","4","5","6","7","8","9");
    $keys = array_rand($string_a, 6);
    foreach ($keys as $name=>$value)
    { $string .= $string_a[$value];
    }
    
    # Define background image URL
    $bgimage = "images/background.gif";
    
    $image = imagecreatefromgif($bgimage);
    $color = imagecolorallocate($image, 0, 0, 0); # Create the color black
    
    # Add the text
    imagestring($image, 4, 15, 15, $string, $color);
    
    In our first example, we'll actually write the file to the server, and then embed it in an <img> tag in order to show it:
    $outfile = "images/random.gif";
    imagegif($image,$outfile);
    
    echo <<<END
    
    Here is a PHP-generated image with random text:
    <p>
    <IMG SRC="$outfile" border="1">
    
    END;
    
  • PHP-Generated Image #1 | Source Code

    In our second example, the image data is actually sent straight to the browser

    # Output the image
    header("Content-type: image/gif");
    imagegif($image);
    
    and this script, which we'll call "gd-image.php", can be used as the source in the image tag in another script:
    Here is a PHP-generated image with random text:
    <p>
    <IMG SRC="gd-image.php" border="1">
    
  • PHP-Generated Image Data | Source Code
  • HTML Wrapper | Source Code
  • GD Library Reference

    For resizing, you first choose the method of how you want to resize - either to a set width and height, or to a percentage. You then call imagecopyresized() and output the data to a file. Because of security, newly created image files will have permissions set to 600, which means they can't be seen in a webpage. Invoking chmod() to change this to permissions 644 will fix this:

    # Image names
    $filename = "images/original.jpg";
    $outfilegif = "images/thumb.gif";
    $outfilejpg = "images/thumb.jpg";
    
    $filetype = "";
    if (eregi("[.](jpeg|jpg)$", $filename)) $filetype = "jpg";
    if (eregi("[.](gif)$", $filename)) $filetype = "gif";
    
    # Get new sizes
    list($width, $height) = getimagesize($filename);
    
    # set as a percentage
    $percent = 0.5;
    $newwidth = $width * $percent;
    $newheight = $height * $percent;
    
    # or set explicitly
    $newwidth = 80;
    $newheight = 80;
    
    # create the thumb image
    $thumb = imagecreatetruecolor($newwidth, $newheight);
    
    # get the source data
    if ($filetype=="gif")
    {
      $source = imagecreatefromgif($filename);
      $outfile = $outfilegif;
    }
    elseif ($filetype=="jpg")
    {
      $source = imagecreatefromjpeg($filename);
      $outfile = $outfilejpg;
    }
    
    # resize
    imagecopyresized($thumb, $source, 0, 0, 0, 0, $newwidth, $newheight, $width, $height);
    
    # create the thumb
    if ($filetype=="gif")
      imagegif($thumb,$outfile);
    elseif ($filetype=="jpg")
      imagejpeg($thumb,$outfile);
    
    # Change permissions on the new file - needed for proper web display
    #  default of newly created file is 600 - need it to be 644
    chmod($outfile, 0644);
    
  • Resizing Images w/PHP | Source Code

    All of these examples are served over the web, but they just as easily could have been created via the command-line interface and scheduled via cron. If users are uploading lots of images, and you wish to have thumbnails of these assets, you could scehdule a script to run every 5 minutes that resizes any newly uploaded images and deposits the output in a thumbnail directory.


    Intro to Regular Expressions   [top]
    One of the most powerful tools for matching and manipulating text is the regular expression (regex). In short, it's the ability to match specific patterns in text, and then perform tasks based upon how these patterns match strings of interest. For example, if we look at a US phone number:

    212-555-1212
    
    we inherently know that we should see 3 digits for the area code, 3 digits for the exchange, and 4 for the rest. But what if, in an HTML form that we were going to parse and put into our DB, someone typed:
    212-5551-212
    
    A typo for sure, but the meaning is clear. In the end, as humans, we look for 10 digits in a US phone number. If we see 11 digits, and the first digit is a "1", we know we don't need the leading "1":
    1-212-555-1212
    
    Using a regular expression, we could strip out all non-digit chars and then check the length to see if it's legit.

    Regular expressions come in a few different flavors, with the most popular based upon the syntax of Perl:

  • Great Perl Regex Tutorial
  • Nice Perl Regex Reference
  • The Gory Details

    For reference, the atoms of pattern matching in Perl are:

    Metacharacters
    ^   Match the beginning of the line
    .   Match any character (except newline)
    $   Match the end of the line (or before newline at the end)
    |   Alternation
    ()  Grouping
    []  Character class
    
    Useful Patterns
    \w  Match a "word" character (alphanumeric plus "_")
    \W  Match a non-word character
    \s  Match a whitespace character
    \S  Match a non-whitespace character
    \d  Match a digit character
    \D  Match a non-digit character
    
    Quantifiers
    *      Match 0 or more times
    +      Match 1 or more times
    ?      Match 1 or 0 times
        Match exactly n times
    {n,}   Match at least n times
    {n,m}  Match at least n but not more than m times
    

    You can test these out using these interactive regular expression tools for PHP below. Note that they are password-protected. Please use your regular Stage login to access:

  • Regular Expression Testbed - Matching
  • Regular Expression Testbed - Replacing
  • Regular Expression Testbed - Both

    For some more background on regular expressions, here's what the WikiPedia has to say, along with a set of bookmarks for further study:

  • WikiPedia entry for Regular Expressions
  • Class Bookmarks on Regular Expressions


    Using HTTP in PHP   [top]
    In the same way you used the mail() function to interface to SMTP to send e-mail, you can use HTTP to get the contents of a web page into a string in PHP. The Client URL (Curl) library excels at this:

    $urlToGet = 'http://www.yahoo.com';
    
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $urlToGet);
    
    # This option causes the curl_exec function to return response as a string
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    
    # Use the following if you need to submit a FORM
    # curl_setopt ($ch, CURLOPT_POST, 1);
    # curl_setopt ($ch, CURLOPT_POSTFIELDS, "zip=10003");
     
    $rawText = curl_exec($ch);
    curl_close($ch);
    
    $statusMsg = "Successfully obtained: $urlToGet";
    
    The real trick with getting web pages is parsing them. Fortunately, PHP has lots of methods for manipulating text. One method is to use unique tokens to define where the text of interest starts and where it ends. For example, if we're trying to find all the links in a page, we know:

    <a : indicates that a link is present
    href=" : indicates start of a link
    "> : indicates end of a link and start of the linked text
    </a> : indicates end of the linked text
    
    So we can use a regular expression to find out where these things occur. Then with the preg_match_all() function, these values will be populated into an array, which we can then traverse to find all the URLs and link titles:
    $parts = array();
    $pattern = '/<a .*?href="(.*?)".*?>(.*?)<\/a>/s';
    preg_match_all($pattern,$rawText,$parts);
    
    # $parts[0] now contains the entire part of the text that matched the expression
    # $parts[1] now contains the URLs that matched between href="(.*?)"
    # $parts[2] now contains the linked text that matched between >(.*?)<\/a>
    
    # Uncomment this is you want to view the entire result array
    # print_r($parts);
    
    # htmlspecialchars() subs the HTML equiv for certain chars, i.e. & for &
    for($i=0; $i<count($parts[1]); $i++)
    { $parts[1][$i] = trim($parts[1][$i]);
      $parts[2][$i] = trim($parts[2][$i]);
      print "<li>".htmlspecialchars($parts[1][$i])." - ".htmlspecialchars($parts[2][$i])."\n";
    }
    
  • PHP HTTP Example | Source Code
  • PHP Curl Library
  • Regular Expressions - Perl-Compatible


    Using FTP in PHP   [top]
    PHP has an included FTP extension that allows scripts to transfer files between the local server and a remote FTP host.

    # Get our FTP account info (username, pw, remote FTP host)
    require "info_ftp.php";
    
    /*
    # Info to define in info_ftp.php
    $username = "[your FTP username]";
    $password = "[your FTP password]";
    $host = "ftp.domain.com";  # Change to name of remote FTP host
    */
    
    $source_file = "test.pdf"; # local file to send to remote host
    $destination_file = "test.pdf"; # name of this file on remote host
    
    $remote_file = "test.pdf"; # remote file to transfer to local server
    $local_file = "test_downloaded.pdf"; # name of this file on local server
    
    $file_type = FTP_BINARY; # Constant known to PHP
    # $file_type = FTP_TEXT; # Use this if transferring a text file
    
    
    # Set up basic connection
    $ftp = ftp_connect($host); 
    
    # Login with username and password
    $login_result = ftp_login($ftp, $username, $password); 
    
    # Check connection
    if ((!$ftp) || (!$login_result))
    { 
      print "FTP connection has failed!\n";
      print "Attempted to connect to $host for user $username\n"; 
      exit; 
    }
    else
    {
      print "Connected to $host, for user $username\n";
    }
    
    # Turn passive mode on - needed if the script is running on a server behind a firewall
    #  This means that the script will initiate the data connections, not the remote host
    ftp_pasv($ftp, true);
    
    # Get listing of the current directory and output
    $contents = ftp_rawlist($ftp, ".");
    print "\n--- Before ---\n";
    foreach ($contents as $line) print "$line\n";
    print "\n";
    
    # Attempt to upload this file and check status
    $upload_status = ftp_put($ftp, $destination_file, $source_file, $file_type); 
    if (!$upload_status)
    { 
      print "FTP upload has failed!\n";
    }
    else
    {
      print "Uploaded $source_file to $host as $destination_file\n";
    }
    
    # Get listing of the current directory and output
    $contents = ftp_rawlist($ftp, ".");
    print "\n--- After ---\n";
    foreach ($contents as $line) print "$line\n";
    print "\n";
    
    # Attempt to download this file and check status
    $download_status = ftp_get($ftp, $local_file, $remote_file, $file_type); 
    if (!$upload_status)
    { 
      print "FTP download has failed!\n";
    }
    else
    {
      print "Downloaded $remote_file from $host as $local_file\n";
    }
    
    # Close the FTP connection 
    ftp_close($ftp);
    
  • PHP FTP Example | Source Code
  • PHP FTP Extension
  • PHP FTP Examples


    Using IMAP in PHP   [top]
    IMAP stands for Internet Message Access Protocol and is the one of the main methods for retrieving email from a remote server (along with POP3). PHP has an included IMAP extension that allows scripts to retrieve email messages via both IMAP and POP3. This means you can use PHP to get email from Gmail, NYU, or any other mail provider that allows remote access via IMAP or POP3.

  • Sample raw email message
    # Get our email account info (email, pw, mail host connection string)
    require "info_imap.php";
    
    #########################################################
    # Some possible connection options to define in info_imap.php
    #########################################################
    /*
    
    # Non-secure POP3 - Change "mail.domain.com" to your POP3 server name
    $username = "[your email address]";
    $password = "[your email password]";
    $host_conn = "{mail.domain.com:110/pop3/notls}INBOX";
    
    # Non-secure IMAP - Change "mail.domain.com" to your IMAP server name
    $username = "[your email address]";
    $password = "[your email password]";
    $host_conn = "{mail.domain.com:143/imap/notls}INBOX";
    
    # Secure POP3 - NYU acct
    $username = "[your NYU email address]";
    $password = "[your NYU email password]";
    $host_conn = "{mail.nyu.edu:995/pop3/ssl/novalidate-cert}INBOX";
    
    # Secure IMAP - NYU acct
    $username = "[your NYU email address]";
    $password = "[your NYU email password]";
    $host_conn = "{mail.nyu.edu:993/imap/ssl}INBOX";
    
    # Secure IMAP - Gmail acct
    $username = "[your Gmail email address]";
    $password = "[your Gmail email password]";
    $host_conn = "{imap.gmail.com:993/imap/ssl}INBOX";
    
    */
    
    # Open a connection to the email server with the proper credentials
    $mbox = imap_open($host_conn, $username, $password);
    
    # Get the number of msgs
    $num_msg = imap_num_msg($mbox);
    print "Num of msgs: $num_msg\n";
    
    # Loop through the msgs (email msgs start at 1)
    for($msg_num=1; $msg_num<=$num_msg; $msg_num++)
    {
      # Get the header info for the current msg and put into object
      $header_obj = imap_headerinfo($mbox,$msg_num);
    	
      print "$msg_num: Message ID: " . $header_obj->message_id ."\n";
      print "$msg_num: Date: " . $header_obj->date ."\n";
      print "$msg_num: From: " . $header_obj->fromaddress ."\n";
      print "$msg_num: Subject: " . $header_obj->Subject ."\n";
    }
    
    # Close the connection
    imap_close($mbox);
    
  • PHP IMAP Example | Source Code
  • PHP IMAP Extension
  • imap_open() function
  • imap_headerinfo() function


    Creating PDFs in PHP   [top]
    FPDF is a free PHP class which allows to generate PDF files with pure PHP and requires no special installation other than uploading a folder to your server account. There are other libraries available as well, but creating a "Hello, World" PDF using PHP and FPDF is as easy as:

    <?
    require "fpdf.php";
    
    $pdf=new FPDF();
    $pdf->AddPage();
    $pdf->SetFont('Arial','B',16);
    $pdf->Cell(40,10,'Hello World!');
    $pdf->Output('output.pdf');
    ?>
    
  • PHP PDF Example | Source Code | PHP PDF Example
  • FPDF Library


    Site Architecture Basics   [top]
    One of the basic premises of building larger websites consisting of many pages is that you want to write a particular piece of code once and only once. If certain sections of PHP/HTML are used over and over throughout the site, then it is better to separate that code into its own PHP file, and then use the include or require keyword on that file whenever it is time for that code to appear.

    If you look at the class notes carefully, you'll see that until the actual notes start, the top portion of the page is always the same. Similarly, you'll see that the bottom section below the horizontal line is always the same as well. These two chunks of code could be separated into their own files, say, header.php and footer.php. Then in our main PHP file, we could say:

    <?
    include "header.php";
    
    # Write our main content here
    
    include "footer.php";
    ?>
    
    When a site has multiple items of the same type to display (blog posts, news items, products, et al), the most common way to display them is in a laundry list. The laundry list usually presents a quick synopsis of an item - for a news item, this might be the title, an image thumbnail, the time it was filed, and the source (AP, Reuters, etc). Most importantly, it will contain a link to a more detailed version of that item, commonly called a profile page. On this page, all details relating to this item will be shown. Commonly, there is a template file for showing a laundry list of items, and this template will usually allow the user to "page" through the results, showing one set of items at a time. Each link in the laundry list usually points to a single profile template, and the database ID of that item is passed along on the URL string to make it unique:
    <b>ITP News Flashes</b>
    <p>
    <ul>
    <li><a href="item.php?i=57">Boredom Overtakes DWD Class</a></li>
    <li><a href="item.php?i=42">Dan O'Sullivan is America's Next Top Model</a></li>
    <li><a href="item.php?i=79">New Swimming Pool Installed in P-Comp Lab</a></li>
    </ul>
    
    Inside the profile template (item.php), you look in the $_GET superglobal for the ID of the item of interest ($_GET["i"]). This ID can be used in a SQL SELECT statement to get all the info for the specific item:
    <?
    $SqlStatement = "SELECT * FROM item WHERE id=".$_GET["i"];
    ?>
    

    Thus, for interaction like this, we would need a template for the laundry list, and a template for the profile page. Let's see how these templates function in a sample site: Music 101

    The Music 101 Demo Site

  • Excel Spreadsheet of Music 101 demo site data
  • Data Population SQL

    Music 101 Database Tables

  • Musician Table
  • Instrument Table
  • Musician x Instrument Linking Table
  • US States Table
  • Country Table
  • Site Administrator Table

    Music 101 End-User Pages

  • Site Library Functions
  • Site Home Page | Source Code
  • Musician Laundry List | Source Code
  • Musician Profile Page | Source Code
  • Generic Site Template | Source Code
  • Header File | Footer File

    Music 101 Admin Pages (un: itpdb - pw: interact)

  • Admin Login Page | Source Code
  • Edit Musician List | Source Code
  • Edit Musician Profile | Source Code

    HTML Template Integrated Templates (PEAR Module)

  • HTML Template IT Tutorial - This is also covered in your PHP/MySQL book, pp. 232-247
  • Template for DWD Class Notes
  • Contents File for Class #7
  • Wrapper File for Class #7


    Related Resources   [top]

  • GD Library Reference
  • PHP Curl Library
  • Regular Expressions - Perl-Compatible
  • PHP FTP Extension
  • PHP IMAP Extension


  • Home  ·  Syllabus  ·  Submissions  ·  Resources
    © 2003-2009. All rights reserved.