nscgi(n) 4.99.30 nscgi "NaviServer Modules"

Name

nscgi - NaviServer CGI Interface Guide

Table Of Contents

• Table Of Contents
• Description
• What is CGI and How Does it Work?
• Configuring CGI with NaviServer
• How Web Pages Run CGI Programs
  • URLs that Run CGI Programs
• Input to CGI Programs
  • Accessing Environment Variables
  • Standard Environment Variables
• Output from CGI Programs
• Advice for CGI Programming
• CGI Examples
• Keywords

Description

What is CGI and How Does it Work?

CGI (Common Gateway Interface) is a standard way of running programs from a Web server. Often, CGI programs are used to generate pages dynamically or to perform some other action when someone fills out an HTML form and clicks the submit button. NaviServer provides full support for CGI v1.1.

Basically, CGI works like this:

A reader sends a URL that causes the NaviServer to use CGI to run a program. The NaviServer passes input from the reader to the program and output from the program back to the reader. CGI acts as a "gateway" between the NaviServer and the program you write.

The program run by CGI can be any type of executable file on the server platform. For example, you can use C, C++, Perl, Unix shell scripts, Fortran, or any other compiled or interpreted language. You can also use Tcl scripts with CGI, though the NaviServer API will not be available to them.

With NaviServer, you have the option of using the embedded Tcl and C interfaces instead of CGI. Typically, the Tcl and C interfaces provide better performance than CGI (see the NaviServer Tcl Developer's Guide for information on the Tcl interface and the NaviServer C Developer's Guide for information on the C interface).

You may want to use CGI for existing, shareware, or freeware programs that use the standard CGI input, output, and environment variables. Since CGI is a standard interface used by many Web servers, there are lots of example programs and function libraries available on the World Wide Web and by ftp. This chapter describes the interface and points you to locations where you can download examples.

For example, suppose you have a form that lets people comment on your Web pages. You want the comments emailed to you and you want to automatically generate a page and send it back to your reader.

1. The reader fills out your form and clicks the "Submit" button. The FORM tag in your page might look like this:

    <FORM METHOD="POST" ACTION="/cgi-bin/myprog">
   

   The METHOD controls how the information typed into the form is passed to your program. It can be GET or POST. The ACTION determines which program should be run.

   Other ways for a reader to run a program are by providing a direct link to the program without allowing the reader to supply any variables through a form, or by using the ISINDEX tag.

2. When NaviServer gets a request for a URL that maps to a CGI directory or a CGI file extension (as defined in the configuration file), it starts a separate process and runs the program within that process. The NaviServer also sets up a number of environment variable within that process. These environment variables include some standard CGI variables, and optionally any variables you define in the configuration file for this type of program.

3. The program runs. The program can be any type of executable program. For example, you can use C, C++, Perl, Unix shell scripts, or Fortran.

   In this example, the program takes the comments from the form as input and sends them to you as email. If the form method is GET, it gets the input from an environment variable. If the form method is POST, it gets the input from standard input. It also assembles an HTML page and sends it to standard output.

4. Any information the program passes to standard output is automatically sent to the NaviServer when the program finishes running.

5. The server adds any header information needed to identify the output and sends it back to the reader's browser, which displays the output.

Configuring CGI with NaviServer

You can control the behavior of NaviServer's CGI interface by setting parameters in a configuration file. For example, you can control which files and directories are treated as CGI programs, you can determine how to run various types of programs, and you can set a group of environment variables for each type of program you use.

Note that if you're defining multiple servers, you will need to configure the CGI interface for each server configuration.

To enable and configure CGI:

1. Edit your NaviServer configuration file, usually named nsd.tcl.

2. Choose the server for which you want to enable CGI (such as for example the server named Server1). Then add the CGI module to that server such it will be loaded at start time. For example:

    ns_section "ns/server/Server1/modules"
    ns_param nscgi nscgi.so
   

3. Add a section for the server with the suffix module/nscgi to configure the modules with certain parameters. One has to define typically the mappings via the parameter Map to indicate, what HTTP-method with which path should be directed to a corresponding CGI program. Typically, the mapping will point to a directory on the machine in which the CGI scripts are located.

   The value specified via the parameter map is of the form

   method pattern ?path?

   where method refers to the HTTP request method (i.e., HEAD, GET, POST, etc.), pattern refers to either a relative URL or a glob pattern to match on the HTTP request. This is how nscgi determines whether a request is a CGI request. path is optional and indicates either the directory where the CGI executables can be found, or the executable that should be used to fulfill the CGI request.

   If path is not specified, then the URL must refer a file which is the CGI executable. If path is specified and is a directory, then the filename portion of the URL must refer to a CGI executable in that directory. Otherwise, path must refer to a CGI executable which will handle all requests for this pattern. For example:

    ns_section "ns/server/Server1/module/nscgi" {
      ns_param Map  "GET /cgi /usr/local/cgi"
      ns_param Map  "POST /*.cgi"
    }
   

4. If you want to call a CGI program (script) which requires an interpreter (e.g., Perl or bash), you will need to define the CGI interpreters via the module parameter Interps.

   • Add a definition for the Interps parameter to your CGI configuration section using e.g. the name CGIinterps.

      ns_section "ns/server/Server1/module/nscgi" {
        ns_param Map    "GET /cgi /usr/local/cgi"
        ns_param Map    "POST /*.cgi"
        ns_param Interps CGIinterps
      }
     

   • Then add a section under ns/interps for the chosen name (here cgi) and add there the mappings from file extensions to the script interpreters. When a CGI script with a specified extension is to be executed (below .pl and .sh), NaviServer will call the script via the named executables.

      ns_section "ns/interps/CGIinterps" {
        ns_param .pl  "c:\perl\bin\perl.exe"
        ns_param .sh  "c:\mks\mksnt\sh.exe(MKSenv)"
      }
     

     If no mapping of extensions to script interpreters is provided, the called script must have executable permissions.

     Files in the cgi directory without execute permissions are served as plain files when the parameter allowstaticresources is set to true. This way images and such in can be served directly from a cgi bin directory.

5. In case the CGI script or the script interpreter requires additional environment variables, one can define this via the module parameter Environment and a matching section for the detail definitions.

   • Add a definition for the Environment parameter to your CGI configuration section.

      ns_section "ns/server/Server1/module/nscgi" {
        ns_param Map         "GET /cgi /usr/local/cgi"
        ns_param Map         "POST /*.cgi"
        ns_param Interps     cgi
        ns_param Environment cgi
      }
     

   • Then add a section under ns/environment for the chosen name (here cgi) containing the required environment variable definitions.

      ns_section "ns/environment/cgi" {
        ns_param FOO BAR     ;# defines environment variable "FOO=BAR"
        ns_param TMP /tmp    ;# defines environment variable "TMP=/tmp"
      }
     

6. Further configuration parameter for module/nscgi are:

   • gethostbyaddr is a boolean parameter to indicate whether to resolve the peer IP address to its hostname when setting the REMOTE_HOST environment variable. If false, the peer IP address will be used instead. Caution: turning this option on can negatively impact performance due to the overhead of having to perform a DNS look-up on every CGI request. Default: off

   • limit is an integer parameter to indicate the maximum number of concurrent CGI requests to execute. 0 means unlimited. Default: 0

   • maxinput is an integer parameter to indicate the maximum in bytes to accept in the HTTP request. 0 means unlimited. Mostly useful to limit the size of POST'ed data. Default: 1024000

   • maxwait is an integer parameter to indicate the amount of time to wait in seconds in the queue when the concurrency limit has been reached. Server will respond with a "503 Service Unavailable" error on timeout. If limit is set to 0, this setting will have no effect. Default: 30

   • systemenvironment is a boolean parameter to controls whether the CGI will inherit the server process's environment variables or not. Enabling this could potentially leak sensitive information from the parent's environment if the CGI displays its environment variables which is the behavior of some common error-handling code. Default: false

   • allowstaticresources is a boolean parameter to controls whether static resources (e.g. images) can be served directly from the CGI bin directory. Default: false

How Web Pages Run CGI Programs

There are several ways a Web page can run a CGI program:

• Direct Link: A link in your page can reference a CGI program directly. Normally, such links are used when you do not need to send any variables to the program.

• ISINDEX Tag: This tag can be included in the <HEAD> section of the page. For example, you could add this tag:

   <ISINDEX HREF="/cgi-bin/search.pl" PROMPT="Search for:">
  

  Normally, the ISINDEX tag is used to run search programs, but this is not a requirement. You can use this tag to run CGI programs if you don't care where the field is located in your page and your program accepts a single variable.

  When you use this tag, the browser adds a field to the Web page. Readers can type a string in this field. The server decodes the text the reader typed and sends it to your CGI program as separate command-line arguments. The advantage is you don't need to decode the input.

• Form: You can create a form with multiple fields. Use forms to run CGI programs when your readers can choose or type values for one or more variables. The METHOD attribute of the HTML form can be either GET or POST.

  • The GET method causes the field names and values to be passed to the program in the QUERY_STRING environment variable.

  • The POST method causes the field names and values to be passed to the program through standard input. If the input from your form may be long, it is best to use the POST method because long strings can be truncated when they are assigned to an environment variable.

Input to CGI Programs

CGI programs can get input from these sources:

• Command line: The only type of input that go to the command line of a CGI program are values typed into an ISINDEX field and query strings that do not contain an "=" sign (from a direct link). Forms cannot send variables to the command line.

• Environment variables: A number of standard environment variables are always available to the CGI program. You can specify additional environment variables in the NaviServer configuration file. And, if the METHOD for a form is GET, the field names and values are stored in the QUERY_STRING environment variable.

• Standard input: If the METHOD for a form is POST, the field names and values are sent to standard input.

 #include <stdlib.h>;
  
 char *browser = getenv("HTTP_USER_AGENT");

 $browser = $ENV{'HTTP_USER_AGENT'};

 BROWSER=$HTTP_USER_AGENT

 set BROWSER = $HTTP_USER_AGENT

 */*, application/x-navidoc
 */*, image/gif, image/x-xbitmap, image/jpeg

 # -- %23
 = -- %3D
 / -- %2F
 % -- %25
 : -- %3A
 \ -- %5C
 & -- %26
 ; -- %3B
 tab -- %0A
 + -- %2B
 ? -- %3F
 return -- %09

 #include <stdio.h>
 #include <stdlib.h>
 #define MAX_CONTENT_LENGTH 10000
 
 char *inputlenstr;
 int inputlen;
 int status;
 char inputtext[MAX_INPUT_LENGTH+1];
 inputlenstr = getenv("CONTENT_LENGTH");
 inputlen = strtol(inputlenstr, NULL, 10);
 status = fread(inputtext, 1, inputlen, stdin);

 read input (reads contents to $input variable)

Output from CGI Programs

To send output from a CGI program to the reader's browser, you send the output to the standard output location. Different languages allow you to send text to standard output in different ways. Here are some examples:

C or C++

 #include <stdio.h>
 #include <stdlib.h>
 
 printf("Content-type: text/html\r\n\r\n");
 printf("<HEAD><TITLE>Hello</TITLE></HEAD>");
 printf("<BODY>You are using %s.</BODY>",
 getenv("HTTP_USER_AGENT") );

Perl

 #!/opt/local/bin/perl
 print "Content-type: text/plain\r\n\r\n";
 for my $var ( sort keys %ENV ) {
    printf "%s = \"%s\"\r\n", $var, $ENV{$var};
 }

Bourne shell

 echo Content-type: text/html
 echo 
 echo \<HEAD\>\<TITLE\>Hello\</TITLE\>\</HEAD\>
 echo \<BODY\>
 echo You are using $HTTP_USER_AGENT.\</BODY\>

HTTP Headers

Messages sent between a Web browser and a Web server contain header information that the software uses to determine how to display or interpret the information. The header information is not displayed by the browser.

The NaviServer automatically generates some HTTP header information and your program can add other information to the header.

Header Information Generated by NaviServer

When your CGI program sends output to the standard output location, the server automatically adds the following HTTP header information before sending the output to the reader's browser:

 HTTP/1.0 200 OK
 MIME-Version: 1.0
 Server: NaviServer/3.0
 Date: Monday, 06-Nov-95 17:50:15 GMT
 Content-length: 20134

However, if the name of your CGI program begins with "nph-", the NaviServer will not parse the output you send. Instead, the output is sent directly to the client. In this case, you must include the information above in your output. Generally, it is best to avoid using this "non-parsed header" feature because any errors may be sent to standard output and could make the header information incorrect. Also, with non-parsed headers, the server does not interpret the output, so the response code and content length are written out as 0 (zero) and 0 (zero) in the access log file.

Header Information Generated by Your Program

You can specify header information at the beginning of the output you send back to the client. After the header, add a blank line and then start the output you want the reader to see. The blank line is required. Your program should always send the Content-type header (unless you are using the Location header). The other headers listed below it are optional. For example,

 Content-type: text/html
 
 <HTML>
 <HEAD><TITLE>My title</TITLE></HEAD>
 <BODY>text goes here...</BODY>
 </HTML>

Content-type:

You should always use this header to specify the MIME type of the output you are sending (unless you are using the Location header). If you are sending an HTML page as output, use a Content-type of text/html. If you are sending untagged text, send a Content-type of text/plain. If you send images, you might use a Content-type of image/gif or image/jpeg. You can send any type of output from your CGI program -- just be sure to specify the correct MIME type.

Example: Content-type: text/html

Content-encoding:

Use this header if the output you are sending is compressed. The Content-type should specify the type of the uncompressed file. For example, use x-gzip for GNU zip compression and x-compress for standard UNIX compression.

Example: Content-encoding: x-compress

Expires:

Use this header to specify when the browser should consider the file "out-of-date". Browsers can use this date to determine whether to load the page from their local cache of pages or to reload the file from the server.

Example: Expires: Monday, 06-Nov-95 17:50:15 GMT

Location:

Use this header if you want to send an existing document as output. The server automatically sends the document you specify to the browser. You will probably want to specify a full URL for the Location. If you specify a complete URL (such as, http://www.mysite.com/out/response.htm), relative references in that file will be resolved using the information in the URL you specify. If you specify a relative URL (such as /out/response.htm), references in that file will be resolved using the directory that contains the CGI program.

If you send a Location header, you do not need to send a Content-type header. However, you may want to send HTML-tagged text including a link to the location for browsers that do not support this type of redirection. You can specify any type of URL as the output location. For example, you can send an FTP, Gopher, or News URL.

Example: Location: http://www.my.org/outbox/accepted.html

Status:

The NaviServer sends a status code to the browser in the first line of every HTTP header. The default status code for success is "200 OK". You can send other status codes by specifying the Status header.

Some browsers may not know how to handle all HTTP status codes, so your program should also send HTML output after the header to describe error situations that occur.

Example: Status: 401 Unauthorized

Sending HTML

To send a Web page to a reader's browser from a CGI program, first output this line followed by a blank line:

Content-type: text/html

Then, generate and output the HTML tags and content that make up the page. You can send any HTML tags you would normally use when creating pages.

If the file you want to send already exists, you can use the Location header described in the previous section to send that file as output from the CGI program.

Advice for CGI Programming

• Which language should I use? You can use any language you feel comfortable programming in. Of course, programs usually run faster in compiled language, so if your program is computationally intensive, you might want to use C or another compiled language. Most of the examples and shareware programs available on the Web are written in C or Perl.

• How can I prevent CGI programs from causing security problems? A CGI program is basically a program that you let anyone else in the world run on your system. Someone with bad intentions could cause you some problems if you don't follow these rules:

  • Keep your CGI programs in a separate CGI directory or give them the file extension you specify in the configuration file. Don't give outsiders write access to these files and directories This should prevent casual users from reading, modifying, or adding CGI programs.

  • Don't allow server-parsed HTML to run on your CGI directory or on files with extensions mapped as CGI programs.

  • Don't trust the data the browser sends to your program. Parse the QUERY_STRING or standard input. If your program is a non-compiled script, characters with special meanings in that language can cause problems if the browser fails to encode them as hexadecimal values.

  • Check for odd filenames and directory paths in the input. For example, you should be careful about allow paths containing: ., ../, //, or the name of the directory that contains your CGI programs.

  • Be careful with statements that construct and execute a command line or system call using input from the reader. For example, be careful using the eval statement in Perl and the Bourne shell. If the reader sends input that begins with a semicolon (;), they may be able to get your system to perform any command they like. Likewise, if you use calls to popen() and system(), make sure you put a backslash (\) before any characters with special meaning in the shell that will run.

• How can I debug my CGI programs? Errors that go to the stderr location will be available in the NaviServer's server.log file.

  One simple way to debug CGI programs is to temporarily include print statements that send additional diagnostic information to the client or to a file. If your program is written in C and you have a debugging tool on your system, you can call sleep (or use a long loop) at the beginning of the program. Then, you can attach to the program with the debugger while the program is sleeping.

  If your programs are not executed, make sure the program file allows read and execute access.

• How can I debug my nscgi configuration?

  For debugging the nsgci configuration, you one can turn on logging for the error.log via the command:

   ns_logctl severity Debug(cgi) on
  

CGI Examples

You can download lots of examples and working CGI programs from the Web. Here are some places to look:

• Yahoo's page of links to CGI information: http://www.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/CGI___Common_Gateway_Interface/

• NCSA's ftp site: ftp://ftp.ncsa.uiuc.edu/Web/httpd/Unix/ncsa_httpd/cgi/

  This site includes set of functions for decoding the QUERY_STRING environment variable or standard input to your program in C (ncsa-default.tar.Z), (cgi-lib.pl.Z), and Bourne shell scripts (AA-1.2.tar.Z).

• List of Perl archives: http://www.seas.upenn.edu/~mengwong/perlhtml.html

• Virtual Library list of CGI resources: http://www.charm.net/~web/Vlib/Providers/CGI.html

• Various script examples at MIT: http://www-genome.wi.mit.edu/WWW/tools/scripting/

Keywords

module, nscgi