Some basic security

Apache is designed so that every directory where you have created web content should have an index file. This is normally index.html, but you may also add other extensions, such as index.php, index.htm or others. The part of httpd.conf that determines this is:

#
# DirectoryIndex: Name of the file or files to use as a pre-written HTML
# directory index.  Separate multiple entries with spaces.
#
<IfModule mod_dir.c>
  DirectoryIndex index.html index.php3 index.php index.htm index.shtml index.cgi
</IfModule>

Apache, by default, is going to show us the directory listing if we don't have one of these files in a directory. That's probably not a good idea from a security standpoint. We all get lazy and we may place temporary files in a webserver that we don't mean for the world to see. The best thing is to nip this problem in the bud and keep Apache from showing directory listings. You need to find this line in httpd.conf:

Options Indexes Includes FollowSymLinks MultiViews

It's a good idea to remove the Indexes option here. This will prevent a website visitor from seeing what's in the individual directories.

Document root and cgi-bin

The document root means the directory where Apache serves the web pages from by default. You will see a line like this in your httpd.conf:

#
# This should be changed to whatever you set DocumentRoot to.
#
<Directory /var/www/>

You'll find that the Apache developers are good at explaining what things mean. That is, if you prefer your web pages to be in another place, you should change it here. Even if you want them in another place, you may not want to change this right away. Further along, I'll explain the concept of "virtual" websites, which means "hosting" more than one website. However, if you're only going to be serving one set of pages, you may change this to wherever you want. You may also want to have a look at this line:

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/

This is the directory where you can place your cgi-bin scripts. Those of us who have some web development experience will know what a cgi-bin script is. In case you don't, it's a program that's mean to be run from a form on a web page. If you've submitted data to a website in the past, chances are you've used a cgi-bin script. If you've created a webpage with a form to submit information, you'll normally find something like this on the page:

<form action="/cgi-bin/myscript.cgi" method="get">

Your script is placed in the cgi-bin directory and Apache knows where to find it when the form calls it. If you change the above line in Apache to have the scripts located someplace else, you also need to change a line a little farther below:

<Directory /usr/lib/cgi-bin/>
    AllowOverride None
    Options ExecCGI
    Order allow,deny
    Allow from all
</Directory>

Again, as I mentioned above, you may not even need to make these changes if you're going to be maintaining several websites on the same server. More on that further ahead.

Personal user sites

If you give somebody an account on the machine running Linux and Apache this person has the ability to run his/her own personal website. I'm sure many of you have seen sites like: http://www.domain.com/~larry/ . This is because the UserDir module is activated in httpd.conf:

LoadModule userdir_module /usr/lib/apache/1.3/mod_userdir.so

And farther down you will find this section:

#
# UserDir: The name of the directory which is appended onto a user's home
# directory if a ~user request is received.
#
<IfModule mod_userdir.c>
    UserDir public_html
</IfModule>

By default, Apache designates the directory where the public webfiles (and remember, these are public!) are found to be public_html. There's no reason why you can't change this name to website or any other meaningful name. You could even comment these lines out if you don't want the users on your system to have a personal website. If you do allow this, you may want to skip down to the next line:

#
# Control access to UserDir directories.  The following is an example
# for a site where these directories are restricted to read-only.
#
<Directory /home/*/public_html>

There are some options here as well as to how the site will work. You should remove the option Indexes from here as well, as we did earlier.

Server-side includes

As most of us webmasters know, websites, even fairly small ones, can become unruly and hard to maintain. One of the ways to keep your website management tasks to a minimum is to use server side includes. A server-side include is a way to make other files part of the page. That way, for example, you can have the same navigational buttons on every page of your website. This keeps you from having to include HTML code for this in every page you create. You would just insert something this in your pages:

<!--#include virtual="/navbar.shtml"  -->

Apache designates by default the extension .shtml to be used for server-side includes. If you want plain .html files to be used as server-side includes as well, you need to add this to httpd.conf. Open the file and look for these lines:

AddType text/html .shtml
AddHandler server-parsed .shtml

Now add these lines:

AddType text/html .html
AddHandler server-parsed .html

Alias directories

Some applications that run under Linux use the Apache webserver to display some of its content. There are systems to display man pages in the browser. Some Linux distributions use Apache to give you a web-based help system and documentation. They will place their documents outside of the root webserver directory. To access this "outside" content, we need to create "Alias" lines in httpd.conf or else it will be inaccessible from a web browser. In the following example, I'll show you what I need to add to httpd.conf so that visitors could see my mailman mailing list public archives.

I found the following line in httpd.conf:

#
# Aliases: Add here as many aliases as you need (with no limit). The format is
# Alias fakename realname

Then I added these lines:

# Aliases for mailman
Alias /pipermail/ /var/lib/mailman/archives/public/
Alias /images/ /usr/share/doc/mailman/images/

This means that a person only has to type http://www.mydomain.ork/pipermail/ into a browser to see the mailing lists located in /var/lib/mailman/archives/public/. If there are any images on the page, they will also be displayed.

Mailman also works by letting visitors sign up to the mailing list. The whole system is based on Python scripts. These scripts are not in the cgi-bin that Apache knows about. They are in a different place. So we also need to add these lines below so that Apache can find these scripts.

#ScriptAlias for mailman ScriptAlias /mailman/ /usr/lib/mailman/cgi-bin/

This is known as a ScriptAlias and you will find this section below the Alias section in httpd.conf.

As you can see, Apache is very versatile - allowing us to configure it to use web content from third-party applications with relative ease.

No more 404s

As a web surfer, nothing annoys me more than a "404 not found" page. This is what Apache will show you by default when you request a page that has disappeared.

404 is the Apache code for a request for a page that does not exist. Web-savvy people now refer to a missing page as a "404".

Not Found
The requested URL /bla.html was not found on this server.

Apache/1.3.26 Server at www.dominio.ork Port 80

As it's frustrating as a user to find this page, it's my job as a webmaster to make sure it doesn't appear. There is really no excuse for this occurring. The .htaccess provides a means to redirect users to content if you've moved it. Let's say you have a site that talks about a club you have set up. You have a page dedicated to your August 2002 barbecue. You've created a directory called /bbq. The club is successful and another year goes by and you have another barbecue - this time in August 2003. You decide to make the website more manageable and so you create two directories - bbq02 and bbq03 with pages about the festivities. Now, a problem arises. People might have bookmarked the page dedicated to the hilarious food fight at the 2002 shindig: http://www.ourclub.ork/bbq/foodfight.html. Now, of course, you've moved it. I would say that it's your duty as a good webmaster to provide a re-direct. Since /bbq no longer exists, we can create an .htaccess file in our webserver root directory and add the following entry.

# redirects

RedirectPermanent /bbq/foodfight.html http://www.ourclub.ork/bbq02/foodfight.html

You should add any and all web pages that you've moved to /bbq02 to your .htaccess file as well.

Friendly greetings

If you've done your work diligently in providing re-directs for moved pages, then you can be fairly confident that any 404s that are generated in your web logs are probably the result of things beyond your control. Users will often type bad URLs into their browsers and other webmasters may make mistakes providing a link to one of your pages. In these cases, it's probably a good idea to provide and alternative web page to replace Apache's standard 404 warning. Again, .htaccess provides you with this possibility. How elaborate a substitute page you provide depends on you and your imagination (and perhaps your good taste!)

It's a good idea to use grep to look for 404s in your Apache access logs at least once a week or so. You may have re-directed users to other pages but you may have overlooked the fact that people may have bookmarked specific images as well. Apart from the ease-of-use issues, it is also a basic security measure. You may find one IP address generating a lot of 404s. This could be an individual checking out your site as a prelude to a defacing or other attack on your website. You may then want to take steps such as firewalling this IP from your network or, if the situation warrants, contacting the owner of the netblock.

First, as a website administrator, it's probably a good idea to create a directory for administrative needs. Call it what you like - something meaningful to you. Now you can create an alternative page for your 404s and place it in this directory. The page normally has a simply greeting- maybe something like: Oops! We can't find that. and maybe a link back to your home page. If you have search capabilities on the site, you may want to link to those. Again, it is up to you as a web administrator to create something that works for you and your site.

Password protection

Apache also provides a means of keeping people out of certain directories. Again, this depends on some lines placed in .htaccess. Let's go back to your club's website. You may want to create a members-only section to the website that's restricted to those to whom you've given a password. To do this, you would first create the directory and then create an .htaccess file in the directory. Then add the following lines:

AuthUserFile /home/club/.htpasswd
AuthGroupFile /dev/null
AuthName "Our Club - Members Only"
AuthType Basic
<Limit GET>
Require valid-user
</Limit>

Now you must create the file with the users and passwords in it, called .htpasswd. You will notice that we have placed it outside of the web directories as a security precaution. Apache can read it just fine there and there is no risk of it being read by a nasty spider. Here's how you create the .htpasswd file:

htpasswd -c /home/club/.htpasswd joe

Where joe is the first user in the file. That's important because the -c option creates the file. From now on, for every user you want to add, you don't use the -c option. Apache will ask you for the password twice, as is standard in Unix-type applications. Now, when you go to http://www.ourclub.ork/members/secret.html you will get this in your web browser:

Scripts in alternative locations

Another feature we can get via .htaccess is the ability to use scripts outside our cgi-bin directory. This is another good way to increase the manageability of your website. Let's say you have a section of your website for news about your club . You have it in a directory appropriately called /news. You may have a small Perl script that takes news items out of a MySQL database. You could create a directory in /news called /script and then create an .htaccess file with the following lines in it:

Options +ExecCGI
AddHandler cgi-script .cgi

Now, any script with the .cgi (dot-cgi) extension can be executed as a script. Normally Apache wouldn't allow that but these two lines will override that behavior. Of course, there is a good reason for this not being provided by default. It is a potential security risk. Most websites place their cgi-bin directory outside of the web directory - and for good reason. Any script can be executed from it. It's much more difficult for someone to get at the cgi-bin directory if it's in some other place. But if we place it inside a website's content directories, the possibility of someone manipulating it increases. If you do choose to use this feature, make sure that the scripts are well-written and free from exploitable bugs, such as cross-site scripting vulnerabilities and that few people - the fewer the better -have upload privileges.