(Image: Header Graphic)

Friday, December 19, 2014

Doug's Domain

Doug Vetter, ATP/CFI

dvatp.com has been updated!

Some URLs have changed but you will be automatically redirected to the new locations. Please update your bookmarks! Read more...

Website Design and Updates

While the primary goal of this site is to provide information I think others would find helpful I also use it as a laboratory to experiment with web technologies.

Major changes to the design (and design philosophy) are documented here for those that care about these sorts of things.

Changes are listed in reverse chronological order.


Migration to HTML5

After painstakingly migrating a bunch of sloppy HTML4.x code to XHMTL 1.0 transitional a few years back I jumped ship from XHTML because it was essentially declared DoA as a web technology about a year ago. Fortunately, because XHTML syntax is strict by design and HTML5 is nearly as strict, as well as compatible with XHTML syntax, I didn't have to do much to convert the site.

The most odd thing about HTML5 I found is that anchors using the "name" parameter are eliminated in favor of the "id" parameter. I discovered that one of my anchor names conflicted with a CSS id used throughout the site so I just changed the anchor name and moved on.

The entire site was tediously sent through the W3C's validator (in beta at this writing) one article at a time and the entire site is now HTML5 compliant. I'm not really using many of the new features, but it does open the door to new experiments.

Core Overhaul

As I've worked with Codeigniter on more personal and professional projects I've developed a better system to organize views. I began with the simple goal to migrate that organizational structure to the site but a few glances at code I hadn't looked at in years was all it took for me to realize it was time for a major overhaul. I began by installing the latest version of Codeignitor, and that required a transition to PHP5. Before long, I had rewritten the core.

There are three externally visible changes resulting from this overhaul.

  1. The content hierarchy has been flattened once again, so all URIs are in the form:


    You may notice the new URI format is identical to the typical CI URI (controller/method/params) design I maligned years ago. This change comes from a somewhat belated realization that any relationship between the end-user visible site structure and the URI is coincidental. URIs are now used more to transfer web application parameter data than help orient users in a filesystem-like hierarchy as we have breadcrumbs for that.

    The last remnant of the "URI = Site Structure" philosophy here is the section parameter (bmw, aviation, etc.), and I'm not sure how long that will last. For a brief moment during development I actually considered replacing it with something generic like "/article/" or "/a/" for short but ultimately decided to keep the traditional section names to minimize the number of 301 redirects I had to provision.

    This change also reflects changes to the way people look for and view web data today. No one types URIs manually anymore except to get to the site hostname and most everything is found via search engines (nearly 70% of my visitors come via Google or other search engines), which even further reduces the need to type a hostname. It's not surprising, therefore, that some browser developers are thinking of doing away with the traditional URI bar altogether. I'm not sure I'm ready for that as a user, but as a developer I can see the advantages in hiding my application parameter data from the average end user, particularly if the web developer is given control of the URI bar visibility, i.e. if the server application can tell the client's browser to hide the bar.

  2. I've reworked the top banner to add a search bar which simply points to google (it may point to other less invasive search providers in the near future) and I now utilize some of the nicer features of CSS Version 3 now supported in modern browsers including text-shadow, box-shadow, and transparency to give it greater dimension and visual appeal. All of these things were possible before the advent of CSS3, but with greater effort.

  3. The old CSS-based navigation menu has been replaced with a new version I built more or less from scratch, with inspiration from several online resources. The new menu uses some CSS3 technology for visual appeal and works in conjunction with my core logic to highlight the current section. The markup on which the menu is based is now generated with a menu library I ripped from another open source CMS. It strips me of some flexibility, but precludes me having to maintain the menu markup manually, and that's a good thing.

While some things have changed, one thing remains the same - my use of a flat file database. While most websites have evolved to use SQL based relational databases and I use them professionally, I maintain the same flat file database I created nearly ten years ago. The format remains largely unchanged, and the data preserved. I've never lost any data due to "database corruption", so I'm in no rush to change what works.


In the summer of 2010 I acquired an Apple iPad and I soon realized that certain aspects of the site were difficult to use with a touch interface (for example, page navigation), so I tweaked things as needed.


This year the site's popularity exploded. The average number of unique visitors went up around six times and the monthly bandwidth quintupled. Unfortunately, there was a dark side to this new-found popularity -- the increasing abuse of the site content. For this reason, I decided to rework selected aspects of the backend code again and complete some work I'd neglected to finish in either of the last two major updates.

The results of this update include:

  1. Hotlinking Protection: Images are no longer viewable unless they are visited via one of the site's articles. I used to check the referrer information and give the user a pass if the information was blocked. No more. It really irritates me that I have to take this action, but there are far too many assholes on the net right now. Sometimes I wish it was 1990 again. I'd gladly tolerate the 9600 baud access speeds of the day just to be rid of them.
  2. New CodeIgnitor Controller architecture: Rather than deriving my controllers directly from the CI "Controller" class, I created a "GenericController" class that stores some methods common to all the controllers in use on the site. The new class uses accessor methods (set/get) to manage or override various parameters used to generate the pages. This simple but elegant solution eliminated hundreds of lines of duplicate code and will make it a lot easier to support the site.
  3. New CodeIgnitor View architecture: My views were quite crude, but now they make a lot more sense and are a lot easier to maintain. A "page" view serves only as a template for the page layout with header, content, and footer variables. I added logic that uses a "default" view for each section if I don't otherwise specify it. Those of you who poke at the source of my site will also notice that I now waste valuable bandwidth to explain to people that my content is now available for republication only with written approval. Updating the site to show this text would have been a pain in the ass with my last view architecture, but now it's in a single place and easy to update.
  4. Reorganized and improved CSS: Rather than serving one huge CSS file for each page of the site, I cut up the CSS and modularized it so I only send the CSS I need to render a given page.
  5. Added CSS-based hierarchical site menu (look Ma, no Javascript!) based on ADxMenu. I developed something similar to this back in 2003 following Eric Meyer's lead but never published it because I couldn't get certain aspects of the menu to render properly in all browsers. Rather than sink more time into my own project, I decided to try ADxMenu. Looks good so far.



The simplest websites (like mine) traditionally serve data from a directory structure that rigidly defines the structure of the website as viewed by the user. In other words, the user sees what the developer sees. If you browsed to "/bmw/articles/tools/" on this site, for example, the server would traverse a matching directory tree and serve the data in the "tools" directory. This served its function well but it had recently become irritating and limiting from a developer's perspective for a number of reasons.

The solution to the problem was to pitch the outdated core I'd built many years ago before I really knew PHP or how to build websites effectively and replace it with a new PHP-based framework. I'd contemplated building my own, but only foolish developers reinvent the wheel, so I went on a quest for the ultimate prefabricated framework. I had problems finding a framework that would be simple enough to learn quickly, mesh well with my static website and its flat-file contents, and implement the many good practices associated with object oriented design. I ultimately narrowed it down to CakePHP and CodeIgnitor but wound up discarding CakePHP due to complexity and the fact that its primary benefit over other packages was its scaffolding features -- something I wouldn't use.

As I began to survey CodeIgnitor in detail, I found I liked the thorough documentation, PHP 4/5 support, clean interfaces, extensibility, active development schedule with releases every few months and a notable lack of attitude on the part of its developers. Like most frameworks, it supported the so-called MVC (Model / View / Controller) development model that strives to separate the data from the presentation with a intermediary or decision-maker (controller) to translate the raw data from the database into a form acceptable to the user. This MVC format usually extends to the user in the form of a URI in the form "/class/method/parameter1/parameter2/...".

It became apparent that certain aspects of the hierarchial nature of my site's original design were preferable to the traditional MVC URI format, but I also did some reading and stumbled upon the so-called three-click rule. Right or wrong, it aims to point out that users become frustrated if they're required to click more than three times from the front of a website to find the data they need. This forced me to rethink the structure of my site. The end result is a slightly "flatter" structure that's easier for the user to navigate and easier fo the developer to maintain.

One of the other challenges of the MVC URI is building a breadcrumb history such as "Home->Aviation->Reviews". After experimenting for many hours with other people's breadcrumb classes, including one specifically designed to work with CodeIgnitor and the URIs that result from use of Apache's mod_rewrite module, I decided to dump all the complexity and write my own, The result is so stupidly simple I have no idea why I haven't seen an example like it elsewhere. The proof is in the pudding. The breadcrumbs exist, but the backend organization of the data bears no resemblance to the hierarchial structure implied by the breadcrumb list.

I was also motivated to make changes to the organizational structure of the site so I could better serve images and other types of media and do it in a way that would not be abused by people who like to hotlink or embed my content in their pages in defiance of my site's terms of service. While much of this work is not yet complete, the framework to support the work is now in place.

Change List

The biggest changes occurred on the backend of the site and are not visible to users. However, while I was "under the hood" so to speak, I took the time to give the site a facelift.

First of all, all image and many article URI's have changed as a consequence of the redesign.

The MVC format and coding style mandated one major change in the organizational structure of the BMW section. Rather than the URI being "/bmw/<type>/maintenance", it's now "bmw/maintenance/<type>". I did this to simplify my backend programming, as "bmw" is the controller name, "maintenance" is the name of a method (function), and everything that follows it (including the vehicle type) is a parameter. The biggest benefit to this structure is cleaner URIs (anchors are mostly eliminated, particularly in the BMW maintenance logs), and the fact that users can now directly address and bookmark a single maintenance log entry by date. The perk for me is that my bandwidth should drop because you won't be forced to download the year's worth of log entries when all you want to see is the latest, and I'll be able to better track what log entries you find helpful. That will allow me to direct updates to the most needed sections.

Again, to simplify the backend development, I flattened the Aviation section structure. The "articles" portion of the URI was removed and the articles subsections brought to the aviation index page. Reviews are now located at "/aviation/reviews/" rather than the more verbose "/aviation/articles/reviews". A small change for the user, but a helpful one I think. Think "three click rule".

I'm now benchmarking each page. In the very bottom right corner of each page you'll find a rendering time. That's the time the server required to build the page and send it out the interface. This is a general indication of my coding efficiency but it can also indicate an overloaded server. Before you report that my site appears slow, check the rendering time. If it's less than a second (it will usually be far less than that), the problem is out of my control.

A lot of cruft had formed in various parts of the site over the past two years. That's been eliminated and replaced with proper XHTML and CSS constructs. I'm still serving pages as XHTML 1.0 Transitional with a type of "text/html". Yea, I know that's not correct, and yea, I know the W3C is now backpedaling on XHTML in favor of extensions to good old HTML 4.01. I would have made the necessary changes, but I had different priorities this time around. Perhaps next time. Or, perhaps by next time Micro$oft will get their act together and support proper rendering of "application/xml".

I deleted a couple of articles that were out of date and/or worthless at this point.


State of the Art - XHTML and CSS2

This iteration of the site was fueled by my desire to meet current open standards in order to promote accessibility and consistent presentation. This has become possible with XHTML and CSS combined with recent development in browser technology in support of these standards. This presented the most radial cosmetic redesign and resulted in the red, white and blue theme you see to this day.

Single Vs. Two/Three Column Layout

Let's face it. The typical two or three column web page template was designed to position ads and drive the content per page down so more pages get clicked through to increase ad rotation. Since I don't do traditional advertising to support this site, I made it a point to create a single column page design with navigation links at the top and bottom of the page. This allows me to include large thumbnail images with the content (you remember, the stuff on a website that people want to read), and keeps the optimal browser width down to a minimum -- thus saving you precious desktop space.

Original Designs (1995-2004)

Static HTML, then a misguided dabbling with frames

The initial site used a few static html pages in a single directory. As the site grew, I got tired of maintaining the navigation links on each page, so I transitioned the site to use frames. In retrospect, frames were a crutch. I wanted a menu on the site but didn't want to put in the time necessary to build a site whose pages were dynamically generated. And who could fault me. PHP didn't exist in any substantial form at the time, and I wasn't about to punish myself with the perl of the day.

Frames worked pretty well for a few years and allowed me to focus on content development. During this time, I became more familiar with various web standards and began to transition the site to HTML 4.01.

While I might say that I was motiviated to begin using Cascading Style Sheets at this time because I wanted to begin to separate markup from presentation, in reality I hated the standard look of hyperlinks and wanted an easy way to change their attributes. Psuedo-classes provided by CSS Version 2 provided that capability. At this time, however, I did not use CSS2 for positioning of content on the page (which is ironic, since that's the primary billing of CSS!) because browser support was flaky at best and the dreaded Netscape 4.x was still the dominant browser.

An experiment with Templates

While researching PHP I happened upon an article that described the use and benefits of template engines, and the FastTemplate class in particular. Many of the concepts appealed to the object-oriented designer in me, so I then decided to eliminate frame support and replace it with the FastTemplate class.

I divided the page into three simple logical blocks -- a header, content, and footer. Aside from a slightly different footer template for the front page, all pages used this format and made it easy to make global changes to the header and footer (meaning, the site menu, copyright info, etc.).

The bliss was relatively short lived, however. When my provider upgraded the version of PHP on my webhost, the FastTemplate class balked at a few of the changes and I had to fix them. That forced me to learn more about PHP than I wanted to at the time, but in retrospect this set the stage for future work. I continued using the class for another year or so, but since the class wasn't (IMHO) particularly well written and the author was no longer developing it to keep pace with the updates to PHP, I began a search for other solutions.

I quickly found the Smarty template system and in fact almost used it in the latest iteration, because it's pretty much the top-shelf in templating systems, but I ultimately decided to eliminate templates in favor of raw PHP. I determined that unless a site uses a backend database and has lots of associated logic, it's just simpler and faster to use raw PHP to divide the page into blocks and create a simple template for the site.

In other words, as long as the code is developed with care to separate logic from presentation and you don't need the data conversion utilities provided by the template system, PHP is a fine template system in its own right.