Posts tagged as: design

The value of simple metrics closely watched

(Warning: At the end of this post I'm going to say "It's a journey, not a destination." Whenever I say this at home, my kids run out of the room. When I'm with software developers there is a pause, and then conversation continues as if I hadn't said anything. Nevertheless, I continue to say it.)

Marc Hedlund has an interesting post about some useful Web 2.0 development techniques. I agree with many of them, but the practice of creating internal metrics based on website stats is not a Web 2.0 innovation. We followed the same policy at Andover.net in the late Nineties with a statistics system we built ourselves. Other than the usual web traffic values, such as page views and unique visitors, we had a few ratios that became the basis of almost every design and development decision we made. These metrics were displayed on a continuous graph, so we could see the trend over time. One of my favorites was pages per visitor (PPV). Whenever we redesigned a site, that was the first metric I checked. Since we made our money from advertising, the more pages people viewed the more ads we sold. We convinced advertisers (this was in the stone age, when websites sold ads directly to advertisers) that a higher page per visitor count meant more committed readers. Pages per visitor was also an indirect measurement of server performance, and ease of site navigation; a slow or confusing site had a low PPV. So the PPV was a way of measuring the user's approval of the website experience.

It might seem that a simple way to raise the PPV would be to chop up articles into multiple pages, but that would backfire. Users would suffer from fatigue, and leave earlier than if articles were displayed on a single page. We could prove this by testing the design and then watching the PPV. We did, however, discover the value of previous and next links on every page, which led to additional articles or whatever content the site contained. That simple change could sometimes triple a site's PPV. We called it the "peanut principle," substitute your snack of choice. Another obvious way to boost PPV was to add an archive of past content. None of this is rocket science, but it meant that we could acquire new sites that didn't follow these principles, and be reasonably sure of doubling or tripling the traffic overnight with a few design changes.

Another great example of using a simple metric as the indicator of progress was when we acquired Slashdot. We quickly discovered that the site was failing due to server overload. Unfortunately, there was no way of monitoring the code, so we couldn't tell where it was failing. We knew that the MySQL database was rolling over, so a new database architecture was a first priority. At the same time that I assigned people to the task of rebuilding the schema, I also had one person dedicated to building a monitoring system with a continual graph of database metrics that anyone on the team could reach on a webpage. The most important metric proved to be queries per page, which were extraordinarily high when we started. There was one memorable day when we hit the switch to introduce a new version of the Slashdot code, and this graph of queries per page dropped like a rock. We all cheered, and the coder who had designed the new database schema pasted that part of the graph on his door. This simple metric had not only allowed us to see our progress, it gave the people responsible for fixing it a tremendous amount of pride.

While all of this may seem like common sense, it is funny how coders often react cynically when I tell them about it, as if this were a marketing trick. I had lunch recently with a group of young programmers, and when they asked me how to scale a database to handle more website traffic, I told them this Slashdot story. I then explained that scaling was a journey, not a destination. They had to find a way to see their progress, not just look for an endpoint. There was a pause, and then they started talking again as if I hadn't said anything.

Book Note: Hackers & Painters

Paul Graham seems like a very smart person, I'm just not sure he is as smart as he thinks he is. His book is a collection of thoughtful essays on everything from why nerds are unpopular to why expensive watches aren't as reliable as cheap ones. If this sounds like too wide a range for a single book, you are right, but within it there are enough useful insights to make it well worth the time of anyone involved in making software. The two basic themes that run through the essays are Graham's experiences with the Viaweb startup, which was sold to Yahoo!, and the advantages of Lisp over other programming languages. I don't buy his arguments in favor of Lisp, but I found his comments on managing programmers to be right on target.

What I have a problem with, is his tendency to make sweeping generalizations about the history of technology, such as: Yes, and corporate DP departments weren't able to keep up with the demand for custom applications at the departmental level (dBASE), and word processing systems on PCs were more responsive than dedicated word processing systems on terminals (WordStar), and small businesses could run accounting software on PCs for much less than an account on a timesharing system (Peachtree), and Baby Boomers were just having their first children and wanted computers that they could use for work and to run games and educational software (Apple), and a host of other reasons. Graham may have gotten a Computer Science Ph.D. from Harvard, while I only got a Masters, but mine was in History of Science. After being called simplistic enough times for making statements like his, I learned that there is rarely a single cause for anything, especially in technology. Despite his fondness for historical generalities, I found the book to be a good read and recommend it highly.

Applying the Baldwin Effect

A friend of mine mentioned the Baldwin Effect in conversation yesterday, and it set me thinking about the value of demos during the early stages of software development. Briefly, the Baldwin Effect is an evolutionary theory that learned behavior can eventually become built into the genetic makeup of a species. One of my professors explained it as follows, suppose there was a cat that learned to look both ways when crossing a street. It would naturally live longer than cats that didn't, at least city cats. What if this cat also taught this behavior to its offspring? Eventually a fitness landscape would be created that favored cats that looked over those that didn't. If a mutation for looking both ways arose at this time, it would find an environment in which it would naturally flourish. In effect, behaviors can eventually become built-in. This has been investigated with lots of math that I can't follow.

My interest is its application to software, and specifically the relation between demos and design. When a new product is first created there are expectations about what it should do, but nobody knows exactly how it should be used. In time a common pattern of usage evolves, which is taught by experienced users to novices. This pattern becomes the assumed behavior which is then copied by competitors. New features accumulate along this usage path until the product is what the original demo showed it to be.

A classic example of this is VisiCalc. It was originally an electronic sheet of paper, which could be used for many things, including managing financial tables. The early demos taught to salespeople were based on the profit and loss example we all know so well. In time this demo became the assumed pattern of usage, and the product was known as a spreadsheet. All of VisiCalc's competitors focused on features that made a better spreadsheet, resulting in the spreadsheet metaphor we now all use.

The principle that "the software becomes the demo" is especially important in a start-up. Since software is so plastic, there is often a competition to incorporate everyone's pet feature. If a good demo is developed early, even if many of the features don't work yet, it can serve as a focal point for new features. If a feature makes the demo better include it, if not don't. This rule can still be applied to webapps, even if there are few chances to actually demo the software directly to a user. Even if the demo is only used in-house and with investors it can still serve as a beacon for developers, continually pointing them back to the planned course.

Tweaking the CSS

I've been playing with the CSS design for the site. I really love these dotted lines. I think I'm getting a little carried away with them. I'm also cutting back on the tables. I hope to have no tables left when I'm done.

I can see this getting out of hand

Is it a bad sign that I am sitting here at 6am making minute tweaks to the CSS template? It should be clear by now that I'm slightly obsessive, and I don't think CSS will help. I'm also a database person at heart, and the primary rule for databasers is "eliminate redundancy." I love the idea of changing one line in a template and having the whole site change.

Book Note: Eric Meyer on CSS

I read two more chapters this morning to help prepare for the CSS redesign, and I still have the same complaint. He is very weak on explaining why he is doing something. I think this book might work better if he started each chapter by showing what he wanted to accomplish and discussed the best path to achieve this goal. Instead he is forced to keep saying "I'll explain later why I'm doing this" and then he never does.

I was especially disappointed in the third chapter, which covered a web calendar, because I needed to redesign my archive pages. He actually hand coded the CSS for a specific month, with unique elements for each day. I can't believe he expects people to do that every month.

I'm not going to give up on this book, but I think I'll just skim it for good ideas, and then maybe come back again after I'm more proficient. Maybe I'll understand his decisions once I know more.

New CSS design

I spent the day creating a new CSS based design for the site. There are still some things I'd like to improve, but this is a lot better. I like the level of control CSS gives me. I can adjust elements 1 pixel at a time. There are some odd behaviors however, and I can see that there is a lot more art than science in using CSS for design. I still don't know when it is better to control the TR or TD tag, and I see that I'll still have to rely on tables for the main structure until I get much better at this. I've been testing some CSS editors, and I'll post a round-up after I'm more comfrotable with them. I don't want to blame a program for what may be my lack of knowledge.

Must spend more time on CSS

I'm happy with the basic functionality on this and the Ruby blog, but they are both ugly as dirt. I need to put in a few days with the Eric Meyer CSS book.

Book Note: Eric Meyer on CSS

I just finished the first chapter and I have mixed feelings. The first thing to realize about this book is that it makes no attempt to teach CSS. It assumes that the reader knows the syntax and wants to learn how to apply it. The really challenging part of Meyer's approach is that he is starting with a completed page in HTML, stripping out almost all of the formatting, and then adding back CSS to recreate the original design. This means that he keeps switching between the old version and the new one. This is very challenging and he does it well. He moves in very small steps, and all of the sample files are on the book's website, so it is easy to follow along. My biggest complaint is that he almost never says WHY he is doing something a certain way. Adding a font or border is obvious, but when he has a choice between using a distinct table or <div> tag for example, he chooses one or another without discussing his decision process. Overall, I'm impressed by what can be done with CSS and look forward to rewriting this blog's design. The next chapter is a calendar, so I may try to rewrite my archive pages when I finish it.

Book Note: Eric Meyer on CSS

Back when CSS first appeared in the late Nineties I was too busy running the Andover.net websites to adopt a new model for web design. My philosophy was to get the cleanest site design possible while maintaining compatibility with the greatest number of browsers. We were a mass market collection of sites, so I refused to adopt any technology that excluded lots of trailing-edge users. This included Java applets, which crashed Linux versions of Netscape, Flash animations, and any form of plug-in that required a download in advance. We managed to create some great designs, but they were extremely table heavy. Now I have the time to explore CSS and it has had the time to mature. I guess the lesson is that Web technologies really need about five to six years to fully mature and for browsers to catch up wth support.

The current design for this site is again based on lots of tables and is deliberately simple, since I knew I would be rewriting it in CSS before too long. The one technique I have insisted on since I started building sites in 1996 is a template/page generation model. This means that all of the layout HTML is maintained in a small set of templates which are then combined with the data for each page by a program written in something like FoxPro or Perl. At Andover we had over a dozen sites with tens of thousands of pages overall, yet we were able to update the design of the entire network by changing a few lines in less than a dozen templates. Another design rule I insist on is separating the design from the code. I hate the idea of having a web designer mess around in a file with program code. This has always biased me against PHP and Javascript. I know that I'll run into this issue again when I start to learn AJAX.

I have kept up with CSS enough to know the basic principles, so I'm looking for books that will emphasize design and coding techniques rather than a simple how-to introduction. Eric Meyer seems to be the guy everyone turns to for this type of information. This book is based on a series of projects, and starts with the real-world issue of converting an HTML-only page into CSS compliance. So it seems perfect for my needs.

I have just started to read it, but I already like the techniques Meyer is using. For example, he starts with a simple piece of CSS that can be added to any page to display the underlying table structure:

<head>
<title>Darwinian Web</title>
<style type="text/CSS">
table {border: 2px solid red; margin: 3px;}
td {border: 1px dotted purple; padding: 2px;}
</style>
</head>

Here is a current archive page from this site with the CSS added.