A CTO's guide to Web 2.0
A couple of days ago I had breakfast with a former Chief Technology Officer of a REALLY big telco. He had attended the RSS Alley Geek Dinner the night before, and I could tell that even though he was one generation ahead of me, we had a similar take on software and computer technology. He was in Boston to have meetings with various people as a way of learning more about Web 2.0, so I volunteered to get together with him the next day to share my definition from a fellow CTO's perspective. I won't give his real name, because I didn't ask his permission, and this post isn't really about him. It is more about what any CTO needs to consider when trying to run a software development effort in the current Internet environment. For the purpose of this essay, I'll call him Jack.
The funny thing is that Jack's previous company had about 4,000 times more employees and sales than my company, yet we had exactly the same concerns about the new philosophy of development and business surrounding Web products. The insane thing is that Jack's company was valued at only 100 times that of my company when we got acquired, but that was the craziness of February, 2000.
I talked to Jack about four broad areas of change that any CTO needed to think about, but they all came down to one basic issue, a lack of control. It isn't that CTO's have to be control freaks, although they should be. It is a CTO's job to think ahead to what can go wrong, and try to make sure those blocks don't interfere with whatever technology tasks the company needs to accomplish. In a way, a CTO is like the lawyer for a company's technology, always looking for pitfalls well before they are reached. Web 2.0 forces a company to adopt the one thing any good CTO should loath, dependencies. You have to allow your company to be dependent on other people's code, their voices, their data, and their personal motivations that can't necessarily be overridden by money. Let me go through each of these dependencies:
- Open Source. While much of Web 1.0 was built using Linux, Apache, Sendmail, and languages such as Perl and PHP, the philosophy of Open Source didn't become pervasive until the turn of the century. There are now Open Source components throughout a typical Web 2.0 application. For example, collective voting has applications in many areas beyond the traditional uses in sites like Digg.com or Reddit.com, and is now available through the Pligg software, which is Open Source. Other common Open Source components are found in blogging tools and wikis. Companies also have to consider the desire of their programmers to release their work for the company as Open Source. While this has obvious implications for intellectual property, it also creates a labor force of more productive programmers, because they can bring portions of their code with them when they change jobs.
Jack was understandably concerned about quality control when using code that isn't delivered and supported by a commercial vendor, but the benefits of a larger and more open community of users can deliver a more robust solution than one used by a few hundred or even thousands of commercial customers. Building with Open Source code also means faster development cycles, so instead of working for years and trying to deliver a perfectly specified and tested system, a more incremental approach based on existing components allows you to work towards a solution in an evolutionary fashion. The reality is that a project that takes several years to reach "perfection" has so much invested in it that it may be impossible to stop and rebuild when problems are discovered, so they are just built over with ever increasing layers of patches. In the long run, a CTO using Open Source code does have to reject the traditional Not Invented Here syndrome, and accept a greater dependence on other people's code. The trade off in shorter development cycles is worth it in my opinion. - Blogs. Web 2.0 also brings about a shift in the way a company's technology efforts are communicated to the outside world. Instead of thinking in terms of versions that are announced at long intervals through a traditional PR campaign, the use of corporate blogs helps customers stay much closer to the development process. This also means a cluster of independent bloggers interested in an area of technology can form around the companies working in this space. These tech bloggers have replaced the traditional trade press. It means that a CTO is dependent on voices that are not as tightly controlled as in the past, but these bloggers can also act as an important buffer when problems arise by explaining to the wider circle of users that the company is indeed working on solutions.
- XML. The most common form of XML currently in use is RSS, but OPML is on the rise, and RDF based standards, such as Atom, are also gaining ground. In the long run, some form of global database resembling the Semantic Web will materialize. The key to all of this use of XML is the availability of a company's data outside the corporate database. While much is made of the emergence of APIs, it is the XML data that is available from these APIs that will cause the real changes in technological architectures. Just as Web 1.0 was built on loosely joined websites connected through HTTP and HTML, Web 2.0 will be built on loosely joined data structures based on data produced by many sources. So instead of a CTO building an application on a tightly controlled proprietary database schema, it will be necessary to plan for dependencies on data over which there is no control.
As a long-time database guy, Jack found that disturbing. I share his concern, but what must be understood is that users will demand this type of cross application sharing of data, because it is their data that is being combined from multiple sources. Sure there is a greater possibility of failure, and this must be handled by a CTO to allow for soft failures, instead of hard crashes. The one great fallacy that the XML proponents adhere to is the perfectability of XML data. Their motivation in building a Semantic Web is the goal of a Web that isn't filled with invalid data. I don't think that will ever happen, so a CTO should plan for badly formed XML, as is already the case in the RSS world. - Fear of excessive valuation. The traditional way to motivate developers, especially in a start-up situation, has been to offer them stock options. While that is still useful, the arithmetic has changed, because programmers who went through the Dotbomb have a deep fear of hype. A business journalist who was a former Dotcom employee recently told me that she still suffered from post traumatic stress disorder that prevented her from considering a start-up job. In the Web 1.0 period, there was an expectation of an IPO that would yield valuations in the hundreds of miliions of dollars. If a Web 2.0 company gets acquired for $10 - $20 million, that may be great for the founders, but it doesn't do much for a coder with a few thousand options. It is not just that the value of software companies have dropped. There is now deep suspicion of any claims of higher valuations in the future. Without the promise of getting rich, it is harder to persuade developers to put in the 18-20 hour days that helped build Web 1.0. This means that the CTO is more dependent on an employee's personal motivations, such as being able to build code that can earn them greater fame in the Open Source world.
