Darwinian Web

You can find me at Feedonomics.Grazr.Com

Adam Green — Fri, 23 Jun 2006 14:50:00 EST

The initial funding and paperwork for Grazr Corp. is now complete, and we are in the middle of hiring some of the best coders from my last company. Mike Kowalchik will continue to run the development process as Grazr's Chief Scientist and I will be the CEO. My principal focus at first will be documenting the whole area of feed management and Grazr's role in that space, which is why I have started a new blog called Feedonomics. I hope you can join me there.

Blogging memories in Tabblo

Adam Green — Mon, 15 May 2006 05:30:00 EST

My good friend Antonio Rodriguez has just launched the beta test of Tabblo, his new photo and text publishing site. So I thought this would be a great opportunity to gather up some of the more memorable images from my many months of blogging at Darwinian Web into a single commemorative poster. Thanks to Tabblo, you can even order a printed copy of this poster for your home.

Running away to play with Grazr

Adam Green — Tue, 09 May 2006 06:20:00 EST

When I started this blog, way back in last October, I wrote that one of my goals was to come up with an idea around which I could start a new company. At the time I assumed that this would take at least a year. The search ended up taking half that time. I was lucky enough to meet Mike Kowalchik in late January, and that friendship has helped produce Grazr. I originally stated that I didn't want to take an equity position in Grazr, because that would conflict with being an independent blogger. A few weeks ago I started helping Mike raise money for Grazr development from angel investors. I soon found that the story I was telling people was so compelling that I ended up selling myself on the idea of investing. Mike and I have now come to a verbal agreement to have me invest and to join the company full time as president. Since I've always stated here that I wouldn't invest in companies that I blog about, this means that I will be changing the way I blog on this and my other sites. I'll probably stop posting anything here other than updates on OPML Camp in two weeks. When Mike and I have sorted out the goals we want to reach, I'll start blogging here about my work at Grazr. I'll use other company blogs as models to decide how I will discuss products that compete with Grazr.

I didn't even notice others had stopped blogging too

Adam Green — Mon, 24 Apr 2006 18:28:00 EST

A week ago I decided to just take a break from blogging. I was basically playing hooky. I finally started reading blogs again and found that lots of people were quiting all around me. I'm not going to stop blogging, but I may change my focus. With all the free time I had, I was able to come to some interesting decisions. Until these play out I'll just stay quiet here. I am still working on the OPML Camp for May 20th. That is doing great. We have 40 people registered, and plenty of speakers lined up. Most of all we have many of the OPML tool authors, so we'll get to put applications people and developers in the same room for an unconference about OPML and structured content.

I'll be flying to Dublin on Wednesday to speak on a panel on Web 2.0 technologies. I'll try to blog about that on Friday.

Feeling the urge to code

Adam Green — Mon, 17 Apr 2006 16:20:00 EST

I haven't been blogging much lately, because I've been spending my time reading Python books. That's one thing that separates Python from Ruby that people don't seem to mention when comparing the languages. There are tons of Python books, while there is basically just one good Ruby book. I'm going to spend a day or two more of just reading, and then I'll start doing some basic RSS coding to test out the Python libraries. Once I feel comfortable with the language, I'll get back to exploring the various forms of microcontent and their potential relationship to OPML. Don't expect much blogging here for a week or so, but I will report on my initial RSS library experiments on my code blog.

What problems are RSS libraries trying to solve?

Adam Green — Sat, 15 Apr 2006 20:18:00 EST

I spent the day reading Python books, and now I'm starting to look at the many libraries that are available for reading and writing RSS in Python. There are certainly more libraries available than in Ruby, and that makes Python seem like a place I should spend some time, but all of the libraries seem to have the same perspective, which is to treat RSS as a specific variety of XML. What if I don't care about DOM and SAX and stepping through a tree of XML? I keep thinking about what happened when dBASE first appeared in 1981. At that time the standard way to do database programming was to use CBasic or Microsoft Basic to construct a record in memory using string variables. It was up to the programmer to do what was called record blocking, where the strings were padded out to fixed lengths and then written out to a text file. If you wanted an index as well, you had to find a separate library and manage insertion and management of nodes in a B-tree. Then dBASE came along, and all you had to do was say APPEND or EDIT in a program. dBASE wasn't about database details, it was about application building. If you wanted an index, you just said "INDEX ON <field>," and the index was created and automatically maintained. Instead of being a database programmer, you could be an inventory programmer, or a doctor's office programmer. In fact, doctors could become programmers and create their own applications. That is what I want with RSS and OPML. I don't care whether a feed is RSS or RDF, or what the specific XML tags are. I want to be able to say "Give me the posts from this list of feeds that are less than 1 week old. Now combine them all into a new feed." That should be 3 or 4 lines of code. There has to be a reason that RSS programming is still about creating generic aggregators instead of RSS applications. It is a combination of tools and coders who are thinking about RSS as streams of XML data, and not application specific data that should be represented as vertical applications that have nothing to do with reading blogs.

Moving on to Python

Adam Green — Fri, 14 Apr 2006 10:20:00 EST

I hesitate to say that I've decided to move to Python from PHP. I got plenty of emails from people telling me that PHP would surely solve my XML programming problems, but it doesn't seem to be able to do it for me. There are lots of great tools written in PHP, so it must be a personal issue. I'll know when I've found the right language, because I'll be able to do XML programming without repeatedly getting random errors. I plan on working my way through the "Hacking RSS and Atom" by Les Orchard, and "Beginning RSS and Atom Programming" by Danny Ayers and Andrew Watt. Both books use Python for their examples, so that should give me the background I need to see how the pros do it. After Python comes XSLT, which I also get lots of emails about. The funny thing is that I can get everything I need done with each language I try, except for XML parsing. Reading and writing files from the Web, parsing for hyperlinks, etc. are no problem. This blog is generated with my own FoxPro code and it seems to be working fine. I may be rusty when it comes to serious coding, but I still think there is something missing in the area of XML that isn't my fault.

Exploring the tacit knowledge between RSS and the Semantic Web

Adam Green — Fri, 14 Apr 2006 08:48:00 EST

I started reading about the Semantic Web again last week, and my immediate reaction was the same as the first time I tried a few months ago. This is such a perfectly specified, intellectually rigorous collection of standards and practices that it seems almost impossible to find an entry point. If it is so hard to get started, then how does anyone work with it? The answer is that the people who understand it now are the same people who helped to build it. Each of the many sub-standards and protocols were introduced in reaction to a specific problem discovered during the creation of some other portion of this edifice. An analogy is trying to understand how an immense cathedral could possibly have been built by walking around the finished building. Once the scaffolding and the masses of workers are long gone, it seems like every part fits seamlessly into every other, and the thousands of decisions that were made during its construction are erased.

I decided to pull back a step and look at the areas of namespaces in RSS and the many competing standards for structured microcontent on the Web. This is much messier and clearly a work in progress, but once again as with the Semantic Web, the same individuals keep popping up in these many projects. The problem with the social nature of the construction of microformats, structured blogging, RSS, Atom, etc., is the unspoken, or at least underdocumented, aspects of the decision process. Why are there two competing sets of blog microcontent formats? Why are there apparently dozens of overlapping collections of RSS namespaces? The answers are lost in the maze of blog posts and standards announcements made over the last few years. Why isn't everyone involved with this area terminally confused? Because they lived through the process and understand the political, social, commercial aspects of each of these multiple body collisions.

What we now have is a continuum from the ultra-simplistic, under specified formats of RSS and OPML to the ultra-rigid, crystalline perfection of the Semantic Web. In between is a rabbit warren of partially completed, interconnected attempts to add more structure and functionality to RSS and HTML.

So what is the solution? I'm not conceited enough to believe that I can unravel the current mess lying between RSS and the Semantic Web, and I'm also not smart enough to try to storm the castle of the Semantic Web by brute intellectual force. What my past history has shown me is that I am capable of helping people build tools and writing documentation that can help bridge this gap. The process I'm going to follow is to start studying and coding with the RSS namespaces and microcontent formats until they gradually make sense, and then try to get tools built by others that will provide a more accessible conceptual model. In other words, I'm going to live there until I grok the neighborhood.

I went through the same process when I moved to Boston. The classic line when trying to explain how to navigate witihin Boston is "I can't tell you how to get there, but I can take you once and show you." This is a perfect example of tacit knowledge. It is something you and your community knows, but which can't be explained in words. It may be an urban legend, but there are many stories of truck drivers paying taxis to lead them through Boston's streets to a specific location. The only way to deal with Boston's streets is to carry a map for the first few weeks until your brain somehow builds the tacit knowledge you need to feel comfortable.

Grazr update

Adam Green — Wed, 12 Apr 2006 19:30:00 EST

Grazr has now been in an Alpha test for a little under a month, and the response has been overwhelming. I believe that there is an intrinsic demand for any piece of software, and you never know how big that will be until the product is offered to the public. In the case of Grazr, the possible uses have completely outstripped the original conception of a simple OPML viewer. One big area that wasn't anticipated is the use of MP3 files within OPML. Dan MacTough supported this first in his Optimal Browser, and Mike Kowalchik quickly followed suit and added an MP3 player to Grazr. James Corbett came up with the term Podgrazing to describe this type of application, and within a week or so one of James' countrymen launched Podgrazr.eu to showcase Irish podcasters. Another Irishman, Robin Blandford, also launched CommentCasting using Grazr as the user interface.

Another great idea from James Corbett has been the reinvention of the old webring idea in the form of OPML rings. James calls this grazer blades, but thankfully everyone is ignoring that particular pun. You can see this demonstrated in the Grazr box on the left side of any page of James' blog. Click on the link for "My Grazer Blades," then click on the link for "Pieter Overbeeke." At the bottom of this list of links you'll find a link for "James Corbett" that brings you right back to where you started. James is also making a good start on a world wide outline through the rapid growth of his Open Irish Directory, which is in the same Grazr box on his navbar. This makes use of the include capability of OPML, which allows the growth of distributed outlines with many sources and independent editors.

Ireland is a real center of Grazr and OPML activity, thanks to James, but the overall global appeal of Grazr has been a real shock. The Grazr below shows the results of a Google blogsearch for "grazr," and the number of links in languages other than English is impressive. Mike had to spend some time in Unicode Hell to get all the languages working well, but that has really paid off in international adoption.

Another area of interest that wasn't anticipated, but should have been, is the use of Grazr on mobile devices. Russell Beattie wants this so badly that he even created a mockup of it on his blog to show everyone what it would look like.

Grazr will be entering Beta test in a few weeks, and the new features that will be added then should really open some eyes. Right now Mike is working on making the backend as stable and scalable as possible, but once that is under control, the real fun can begin. People assume that the current interface is the only one available, but that is just the starting point. Grazr is built on a Javascript library that makes all types of interfaces possible, such as an expand/collapse outline, and an automatically scrolling "river of feeds." Different types of transitions, similar to what are used in video editing are also in the works.

Once again, I'll make my disclosure that I have no financial connection with Grazr, and hold no equity position. I just love the idea of seeing good software get built and helping to grow a community around OPML.

There has to be a better way to do XML programming

Adam Green — Tue, 11 Apr 2006 08:26:00 EST

Two weeks ago I decided to do some programming for analysis of link patterns among bloggers. I had just given up on Ruby out of frustration over its poor XML lbraries, so I decided to try out PHP. I haven't used PHP since the late Nineties, but it is a simple enough language, so browsing a few books showed me what I needed to know. I was able to write the code to parse out the links from Tech Memorandum and then autodiscover the RSS feeds on these pages without much trouble at all. I'm not a great coder, but I can pick things up fast, and can generally force my way through most programming issues. Then I ran into the XML libraries in PHP and came to a dead halt again. I need to read through the RSS feeds of each blog I find on Tech Memeorandum to find the links to other blogs, and that means parsing the XML of these feeds. I've been beating on this problem on and off for the past week and a half, and am about to give up again. Giving up on a programming problem is not something I do lightly. The whole point of being a programmer is never letting the machine beat you. I also have enough confidence to think that if I'm having so many problems lots of other people are dealing with the same thing.

What I've decided to do in response is work with one of my favorite programmers from my Andover.net days to try and build a better language solution for XML processing, with an emphasis on RSS and OPML. A few weeks ago John Casey emailed me after I posted my frustration with Ruby, and asked why I don't just write my own language for this type of work. We've been talking about this ever since, and now I'm ready to go ahead. I'm not capable of writing my own XML parser, at least not one that isn't a horrible hack, but I do know a lot about language design, especially about making programming languages easy to use. John, however, is a great coder, and if he thinks he can write a clean, fast parser, I believe him.

The idea at first will be to create a library of functions that are real smart about RSS and OPML. We're not sure what language this will be working with, but since the library will be written in C, it should be possible to add it to all of the standard Web languages, like Perl, Python, PHP, etc. I'm interested in having the library handle all the standard tasks you would need when working with RSS and OPML, so it should be possible to read multiple feeds and combine them in interesting ways in just a few lines of code. Once this library is built, we can see about possibly extending it into more of a mini-language.

The working title for this library/language is OPML Script, but that name may change as its functionality expands to more general XML tasks. This will be released under an Open Source license of some type, so it will be available for no charge. John and I will share the ownership of the copyright, although there doesn't seem to be any likelihood of ever making money from it. I've said in the past that I didn't want to get directly involved with any startups for at least a year, but this is something that I need for my own work, so I don't have any choice. If I want something that will let me program in an easy manner, I'm going to have to help build it. We don't have any delivery schedule yet, but we hope to have something we can demonstrate by OPML Camp on May 20th.

Web Innovators Meeting on May 8th

Adam Green — Mon, 10 Apr 2006 19:24:00 EST

The next Boston Web Innovators Group meeting will be on May 8th, at the Hotel @ MIT. The last one went very well. There was a good crowd and some interesting demos. I typically get a group of people together for dinner afterwards, so be sure find me if you want to join us.

Hiring and retaining high quality programmers

Adam Green — Mon, 10 Apr 2006 09:35:00 EST

I recently had to chastise one of the local startup guys here in Boston for mishandling the hiring of an old programmer of mine for some contract work. The exact details aren't important, but it reminded me how much difficulty most people have trying to hire and retain good coders. During the Nineties I managed to keep almost every programmer I hired in one of the craziest job markets for programmers we'll ever see. Out of a final team of about 20 people, I never had to fire any of them, and only had one quit over 6 years, and that was because his wife became ill. It wasn't that I paid the highest salaries. I paid well, but we were about 20% below the area's top salaries. It looks like we're moving into another programmer shortage as so many new startups get funding and try to fill out their staffs, so I thought it would be helpful to review some of the lessons I learned in Web 1.0. Some of the programming techniques and business models may have changed, but programmer psychology is still the same. These rules apply equally well to hiring consultants and full time staff, although I've usually had better results with full time employees. The pressures on consultants to juggle the maximum number of projects doesn't always allow them to focus enough on each job.

Professional Student. The first mistake employers make is insisting on a programmer who is an expert on exactly the type of application they are being hired to write. While experience is obviously important, the best programmers are professional students. They move freely between different programming languages and application types. In many cases the hot new technology is really much easier to pick up than you think, so it isn't necessary to hire an Ajax "expert" even if you know Ajax will be used in the job. More importantly, there will always be a hot new language or technique, so I'd rather hire someone who can teach themselves new tools as they appear. Because of this I make sure during the interview to stress the need to keep learning on the job. I want someone who gets excited when they hear this. I'm also biased towards self-taught programmers. A computer science degree is useful, but it's what they have learned on their own since graduating that really matters. Speaking of graduating, I haven't found any difference between someone who drops out of college and someone who finishes. In fact, it's usually the ones who quit school because they were impatient to build things that have worked out best for me.
Front or back end. One thing that is important to discover is whether a programmer prefers working on the user interface or internal engine of an app. This distinction seems to be fairly strong, and it's hard for most people to work equally well in both areas. User interface programming doesn't necessarily mean graphic design, which is another set of skills, but it does require focusing on lots of annoying details, none of which are particularly hard in isolation, but when combined they can be a nightmare. Engine programming, on the other hand, requires an ability to conceptualize a complex algorithm and break it down into a simple and elegant solution. Instead of asking this question directly, you can ask a potential hire to describe some of the most rewarding projects he or she has completed. These will generally be gathered in either the front or back end of an application.
Let programmers pick their colleages. It's essential to have the other programmers meet all potential hires, both individually and in groups, and to have a say in whether they are hired. I want everyone to be able to work together, and to count on each other for different skills and areas of expertise. It takes a while to build this type of ego-less team structure, but once it's in place, they will look for people who can complement the team's skills. I learned the hard way to adopt a firm "no assholes" policy. No matter how badly I wanted someone who seemed to know exactly how to solve the problem they were being hired for, I refused to hire anyone if I and the others felt we couldn't be comfortable hanging out with them.
Programmers don't work by the hour. The biggest mistake you can make in hiring a programmer is to think in terms of the amount of hours they will be working on a specific task. There is no way to know when someone is actually coding, even if they are sitting in front of you. The other variable is the wide range of productivity between coders. In my experience, it can be as much as 100 to 1, and 10 to 1 differences are common. The only way to measure the amount of work a programmer is doing is to set clear goals and then see how long it takes to accomplish them. Within that time period, I couldn't care less how many hours were actually spent coding. I also don't believe in holding programmers to 9 to 5 schedules. As long as the work gets done quickly and with a high level of quality, they can be as flexible with their hours as they choose. All of this is dependent on you or the person who manages the programmers actually knowing what they want to get done, and being able to tell when it is completed successfully. It's the managers who don't have a clue about what software actually is that depend on hours and timeclocks.
Let them own it. Once a programmer completes a task, they should own it, and the rest of the team should know this. I don't mean legally, but within the structure of the development staff. Pride of authorship is the best way to get high quality work and to keep code free of bugs. When something breaks, everyone should be able to turn to the person who wrote it. Over time people will develop their own areas of expertise within the team, and others will know they can trust that person to handle a specific type of task when it comes up. This allows them to self-organize when a new project comes along that requires multiple skill sets. Eventually the number of modules or applications that a coder "owns" will grow, and you have to make sure to leave them time to take on new tasks as well. Nobody wants to be stuck doing nothing but maintenance, at least nobody you would want working for you. If there are too many maintenance jobs, then it is time to hire another programmer to take over some of this code. This new programmer should be allowed to rewrite the code he takes over if he wants. It seems like a waste of time, but you generally get new features that pay for the extra work. The important thing is to make sure the new owner also feels pride of authorship.
Keep them learning new things. It's important to allow programmers time to experiment with new languages and tools. Sometimes the best new ideas come up when "playing" with a piece of code that has nothing to do with the normal team's work. You should be willing to pay for programming classes in the evening, and budget a certain amount for trips to developers conferences. One trick I adopted was to always have a bookcase full of the latest O'Reilly books. Programmers were encouraged to take these home and keep them. If they could get one good idea from a $30 book, it was money well spent.
Keep the lines of command clear. Programmers have to know who is allowed to assign them tasks. The worst thing is when the VP of Sales and the CEO think they can wander up to any coder and start asking them for features. There should definitely be a way for the entire company to make suggestions for future development, but getting switched from one task to another by every executive who walks into a coder's office is terribly confusing. What coders need more than anything else is big blocks of time to work on a single task. They can have multiple projects going at the same time, but they need to manage their own time. If every angry call from a customer to sales results in a new rush project, their long-term productivity drops dramatically. There must be a buffer between sales and development to make sure these "emergencies" get prioritized and consolidated into the total work flow.
Never cancel projects. The most important rule of managing programmers is to never, ever cancel a project. There is nothing that breaks a coder's heart like having several months or even years of work thrown away. This means that before you assign a task, you must be sure you will actually see it through to completion. That doesn't mean that requirements won't change and code won't get thrown away. Throwing away a first attempt at an application is fine, as long as the coder makes that decision, and it is a way of starting fresh once the parameters of the job are better understood. That is different from deciding that you didn't actually need that application after all, and all the work on it was a total waste of time.

Managers who think of coders as if they were hourly factory workers who put in their time and don't care what happens to the results of their work will end up with exactly that. If you give a programmer the chance to learn the skills needed, the time to build the code right, and then aknowledge their ownership, you'll get the best productivity and people who will work all night and over the weekend on their own if that is what is needed to get the job done.

Web 2.0 conference in Dublin

Adam Green — Sun, 09 Apr 2006 19:12:00 EST

I've been invited to speak at the Web 2.0 in Ireland conference in Dublin on April 27th. This would be my first actual speaking engagement in the computer industry since some time in 1992. During Web 1.0 I didn't go to any conferences. Who needed to actually be in a room with other people when we had the Internet? It should be fun. I've never been to Dublin for some reason, so I'm looking forward to that as well. My original plan was to go to Limerick at the end of April to visit James Corbett, but he was nice enough to arrange this invite through the event organizer, Fergus Burns. Isn't that a grand name? Too bad I can't roll my R's. Anyway, James will be flying into Dublin instead, so I get to see what the Irish tech revolution is all about and hang out with James as well. After the conference I'll be hopping over to Amsterdam for a little vacation. Now that is a city I am familiar with. James wanted to know what was in Amsterdam. Silly man. What isn't in Amsterdam?

I'm feeling better, but now I'm in Cleveland

Adam Green — Fri, 07 Apr 2006 17:53:00 EST

No, that isn't the punchline for a midwestern joke. I got over my bronchitis by yesterday, and then I had to bring my son out to Ohio to visit Oberlin College. I'm just going to turn this break into a little vacation from blogging, and I get to spend it in a Residence Inn outside of Cleveland. I'll be home tomorrow night and back to work first thing Sunday morning. Thanks for the get well messages. I was never really that sick, but I discovered that when I have a fever my desire to blog disappears. Anyway, I'm ready to get start again real soon.

Down with bronchitis

Adam Green — Mon, 03 Apr 2006 08:53:00 EST

I caught bronchitis from my son this weekend. It's a springtime tradition in our household. I'm going to stay in bed for a day or two, so I'll probably skip blogging.