Influence and Affiliations
Beyond projects that I can call my own, I've had the privilege to be involved or affiliated with some great people and projects. Together we have had quite an impact on the world, changing individuals lives to influencing major technological and cultural phenomena. Here is a place I feel comfortable sharing some of them.
Theses are some highlights with more details below:
- Started DevHouse, "mother of hackathons" inspiring Facebook's internal hackathons
- Ran the website where many indie games were first shared, including Minecraft
- Coined and first evangelized webhooks, influencing use at GitHub and Slack
- Worked with friends at NASA Ames to build what would become OpenStack
- Built a tool called Localtunnel, which directly inspired Ngrok
- Co-developed Docker and helped very early adoption. Later added its plugin system.
- Built RequestBin, the canonical tool to debug webhooks and inspect HTTP requests
- Built Dokku as a talk demo, now one of the most popular open source PaaS projects
- A bunch of other stuff in the Docker ecosystem nobody gives me credit for :)
- Co-founded Hacker Dojo, the largest non-profit hackerspace in the country
- Executive producer for an award winning documentary called Indie Game: The Movie
Most of these trace back to 2005 at the dawn of "Web 2.0" and a legendary event and community I started with friends called SuperHappyDevHouse (which I refer to as DevHouse).
The first couple hackathons seemed to occur around 1999, but hackathons became widespread from 2005 to 2010. Interestingly enough, this is the period the original DevHouse events occured. Given the size and influential location and attendance of DevHouse, it's hard to imagine it was just a correlation.
Yahoo! Hack Day in 2006 was one of the first open, commercial hackathons after the Web 2.0 boom. I attended the first of these and in the invitation request form I mentioned my involvement in DevHouse. They replied citing DevHouse as an influence. From then on, many tech companies used hackathons to help market their platforms and APIs.
In 2013, Jon Gottfried, co-founder of Major League Hacking, described DevHouse as the "mother of modern hackathons".
Ironically, DevHouse was never considered a hackathon in the way most hackathons are run. It was a casual party-like atmosphere with coding and projects, but there were no competitions or extrinsic motivators for attendance. It also had no commercial agenda and was 100% community run.
Facebook started hackathons that were internal and helped re-inforce their values around The Hacker Way. Unlike most hackathons, the Facebook hackathons actually reflected the spirit of DevHouse around unfettered creation. This led to major features including the Like button, which has become a defining feature of the platform.
This tradition of legendary Facebook hackathons turns out to be directly inspired by DevHouse. Apparently Facebook employee Scott Marlette attended DevHouse and told Mark Zuckerberg about it over lunch. Mark Zuckerberg "thought it was a good idea and that we should have one in the office."
Around the time of starting DevHouse in 2005, I became friends with indie gamemaker Derek Yu. He was running a blog called TIGSource on indie games. He was having trouble with their hosting provider, so I offered to help out. I became the TIGSource sysadmin. At Derek's request, I set up forums for TIGSource.
Those forums turned TIGSource into a major breeding ground for indie games of all kinds. Some major indie titles that were first shared there included Fez, Super Meat Boy, Spelunky, World of Goo, and Braid.
There was one blowout success that stands alone: Minecraft. I only found out in that last couple years that it made its debut on the TIGSource forums. The TIGSource forums and IRC were also used to help come up with the name Minecraft.
BarCamp and Unconferences
Modern web pioneer Tantek Çelik was a regular at the original DevHouse events. In 2005 after the second DevHouse, he and a cohort of a few other DevHouse regulars dreamed up BarCamp, a free and attendee-driven technology unconference that inspired thousands of other BarCamps and unconferences around the world.
Although the core ideas behind unconferences had been around since the 80s, BloggerCon and FooCamp brought the term unconference into use in 2003. The explosion of BarCamp in 2005 helped bring it into widespread use.
During the closing remarks of the first BarCamp, Tantek mentioned DevHouse as a source of inspiration for the event. Particularly the grassroots "just do it" attitude of the organizers.
In 2005, Brad Neuberg coined coworking and started the first pilot coworking space in SF. DevHouse co-founder David Weekly introduced him to me at DevHouse and we bonded briefly over a shared appreciation of Will Wright and systems thinking. Brad, too, became a frequent attendee and later was one of several people to bring legendary Doug Engelbart to DevHouse.
With fellow DevHouse regular Chris Messina, Brad co-founded the first work-only coworking space in SF called Citizen Space. They helped turn it into a movement that has now become an entire industry. Roughly 700 coworking spaces exist in the US and one report shows coworking spaces doubling every year.
Hashtag and Twitter Stream
Many of the DevHouse regulars were early Twitter users. A number of the original Twitter developers also attended DevHouse. Chris Messina suggested we use hashtags to identify groups and tag our tweets. Twitter later adopted this as a native feature. As Twitter went mainstream, so did the hashtag.
The early Twitter engineering team was also present at DevHouse in 2007 for one of my first presentations on webhooks and they said it sounded like a good idea. I followed up with them, urging them to implement webhooks (they'd be one of the first), but they had scaling concerns and instead opted for a persistent firehose connection. This became the Twitter Streaming API that enabled huge amounts of innovation around Twitter and was even an early way Twitter tried to monetize their platform.
Homebrew Computer Club
I grew up in the SF bay area and as a techie kid, and I loved the history of Silicon Valley. In particular I was fascinated by an event in the 70s and 80s called Homebrew Computer Club, made famous as an event Jobs and Wozniak first attended to demo their Apple computer. But it was a breeding ground of innovation around personal computing. Among many others, one of their founding members, Lee Felsenstein, went on to design the first mass produced portable computer. Homebrew has been called "the crucible for an entire industry."
I felt that DevHouse was the closest thing to a spiritual successor to Homebrew. Myself and fellow DevHouse organizer Joël Franusic took this very seriously given our respect and fascination with the previous generation of hackers. I mentioned this to a Mercury News reporter that was covering DevHouse. The article got in front of Lee Felsenstein, causing him to show up at the next event. He became a DevHouse regular and participated in a number of community events that spun out of DevHouse.
In 2013, Lee, Joël, and Matt Spergel put together a Homebrew Computer Club Reunion with the Computer History Museum.
In 2006, due to the large number of projects that were coming out of DevHouse, even just my own, I decided to automate setting up project infrastructure: wiki, source control, issue tracking, etc. I used Trac and SVN, so I built a web app to set up Trac and SVN. I called it DevjaVu and attempted to build it as a startup. This was pre-GitHub, but they launched not long after.
A major feature of SVN were hooks. Pre-commit hooks let you verify code was allowed to be committed using your own code. Post-commit hooks let you trigger CI or notify IRC when commits went through, also based on your code. Hooks and callbacks in general I found to be very powerful for integrations and extensibility. But there was a challenge exposing these when SVN is provided as a hosted web app.
It occured to me there was no equivalent of the callback for web APIs. PayPal IPN was the one example I could find, and I was surprised at how simple it was. Even though it solved my specific problem, I was amazed at how little this pattern was used. At the same time I found myself stunned at the possibilities it could open up. I decided to call it webhooks and gave various talks and wrote a few blog posts about it.
Surprisingly, the idea was both hard to communicate to large audiences but quite easy to convince specific developers behind some great sites to implement them quite quickly. I continued to evangelize for a few years. Others joined in. I built lots of tools to support webhooks.
Now webhooks are easy to run into in the world of web APIs. Traction has only increased. Webhooks are responsible for some of the biggest sites' ability to integrate with other tools. Some major services like Twilio use webhooks at the heart of their product.
GitHub launched not long after I launched DevjaVu. I was in the south bay and their team of three was in SF. We never met early on, but we exchanged emails. I was happy to see they linked to my blog post about webhooks right on their front page. They got webhooks and now have one of the most developed and relied upon implementations of webhooks.
Although their success came for many other reasons, I think it's hard to deny the value in being able to integrate such a central tool with other tools in your workflow using webhooks. This made it possible to build other hosted services such as CI services that integrated with GitHub. Without webhooks, these integrations would have to be hardcoded by GitHub, severly limiting growth of the ecosystem.
I've never worked at Google, but I've found a number of libraries I've written and ideas end up being used at Google. Webhooks was the first of these. I was invited to give a Google Tech Talk on webhooks in 2009. It went rather poorly in terms of presentation, but some did see through the nervous, amatuer speaking and really grokked the idea.
Only a few Google products ended up getting any sort of webhooks, however it ended up influencing App Engine in an interesting way. Webhooks inspired the App Engine team to use HTTP as the mechanism to trigger background tasks and scheduled cron tasks.
Later, two Google engineers started a project in their spare time called PubSubHubbub. The idea was to use webhooks to turn RSS feeds into webhook based subscriptions. This found its way into a number of experimental Google projects. I was loosely involved in the PubSubHubbub project. The PubSubHubbub API was later adopted by GitHub, though I think because PubSubHubbub never gained critical traction over the more general pattern of webhooks, most implementors had dropped support for it.
I also found out Google engineers used several of my libraries implementing JSON Web Token in several of their ecommerce and transaction APIs. Mostly likely these evolved into Google Wallet.
The following are longer stories that probably deserve their own wiki pages with shorter summaries here.
A bunch of friends of mine from DevHouse ended up working at NASA Ames. They were self described space geeks excited to work with legendary NASA. Unfortunately, they found NASA was not the innovation powerhouse it once was. There was a saying, "they forgot how to get to the moon." Although there were still pockets of absolutely amazing scientists and engineers, they lived within a monstrous bureaucratic organization with all the trappings of some of the worst large organizations.
Despite this, this group continued to push forward and bring modern web 2.0 ideals to NASA projects. Partly in an attempt to "make NASA cool again" but also to unlock so much of the potential in participatory science.
One project was attached to a mission called TESS, which was a survey satelite to take high res photos of space to then process in search of earth-like planets. Besides the primary mission, what if we could allow anybody to submit code that would run on the satelite for their own processing of the image data? This seemed to exemplify the ideas of participatory citizen science. To pilot the concept, a web application would be made to let people propose ideas and push them through a pipeline towards all necessary requirements to run their program on TESS in space.
I was brought on to help prototype and build this application. It was done in a sort of skunkworks fashion outside of the normal web development processes at NASA. I used App Engine to quickly get it running and iterating. Progress was going well until the mission was deprioritized in favor of LCROSS.
The experience inspired this young group to build better infrastructure at NASA to support these kinds of projects. Even if TESS went forward, we'd have to go through normal painstaking provisioning steps to run our web application on government hardware. At the time it was not possible to use EC2 or App Engine. The thoery was that this was one reason more missions didn't attempt ideas like we had with TESS.
The CTO of NASA Ames, young and ambitious, helped our group form a new project called NASA Nebula. Part of the directive was to build cloud infrastructure for NASA and eventually other government agencies. The original vision was of a three layered solution: an IaaS layer emulating EC2 for VM provisioning, a PaaS layer for App Engine style apps, and a collection of high level Django modules to quickly assemble SaaS solutions.
In the end, this was quite ambitious. I was there particularly for the PaaS layer, but it turns out implementing EC2 is non-trivial. Although we originally utilized Eucalyptus, it was eventually dropped and rewritten. Around this time, the project was in slight danger of being killed under NASA bureaucracy and politics. Luckily, Rackspace was releasing their open source S3 equivalent, so a partnership was made called OpenStack that combined the Nova compute engine with the Rackspace object store.
By this time, I had left. Not long after, the NASA project collapsed. The remaining team at NASA either went to Rackspace or founded an OpenStack startup. OpenStack went on to be the canonical open source alternative to Amazon Web Services. Rackspace went all in and rebuilt much of their cloud infrastructure on OpenStack.
PaaS and Docker
I was instantly hooked on App Engine and the PaaS concept. I became a sort of power user. Not in the sense of building huge apps with millions of users, but pushing the edge of what was possible on those platforms. As such, I had many conversations with the teams behind App Engine, Heroku, and dotCloud.
App Engine solved a problem that was important to me. It abstracted away server management and configuration. While this seems like a no brainer for any application, I was interested in finding a way to run web apps as open source services. I believed the only way to do that was to automate and/or abstract server level operations away so a (non-commercial) community could more easily run and maintain a service.
With webhooks, I had also started building lots of prototypes and demos and it quickly became too much to manage on my own servers. App Engine relieved this problem. I also attempted to build a webtask or AWS Lambda style service for webhooks on App Engine. This and a number of other ideas really pushed what was possible on early PaaS systems. While RequestBin (then called PostBin) was a simple web app that could run on App Engine, projects like Mailhook and Localtunnel could not run on App Engine. This meant that I'd never be able to run them sustainably as an open source service.
Another great use of App Engine was for all the systems at Hacker Dojo. App Engine became our standard platform for apps. It meant we didn't have to run or manage servers because as a volunteer-powered non-profit, there were plenty of other more important issues to spend our time on. It also meant onboarding help was significantly easier, deploying was more accessible, and therefore more people could help out and evolve these systems. These included our sign-in system, our sign-up and payment system, our event manager, our website, and pretty much any other service we needed we could whip up on App Engine and all the users on our domain could access them.
App Engine's limitations led me to Heroku as it matured. It developed a more understandable, flexible, and open model referred to as the Process Model (they're just unix processes … in containers) and open source buildpacks for different language stacks. My go-to became Heroku, but I was still running into limitations. Luckily, since I new early employees and hung out there sometimes, I could influence it on occasion. I quickly got quite familiar with the Heroku architecture and had access to a number of hidden, unreleased features. But I still couldn't run a lot of apps on Heroku.
That's when I started turning to dotCloud. Unlike Heroku, dotCloud let you listen on arbitrary ports. I was not limited to HTTP any longer. Unfortunately, the dotCloud experience was overall just not as great. And I still ran into limitations, despite long conversations with their team.
So not only did I know PaaS well, it never satisfied me. So of course when my friends at NASA wanted to build a PaaS I said I'm down to help. Unfortunately, we never got that far. But I did research a number of open source PaaS and App Engine alternatives. There weren't many, and most of them you'll never hear of now.
My next job was at Twilio, which made a lot of sense (in theory) given their use of webhooks. My first year was working on a project that was very cool, but never had the product support to ever be officially launched. Most of the tech ended up in other products. The lack of product support I understood. Compared to all the other products, the margins for Twilio Realtime didn't quite make sense. It also wasn't telephony related at all, and so was a little ahead of its time in the product lineup. What I was more frustrated with was the delivery pipeline we had.
After a year I helped establish the Platform Team to manage our internal infrastructure and build an internal platform for the others. It was sort of devops with aspirations of PaaS. Only we never got organized enough to be able to build anything. Instead, we did a lot of design work and research in preparation.
Based on my experience with PaaS, I was convinced we didn't want another PaaS solution. It would just be another black box with limitations. So I wanted to disaggregate the PaaS into smaller building blocks. It seemed clear talking to lots of companies, not just Twilio, that PaaS is too one-size-fits-all that happens to not fit most companies. The ideal was: make it easy to build your own. In the process of disaggregating, I came to the first base component borrowing Heroku terminology: a dyno manager.
Less than 6 months later, I teamed up with dotCloud to work on Docker, applying this vision for this new tool. My initial use case was that I should be able to at least build a single-host PaaS very easily. I made a demo to show this was possible. In under 100 lines of Bash, I used Docker, the Heroku buildpacks, and a few other components I made to make Dokku. Dokku became popular quickly as it was more useful and understandable to developers than Docker itself. It was an easy sell: "your own hackable Heroku".
Eventually Docker outpaced Dokku in popularity, but Dokku had already inspired the modern generation of open source PaaS solutions built on Docker we have today. Many of them using components I made for Dokku. Some of them I even helped build or advised with.
My goal was/is to dissolve the need for PaaS and make the knowledge and components used to build them just be part of our toolkit for building systems. So far, we're making progress in that direction.
Indie Game: The Movie
Although I never found time to really work on games in my 20s, I did stay involved with the community. The few times I did make games were during game jam events I helped organize as part of the TIGSource community. They were called TIGJam. We borrowed heavily from the SuperHappyDevHouse recipe, which actually made them a little more open ended than most game jams today.
I love the indie game community. You have people pushing the envelope, experimenting with a new medium, creating very personal works, and as game developers they had to be talented across several disciplines. Art, programming, sound, music, production, business. Even within programming, games involve graphics, networking, tools, AI, sound, etc. But rarely did they take themselves seriously. The result was a community of extremely talented, extremely humble and supportive, and extremely fun people.
I felt like I had to share this with more people. Maybe to inspire people to make indie games, maybe to play indie games, or just be aware of indie games and their creators. These were basically brilliant starving artists, so I had a lot of intentions to find ways to support them before we had App Stores, Steam, Stripe, Humble Bundle, itch.io, etc. But I was also just super in love with the vibe of the community and I wanted to share it.
I decided I could make a short, low budget documentary on the community and culture showing the talent and personalities in it. I brought some filmmaking gear to the Game Developer Conference, the one central hub of game developers around the world, including indies. I started doing some interviews. I did one or two and then one of my indie friends told me these two from his hometown of Winnipeg, Canada were doing the same thing.
Initially I was not discouraged. Then I met them. And I saw their work. And I saw that they got it and knew how to capture it better than I did. So I dropped my project and decided to support them.
They self funded the project through several Kickstarters. I supported them in their first at the $100 level, which was maybe the upper middle end of levels. Then in their post production Kickstarter I noticed they had a top level for $5,000. The primary reward on top of all the others was an Executive Producer credit.
Not only did I want to support this project, but I knew how good it was. Not only good but important. And when I thought about it, this was a pretty cheap way to get an Executive Producer credit. So I did it.
By then they knew me and immediately emailed me trying to convince me I didn't need to give them that much. That that level was partly a joke, made for people with yachts and ascots. I insisted.
Because of this, they kept me in the loop. I offered to help them during their tour of the film and then they told me they were getting into Sundance. They invited me to Sundance with them. They ended up winning Best Editing for World Documentary. They later did a theater tour, and later it was released on Netflix, iTunes, Steam, and direct on their site.
It did quite well. It resonated with creative people. It made some people cry. It was a Good Movie and I'm glad I got to support it. As a result, not only is my name in the end credits, but I'm listed on IMDB as an executive producer for an award winning documentary. I'm also featured in the extras since they came to TIGJam and covered it and put it into a special feature.
Also, it's not obvious in the film, but all the developers in the movie already know each other, hanging out at GDC, or coming to TIGJam, or talking on the TIGSource forums. The indie community, especially then, was very small.
Because he is hilarious (though they all are), Tommy Refenes was the first person I interviewed for my attempt at a documentary before I knew about IGTM. I hung out with him and his family during Sundance since I didn't know anybody else there. Tommy and Edmond I'm pretty sure came to some of the first TIGJams, but also have just always been around. Randomly, Edmond's wife Danielle McMillan went to my middle school with me. Jon Blow was a popular figure at GDC even before his indie games. He came to several DevHouse events and occasionally we came to each others social gatherings.
Phil Fish has always been around the indie game scene. At one point quite often in the TIGSource forums. Fez programmer Renaud Bédard used to use the same 3D programming tool I used in high school. We might have even talked on the forums, we don't remember. Fez sound designer Brandon McCartin was a TIGSource editor and admin. Rich Vreeland, who did the music for Fez, was my roommate!
And that's just the games and developers featured in Indie Game: The Movie. There's a reason we all know each other – it's just a great community. I'm glad IGTM captured some of it, and I'm super excited for their next project, whatever it is!