Apr 25 2016

Generative Technology

I've felt for a long time that certain technologies were more appealing to me than others. I'm not talking about which industries or fields the technology is applied, but something deeper that impacts all technology. It's the reason why the computer felt so special to many of us. Perhaps why a lot of us got into software. For years I've been searching for a way to describe it and now I can: generativity.

In 2008, Jonathan Zittrain wrote The Future of the Internet and How To Stop It. The book presents the idea of two types of technologies: sterile and generative. It ultimately focuses on policymaking concerns around generativity, in particular the security implications. Cory Doctorow has a good summary review if you'd like to learn more about the actual topic of the book. However, here I'd like to expand on generativity from a broader, technologist point of view.

Zittrain describes sterile technologies as those with a fixed purpose or use. They're made for one thing and don't often surprise you. Another way to think of them are as appliances. Home appliances exemplify sterile technology. The toaster is pretty much only good as a toaster. The smoke alarm is only good as a smoke alarm.

Generative technologies are those with more potential uses than the creator could even imagine. They're general, repurposable, and often surprising in how they're used. They encourage innovation and tend to result in new technology on top of them. One of the most generative technologies we have is computing. The Internet and the web are themselves also generative technologies. At the content layer on top of the web, Wikipedia is a generative technology.

Toaster and Commodore 64

Most technology falls somewhere between sterile and generative. Or perhaps, are a little of both. Zittrain describes certain devices as "tethered." That is, generative, but not purely generative. The iPhone and the App Store are common examples of tethering. They allow anybody to create new uses for the device by making apps, but Apple is the gatekeeper. Truly generative technologies and platforms don't have a single gatekeeper for their ecosystem. Again, our prime examples: the Internet, the web, Wikipedia.

Zittrain tries to remain neutral on whether sterile or generative is better. He argues that your car should likely be a sterile technology. Generative systems can grow messy, and malicious third parties can often abuse them. Truly generative systems can sometimes be compared to the Wild West. Perhaps in fear of that chaos and of what we don't understand, we let companies sterilize and exert control over these technologies.

The book ends up focusing on the debate around this trade-off. Yet it's the idea of generativity that captures my interest most. I'm not against sterile technologies, but I do have a bias towards generativity. I would love to see more generative technology than sterile, but we live in a world that discourages it.

Blame it on consumerism

Business loves sterile technology. It's more predictable and easier to control. Sterile is also easier to market. A product needs to solve a specific problem for people to see a need for it. This is why products tend to have a fixed purpose in marketing if not by design.

A product with several uses is often harder to market until the uses are well-known. Think of baking soda and personal computers.

Twilio is another example of a generative product. We first marketed to developers because they could see all the ways to use it. Later, marketing to large enterprises required a shift in strategy. We had to focus on narrow, problem-specific solutions on top of it. This is often a necessary strategy for API and platform companies.

Besides marketability, generativity often comes with other business challenges. Having untethered uses is a support nightmare. Generative technology can also be confusing and harder to adopt. And again, malicious third parties can hurt users and product reputation. These all have costs that are easier to manage by limiting generativity.

Another way to say it, sterile technology is more consumer-friendly. By keeping it simple and focused, it's easier to use, easier to market, and easier to run a business around it. These all sound great, but from a technology standpoint the trade-off is the possibility space it allows us to explore. A toaster lets us make toast. A computer lets us do almost anything.

Unprecedented generativity

Generativity explains why the development of the personal computer was such a big deal. Computing represents the most generative technology in human history. You can imagine the marketing challenge in front of early PC vendors: a device you could use for almost anything. Most of them focused on known use cases around business, learned from the previous generation of computers.

Steve Jobs had a different idea. He knew how special the computer was, often citing the magic of one mind and one computer. His mission was to make computing personal and on a massive scale. Beyond hobbyists and beyond the workplace.

Steve Jobs and the Macintosh, Norman Seeff

The way he felt he could do this was to make the computer as much into an appliance as possible. He knew the real generativity was in software, so make the hardware sterile. Plug it in, turn it on, then point and click to enlightenment.

But Jobs was a control freak. He tried wherever possible to make the software sterile, or at least tethered, as well. Can you imagine how excited he was to move Apple into consumer electronics? Actual appliances! If you recall, he even tried to get away with making the iPhone a closed, sterile appliance with no third-party apps.

This sterile appliance strategy started with the Apple II. But since the Apple II was Wozniak's baby, it was more generative than sterile. One argument in particular about the design of the Apple II captured their contrasting values.

Woz wanted 8 slots for expansion cards in the Apple II. With more slots, third parties could add more functionality to the device. This would increase its potential uses, making it more generative. Jobs only wanted two slots for two specific add-ons: a printer and a modem. Fixed, predictable, sterile.

The story of Steve Jobs is all the more remarkable considering his dedication to the sterile appliance strategy.

The Apple II was a success, but it would seem as though it was because of the generativity that Woz kept intact. Yet every computer product Jobs orchestrated from then on applied the appliance strategy in full force. It seemed to only result in financial failure after failure. From the Lisa, to the Macintosh, to the Next Computer. You can imagine somebody in his position might have reconsidered this strategy. He persisted.

Finally, with his return to Apple, the timing was right for the iMac. The technology was cheap enough and the PC market was large enough. The strategy finally had a chance. It worked and it would seem Jobs not once doubted the appliance strategy. In fact, he took it to the next level with the iPod, iPhone, and iPad.

But I don't believe computers should be sterile. It doesn't make sense. They're fundamentally generative. And it wasn't just Jobs that pushed them to be more sterilized. Every business in the industry wanted to sell computers to more people. They were all incentivized to dumb down and sterilize computing. Not just the hardware, but where it hurts the most: the software.

The upside is that computing is now everywhere. Perhaps sterilizing computing was necessary to get here. The problem now is that what we have is a stunted form of computing. I call this Pop Computing.

Generativity and hacker culture

The Apple II expansion slot story was as much about Woz as it was about Jobs. It wasn't just telling about Jobs and his appliance strategy, it was also about Woz having a preference for generative technology. This is significant because Woz is a symbol of hacker culture, and I don't believe his bias for generativity is separate from his values as a hacker.

I'm beginning to define a hacker as a person that needs to use and build generative technology. It might not be a conscious drive, but I believe generativity is part of hacker DNA.

Think about it: they prefer using technologies that have more generative properties. Open source, extensible, programmable … "hackable". They often have a distaste for products that are more sterile. For example, Apple products. Most hacks, including life hacks, are a form of repurposing or applying generativity where it wasn't before. Jailbreaking the iPhone is making a sterile device more generative.

Some say hackers are about freedom, citing open source and free software. Perhaps they're about unchaining from control and authority. But there are plenty of hackers that are okay with some control, or okay with closed source. So I don't believe these are the ideas that define hacker culture as a whole.

Being a hacker seems more about the acts of building, tinkering, and repurposing. But what is the motivating force behind these actions?

I think hacker culture is about a bias towards generativity. Using and building more generative technology. Perhaps hackers are just people that intuit the importance of generativity for humanity.

The point of technology and generativity

In What Technology Wants, Kevin Kelly explored the biases and trends of technology across human history. He broadly defines technology as anything useful created by a human mind. This includes hammers and gadgets, but also law and cities. He found that technology helps us trend towards more choices, opportunities, possibilities, and freedoms.

These are all different ways of talking about the same thing. Every new technology brings us the potential for more new technology. Again, this is not just tools, but art and ideas. Now imagine Mozart before the piano. Or van Gogh before cheap oil paints. Or Hitchcock before film. These creators were made in part by their mediums. If their technologies had not been invented or discovered before their time, their voice might not have been heard.

How many kids are out there today whose medium doesn't exist yet? Kids that might not be able realize their full potential. Having an outlet that resonates so strongly that it fills their life with purpose and meaning, or at least a way to express themselves.

Vine, a micro-video platform, has allowed for a new form of celebrity: Vine stars. This is different from being a movie star, or even being a YouTube star. It's an opportunity that didn't exist before. Or think of Amazon Web Services and how many startups, sometimes just one person, were enabled with on-demand compute technology that was previously only available to the biggest tech companies. If these are too trendy, just think of how many opportunities were opened up by cheap automobiles and later the Interstate Highway System.

In contrast, imagine a world without technology. Where we were incapable of making any sort of technology, including hunting tools, fire, or language. We as a species would probably not survive nature.

So it seems in general, the force of technology is a Good Thing. Toolmaking is what makes us human. It helped establish humanity. And from there it allows us to find new ways to meaningfully exist as individuals and thus as a species.

By definition, generative technology creates far more opportunities and possibilities than its sterile counterpart. Generative technology is more aligned with the basic trend of technology than sterile. Perhaps you could say it's a more potent form of technology. I'm not saying there isn't a place for sterile technology. But understanding the value of technology and generativity, it makes sense some of us would reject a world filled primarily with sterile technology.

Generativity is important to me. I think it's important to hackers, creators, and producers at large. I think it's worth fighting for, even if to just achieve a better balance between the two. We should at least have a choice of whether our technology is sterile or generative. But it seems that a dominantly consumer-driven society will unfortunately always be biased towards sterile technology.


Dec 04 2015

Leadership, Guilt, and Pull Requests

I have a lot of open source projects. Even more with Glider Labs. Some of them are fairly popular. All of them get me excited. But most of them also bum me out. I'm going to share one of the reasons I've had to take a break for the past couple months, and why all my repositories are now looking for more maintainers.

Open source is hard. It seems easy, though. You just write a piece of software and put it on Github, right? Well that was the easy part. Now comes maintenance. And very likely politics. Inevitably, guilt. Multiply that by the number of open source projects you have and their popularity. End result: open source can be a bummer.

Jacob Thornton (@fat), co-author of Bootstrap, gave a talk a few years back echoing the sentiment of many open source authors and maintainers. He calls it Cute Puppy Syndrome. It's not the best analogy, but it gets the point across. Open source projects, like puppies, are great fun when they start. As they get older and more mature, responsibility seems to outweigh their cuteness. One solution is to put your old dog up for adoption and get a new puppy. As you can tell from his delivery, this analogy is intended to be humorous:

He mentions that many authors of popular open source projects have gotten burnt out and look for an exit. Often handing projects off to maintainers, sometimes never to return. Not to avoid responsibility, but to stay sane. Still, much of the time, that sense of responsibility lingers. As Jacob expands on the puppy analogy:

If you have your puppy and it turns into a dog, you put it up for adoption, you give it to a maintainer. And then he over feeds it and it becomes fat and bloated. And you just sit there and you're really sad because you don't really have time to take care of your puppy any more, but you don't want to see it fat and bloated. So you're just real sad all the time.

Alternatively, you can let issues and PRs pile up. Guilt and sadness either way. At least opening the project up lets it survive and continue to provide value to a larger audience. You just have to let go of the project as it will now evolve in ways you might not agree with.

When I did this with Dokku, the new maintainers did a great job at keeping the project and community healthy. I can't thank them enough for that. I had to let go quite a bit, but the project would probably be dead without them.

In fact, there's something interesting about maintainers that didn't author the project. It's probably different from person to person and project to project, but the maintainers of Dokku don't have the guilt or burden that I do. They're happy to help, and as volunteers don't feel like they owe anybody anything. It's really the ideal situation. Perhaps authors shouldn't be maintainers after a certain point.

That said, even with these great maintainers, Dokku really only kept on an incremental path of maintaining the status quo. That's not necessarily a bad thing, but it meant Dokku wasn't able to develop further in the directions I had originally intended. I thought to myself, well eventually I'll find time to do a system-wide refactoring to get it on this path I want and submit these as PRs like any other contributor. That time never came, and the project continued to fall behind from the evolving larger vision. The project I started was not living up to my own expectations for it.

Sadness. Guilt.

Then I did something different. It was so simple. I wrote a wiki page describing what I wanted and why I wanted it. For some reason it came as a surprise to me that the maintainers started moving the project in that direction! Did it happen exactly how I'd do it? Not always. But it still brought the project closer to what I wrote down.

This shouldn't have come as a surprise. In essence, this is leadership. There are different forms of leadership, but at the core is the idea of "saying what, not how". It can be very hard for programmers to get into this mindset because our medium is all about the how. Stepping back and writing what you want with flexibility towards how it's implemented takes practice.

This experiment with Dokku was far from perfect. In fact, that document is still incomplete. Project leadership is just as ongoing as maintenance. However, it's something worth getting better at. It's essential to authoring many open source projects and remaining happy enough to keep going. In the case of my projects, since there is always a bigger picture they fit into, it's even more important.

Dokku is just one of many projects, but Dokku is one of my only projects that I'm not an active maintainer. Dokku isn't why I had to take a break, it was all the others.

Some of you might have seen my ramblings about Megalith. Some of you might even be able to follow them enough to see that most of my open source projects are all basically part of Megalith. Or that Megalith is basically all my projects. You can probably see how this leadership is critical to sustain all these projects while keeping them moving in roughly the same direction.

I don't write open source software to make money. In fact, even solving a particular problem is secondary to working towards a vision of how the world should be. Since that's really what's important to me, I should be spending my time on being an effective leader. At the very least, documenting what I want, the direction, why it's important, what design principles are involved, preferred architectural patterns, and so on. Then helping people understand, integrating their feedback, and letting go of a lot of the details.

To support this, I need to open up our projects to more maintainers. Going forward, I'll be trying a variation of the Pull Request Hack to get more people involved across all projects. If you submit a solid substantial PR or several solid minor PRs to any Glider Labs project, you'll be invited to have commit access across all projects.

Starting now, all public projects under my username or Glider Labs have an open call for maintainers. If you'd like to volunteer to help maintain any of these projects, just join our Slack and in #intros say you're interested in becoming a maintainer.

From there I'll do my best to provide guidance and leadership. Together we'll keep making great things!

Oct 05 2015

The Next 10 Years: Megalith

I've decided what I'm going to be working on for the next 10 years. It's epic and exciting, and I'm going to need your help. It's called Megalith.

Megalith is a symbol of the ideal I've been working towards my entire career. It's constantly evolving and very nuanced. I'm going to be upfront and say that I'm not going to be able to fully explain what Megalith is in this post. Instead, I'm going to start setting up context. To me, context is everything.

For the past few years I've been spending most of my working hours writing open source software related to a project I worked on in 2012 called Docker. Docker was created as a skunkworks collaboration between me and some talented engineers at dotCloud, now Docker. The company pivoted 100% to Docker and is now worth about a billion dollars. As an independent, I didn't stay with Docker, I moved on to the next problem. Docker was one piece of a grander vision.

To help pay for this lifestyle, I've experimented with sponsorships and even fell into lucky situations. Last year I tried to make it a little more sustainable by starting Glider Labs with a friend. We focused on consulting around Docker. We helped some of the first people to actually run Docker in production. We did this to learn and get a better grasp on the Real Problems. Something many vendors in this space don't really do. We learned a lot and as a result we ended up making a lot more open source software.

The problem is that there is a lot more to make. This ideal I have in mind that's been developing in my head for over 5 years is a massive undertaking. I've realized if I'm going to keep pursuing it, I need two things: a better vehicle for the work, and a unifying project to get help around it.

Building a better organization for this work

The software I write that people love comes from a compulsive drive that goes beyond and even against the idea of startups. With the exception of Docker and a few other collaborations, I've never made anything that people loved while working for a startup. Naming and evangelizing webhooks was not something anybody paid me to do. In fact, a lot of projects I've built or think should exist are too small to sustain a startup. Does that mean they shouldn't be built? Or that I should temporarily dedicate my life to maybe make one of them work as a startup?

Even a lifestyle business is quite a commitment to make work. My friend Alan Shreve made Ngrok, inspired by my tool Localtunnel. It's free and open source, but he's also bootstrapped a business out of it. This business is what he spends most his working hours on.

Given my goals and values I do prefer this approach, but it still poses a problem. The time spent writing lines of code to support a business, the time spent figuring out market fit, the time spent on support and operations … this is time not moving forward to me. It's extracting wealth out of something that already exists.

Why do this? So Alan can sustain himself and potentially fund other projects, right? In the meantime, I know for a fact that there's a lot of great open source software that he's not making.

His goal is passive income. For a lot of us independents, that's the dream. It may or may not realize in full, but it's certainly time consuming either way. In that way, it's sort of just a smaller variation of the startup lottery.

Meanwhile, in the same time, I've put out dozens of open source projects that solve problems or work towards dissolving larger problems in the long-term. I actually can't help it. It's compulsive like I said. The only way I see it stopping is if I leave the space altogether. I don't get paid to do 90% of these projects. They help bring me contract work, but seemingly only to take time away from supporting and building a community around those projects.

The other problem, for me, is that running a business causes you to make software differently. You think about building software you can sell, or that supports what you can sell. More than doing one thing well, you think about the features people will pay for. More than making it simple, you think about obscure enterprise and legacy use cases. Conventional knowledge says you must do those in some way at some point because that's How It Works.

The problem is worse for startups that take VC money. Even VCs that "get it" and let you focus on open source traction still expect you to eventually figure out how to monetize and make them millions. To varying degrees this often makes startups:

  • focus on enterprise customers, not regular developers
  • ship software that solves short-term problems, or yesterday's problems
  • prioritize sexy demoware without production hardening
  • increase perceived value with more hires and more partnerships
  • de-prioritize any effort outside the product that makes money

Hashicorp is one of the best examples of companies in this space that have done a good job at taking just enough VC and working against a lot of these forces. However, they still work within the framework. They still have a commitment to exit big someday. The implications of this are not insignificant.

I'm much more likely to bootstrap a company like Alan than take VC money. Not only is it just more my style, but a VC startup just won't play to my strengths. Though, building a bootstrap business doesn't seem to produce the most value for my time either. Or make me very happy.

I'd much rather find a new way. Not just because I want to play to my strengths, but because I know I'm not the only one this applies to. I also know that a different, better kind of open source software will result if done properly.

What I want is something of an independent R&D lab. I want us to re-capture the innovation and invention of Xerox PARC and Bell Labs, but focusing on open source. I want us to have the freedom to explore and build software right with like-minded people. Not to get rich, but to slow cook software. Systems software that further empowers individuals and small groups … enterprise customers of the future, not the past.

This is what I want Glider Labs to transition into. In fact, it's already been operating like this in a way. And I've been exploring and learning ways to make this work for years now. It's part business, part cooperative, part public service. But to make the leap to a lab that supports more than myself, it won't happen over night. And I can't do it by myself.

Sharing the vision, enabling participation

This isn't just about a new organization. It has to have some purpose, some initial unifying project. In order to start from nothing, there needs to be a clear mission of value. Not just boundless experimentation. Luckily, most of my work does fall under a certain theme driven by a nebulous but nonetheless motivating ideal. That seems like a good place to start.

I've been told if I just wrote down everything I want to build and why, people might be willing to help out. This is challenging both because of scope and its constant evolution. I figured if I just keep making projects people will start to see it, but other than a few people I'm not sure that's working out. So I'm going to try a more top-down approach.

The real project this post is about is a meta-project I'm calling Megalith. It's an umbrella project to help unify and bring a common goal to all the work I've been doing for the past 10 years, and over the next 10 years.

I know I can't do it alone, so the project is designed for participation. It will involve many more specific projects that are open source and independently useful. Many already exist. Most do not.

Whether or not the final ideal is achieved, it will be approached. Lots of value will be produced in the process. Not just software and contributions to existing open source, but guides and how-to knowledge of everything I've learned to lead me to my current conclusions, and everything we learn in the process.

Glider Labs and Megalith are separate but related parts of this venture. Megalith is the meta-project, Glider Labs is the organization. The idea is that they support each other. Megalith makes this new Glider Labs a reality, Glider Labs makes Megalith a reality.

Relevant to your interests?

The first step is to explain Megalith and try to communicate this idea in my head, or at least some manifestation of it. Then everything else will start to make sense. It's almost more about approach and values. It's about an idea of simple, composable, extensible tools to make modern end-to-end development and operations sane at both large and small scale. And making the world more programmable…

Anyway, it's more than I can get into here. I've set up an announcement mailing list you can subscribe to. Sign up and you'll get emails about what's next. I might even email you directly to say hi.

Feel free to get in touch with me, leave a comment below, or help out by sharing this post if it resonates with you. I'm pretty excited, especially since a lot of people have expressed interest so far.

Lastly, here's a silly video I made about it:

Subscribe for updates!