Jan 05 2013

Executable Tweets and Programs in Short URLs

A few weeks ago I was completely consumed for the better part of a day that I would have otherwise spent on more practical work.

Yeah, what? Weird, right? It started from a Twitter conversation earlier that day with my friend Joel:

This wishful brainstorming inspired me to start building exactly that. But first, a digression.

The idea reminded me of an idea I got from Adam Smith back when I was working on Scriptlets. If you can execute code from a URL, you could "store" a program in a shortened URL. I decided to combine this with the curl-pipe-bash technique that's been starting to get popular to bootstrap installs. If you're unfamiliar, take this Gist of a Bash script:

Given the "view raw" URL for that Gist, you can curl it and pipe it into Bash to execute it right there in your shell. It would look like this:

$ curl -s https://gist.github.com/raw/4464431/gistfile1.txt | bash
Hello world

Instead of having Gist store the program, how could we make it so the source would just live within the URL? Well in the case of curl-pipe-bash, we just need that source to be returned in the body of a URL. So I built a simple app to run on Heroku that takes the query string and outputs it in the body, a sort of echo service.

Letting you do this:

$ curl "http://queryecho.herokuapp.com?Hello+world"
Hello world

Which you could conceal and shorten with a URL shortener, like Bitly. I prefer the j.mp domain Bitly has. And since they're just redirecting you to the long URL, you'd use the -L option in curl to make it follow redirects:

$ curl -L http://j.mp/RyUN03
Hello world

When you make a short URL from the bitly website, they conveniently make sure the query string is properly URL encoded. So if I just typed queryecho.herokuapp.com/?echo "Hello world" into bitly, it would give me a short URL with a properly URL encoded version of that URL that would return echo "Hello world". This URL we could then curl-pipe into Bash:

$ curl -Ls http://j.mp/VGgI3o | bash
Hello world

See what's going on there? We wrote a simple Hello world program in Bash that effectively lives in that short URL. And we can run it with the curl-pipe-bash technique.

Later in our conversation, Joel suggests an example "app tweet" that if executed in Bash given a URL argument, it would tell you where it redirects. So if you gave it a short URL, it would tell you the long URL.

Just so you know what it would look like, if you put that program in a shell script and ran it against a short URL that redirected to www.google.com, this is what you would see:

$ ./unshortener.sh http://j.mp/www-google-com

It prints the URL you gave it and then resolves the URL and prints the long URL. Pretty simple.

So I decided to put this program in a short URL. Here we have j.mp/TaHyRh which will resolve to:


Luckily I didn't have to do all that URL encoding. I just pasted his code in after queryecho.herokuapp.com/? and bitly took care of it. What's funny is that this example program is made to run on short URLs, so when I told him about it, my example ran on the short URL that contained the program itself:

$ curl -Ls http://j.mp/TaHyRh | url=http://j.mp/TaHyRh bash
http://queryecho.herokuapp.com/?echo "$url"; curl -ILs "$url" | grep Location | grep -o 'http.*'

You may have noticed my version of the program uses $url instead of $1 because we have to use environment variables to provide input to curl-pipe-bash scripts. For reference, to run my URL script against the google.com short URL we made before, it would look like this:

$ curl -Ls http://j.mp/TaHyRh | url=http://j.mp/www-google-com bash

Okay, so we can now put Bash scripts in short URLs. What happened to installing apps in Tweets? Building an apptweet program like Joel imagined would actually be pretty straightforward. But I wanted to build it in and install it with these weird programs-in-short-URLs.

The first obstacle was figuring out how to get it to modify your current environment. Normally curl-pipe-bash URLs install a downloaded program into your PATH. But I didn't want to install a bunch of files on your computer. Instead I just wanted to install a temporary Bash function that would disappear when you leave your shell session. In order to do this, I had to do a variant of the curl-pipe-bash technique using eval:

$ eval $(curl -Ls http://j.mp/setup-fetchtweet)
$ fetchtweet 279072855206031360
@jf you asked for it... Jeff Lindsay (@progrium) December 13, 2012

As you can see by inspecting that URL, it just defines a Bash function that runs a Python script from a Gist. I cheated and used Gist for some reason. That Python script uses the Twitter embed endpoint (same one used for the embedded Tweets in this post) to get the contents of a Tweet without authentication.

The next thing I built installed and used fetchtweet to get a Tweet, parse it, put it in a Bash function named by the string after an #exectweet hashtag (which happens to also start a comment in Bash). So here we have a Tweet with a program in it:

To install it, we'd run this:

$ id=279087620145958912 eval $(curl -Ls http://j.mp/install-tweet)
Installed helloworld from Tweet 279087620145958912
$ helloworld
Hello world

We just installed a program from a Tweet and ran it! Then I wrapped this up into a command you could install. To install the installer. This time it would let you give it the URL to a Tweet:

$ eval $(curl -Ls http://j.mp/install-exectweet) 
Installed exectweet
$ exectweet https://twitter.com/progrium/status/279087620145958912
Installed helloworld from Tweet 279087620145958912
$ helloworld
Hello world

Where would I go from there? An app that calls itself into a loop, of course!

$ exectweet https://twitter.com/progrium/status/279123541054595074 && recursive-app
Installed recursive-app from Tweet 279123541054595074
Installed recursive-app from Tweet 279123541054595074
Installed recursive-app from Tweet 279123541054595074
Installed recursive-app from Tweet 279123541054595074

Obviously, this whole project was just a ridiculous, mind-bending exploration. I shared most of these examples on Twitter as I was making them. Here was my favorite response.

You may have noticed, it just happened to be 12/12/2012 that day.

comments powered by Disqus