Steve Sloan

Steve Sloan

Hacker, drummer, primate

ODIN: Secure P2P on the Web

The first thing you need to understand for this to make sense is that the magic of OFF is all in the block encoding which turns meaningful bits into ownerless, quasi-random blocks identified by their SHA1 hashes. Once this transformation is performed, there is no way for anyone to know the content of a block on the network, as it has no meaning until it’s combined with other blocks (and, in that way, may actually “belong” to multiple files, simultaneously).

The thing that first screamed “Rails app!” to me is the fact that, once the content has been OFF-encoded into blocks, any transport mechanism can be used to move these blocks around, allowing for a network-agnostic peer-to-peer system. Using HTTP, I could write my own smart, caching web peer (with combined web UI) in Rails, but since it’s using HTTP, I could also use dumb web server file servers as mass block stores (no CGI required!). Eventually, I could add (trivial) support to talk to the existing OFF network (also HTTP, AFAIK). If I watned to get really crazy, I could make the app a 3-way network bridge with BitTorrent (which also uses SHA1-identified data blocks).

This is an extension of earlier musings about extending HTTP (or coming up with some other standard) for enumerating mirrors of a particular web resource, something like a per-directory MIRROR file (in XML). A local web-proxy/daemon would cache the status, speed, and locality of various mirrors, and attempt to provide the best location for a particular resource (Use Case #1: Linux RPM repositories). Throw in resource discovery, and you have a basic peer-to-peer network, all over HTTP. While this project assumes that each peer is serving OFF-encoded blocks, there’s no reason why this assumption couldn’t be refactored-out, leaving a generic HTTP P2P system.

To date I’ve already completed most of the functionality of ODIN: fast block encoding (more on that later), file insertion and extraction, basic file transfer and peer discovery. I’ll post more when I have something to show-off. ;)

Tags:

Updated:

You may also enjoy

Ruby Gotcha: Chained Assignment

I ran into a tricky bug that involved an obscure bit of Ruby behavior, involving the chained assignment idiom and overloaded assignment operators.

HTTP Caching Like an Idiot: Redux

Apparently Steam didn’t read my last blog post, as they’ve discovered their own way to Use HTTP Caching Like an Idiot. Come on, people, what’s the point of m...

How to Use HTTP Caching Like an Idiot

I’ve found that it’s pretty easy to run your own custom blog for with zero hosting cost, and even survive some significant traffic spikes, with liberal usage...

Small Footprint MongoDB

MongoDB comes configured out of the box for maximum performance and reliability on production databases. But it can be a bit of a disk hog, and if you’re usi...