This is where I've got a couple projects I've written or am currently working on.
9px:
We all know NFS sucks. Bitching about NFS is a favorite pastime for many unix admins, myself included.
Unfortunately, nobody's really gotten around to doing anything about it. NFS has gotten somewhat better
over the years; tcp instead of udp helps some, but the protocol just sucks in fundamental ways.
There's been attempts at creating alternatives; AFS and various derivates, CIFS, etc. - but none of them
really do what NFS does, which is provide an easy way of sharing arbitrary directories to arbitrary hosts
completely with (mostly) posix semantics.
Enter 9p: a protocol with none of NFS's failings; it's simple, TCP based, and fast. A USENIX paper shows it to be nearly twice as fast as NFS on the postmark benchmark, with a server (u9fs) that's probably not very optimized. There's even a client in the Linux kernel.
Only problem is, Linux development seems to have kind of... stagnated. There's no authentication support; u9fs
is ancient and the project for the new server doesn't even have any files you can download, and the protocol can't
quite provide posix semantics.
So, I'm writing my own server. Once I've got 9p2000.u implemented, I'm going to define my own extensions to the
protocol, document it and implement it; the goal is something that can be used as a nearly drop-in NFS replacement,
and a whole lot more. Users will be able to mount any directory they have access to, without configuration.
I'll have full (hopefully) posix semantics, host authenticated mounts (like how NFS does, with a similar config file),
and better reliability than NFS. I hope - I've still got a ways to go.
Currently, I'm pretty close to running my 9p server for the first time; all I've really got left is reading
from directories. After that, it should work similar to NFS, with host based authentication; real authenticion
so users can mount will take more work.
I'm planning on integrating SSH style authentication; it should use host and user keys, so
you'll have better security than nfs (no spoofing). Plan 9 has an interesting and powerful authentication
architecture, but I'd like something that works easily and out of box on Unix (plus I haven't been able to find any
real documentation on Plan 9's authentication).
I've got some other interesting plans for 9p2000.x, including notification via Inotify to allow for aggressive
caching; writes from multiple clients should also be a lot saner than with NFS. At some point I'm going to have
to implement these extensions in the kernel; that'll take some work, but it ought to be fun.
As soon as I get 9p2000.u completely implemented, I'll put up some concrete specifications for what I plan for
9p2000.x.
This is a program that uses multicast to send files to multiple machines at the same time.
I wrote it for syncying VMware images out to a bunch of lab machines; 8gb * 15 takes awhile with http or scp.
It listens on the local machine and uses ssh to start the receiving end, which connects back for the tcp stream.
It's pretty functional; the default mode takes a source and dest, so it's like scp with a bunch of targets.
Or you can use it in sync mode (-S), which sends files to the same location as on the local machine. Very handy
for setting up one lab machine, then syncing all the relavent files out to the rest of the lab with a single command.
It sends md5sums and doesn't overwrite anything until the transfer's been verified, so it'll never corrupt data.
It also falls back to tcp if for some reason multicast isn't working.
It could use a little more polish, but mostly it needs some real documentation; I haven't worked on it awhile, and
I'm the only person who really uses it. At some point I'll include some example scripts that make it easy to use.
Say you've got a really fat but high latency link, and you want to run something over it that uses tcp... but it
can't use the whole link, because of tcp windows or whatever. At least that's what they told me happens, so I
wrote this. I don't have any big fat but high latency links to use it on, but it does work.
What it does is set up a proxy between two machines, each running showerhead; you connect to one, and the showerhead
on the other end connects to whatever you told it to when you started it. All your data gets sent over some large
(configurable) number tcp streams, and reassembled on the other end.
You can run a completely arbitrary number of connections over it; I've tested up to around 20
(netcatting huge files, mostly).
This one should be better documented and easier to use.
Contact me