Red Echo

April 23, 2016

This is the kind of detailed, hacker-friendly explanation of ELF for which I really wish I’d had a Mach-O equivalent when I was writing the Mach-O linker for REALbasic.

April 18, 2016

I spent all weekend working on the Doom Truck with AJ. We are upgrading the shop box, making it a bit nicer to camp in, which mostly means cleaning up all the interior surfaces and building in a permanent bed frame with storage underneath. We’re not going to go full RV style with cabinets and fold-outs everywhere, because we like the open feeling it currently has, and we want it to continue being versatile.

First of all we have to stop all the water coming in, which caused a bunch of problems over the winter. One of the window frames leaks – but there are also lots of holes drilled through the walls and floor where previous owners had bolted down various workbenches and storage containers. Our plan was to weld plugs into all these holes, then grind ’em flat. We can always weld tie-down tabs onto the sheet metal if we want to.

Well… it was a long complicated process, and I started to bore myself trying to recount the details, so I’ll just say that it took half a dozen trips to various hardware stores and over $1000 spent on tools and supplies before nonstop labor finally got us to the point that we could start welding around 4 PM on Sunday. Oh, my.

We did end up making a fair bit of progress once we got going: all the floor holes are filled and ground down, so the wheels won’t kick water up into the box anymore, which means we can put down that nice cork flooring without having it rot from underneath… And I patched up the assortment of bolt holes down one side of the truck with big round rivety-looking carriage bolt heads, welded in place. The welds look terrible and it’s going to need a bunch of cleanup, but at least there’s no more daylight leaking in.

I didn’t get to the bolt holes on the other side of the truck; AJ thinks she will try to tackle those during the week. There are also a couple of big square access panels for equipment long since removed, with extremely janky covers… we’re going to weld the covers on, stuff some insulation in the gap, and weld them shut on the inside as well. Finally, we need to cut away and rebuild the threshold, which has leaked enough water in that the wood is a little rotten…

But hey, this is not a bad set of problems to have. The truck works great; we don’t have any mechanical problems to fix right now. It took a long time to get our metalwork process going but now things are moving quickly. And wow is it going to be nice to park our giant clanky apocalypse-proof monster truck at every festival and camping trip we go on this summer, then throw open the doors and have a clean, stylish, convenient, and actually pretty damn swanky home-away-from-home inside. I’m looking forward to it.

Also looking forward to installing a muffler, now that I can weld stuff in my back yard, because *damn* does that thing need one.

Oh yeah! I just discovered the wonder that is turbodiesel propane injection. It’s like nitrous for a gas engine, giving a significant horsepower boost, which will be super useful when climbing up hills – but not only that, it actually improves fuel efficiency, cleans up the exhaust, *and* puts no additional stress on the engine, since it actually *lowers* burn temperature slightly. Way simpler and cheaper than any of the other power-boosting improvements we’d been considering, and we’re already planning to have a propane tank on board to power the stove, heaters, etc. Super cool, it should mean it’s no longer a hassle trying to convoy out to an event with other vehicles.

April 15, 2016

A convenient list of lightweight, embedded-friendly alternatives to common, elaborately developed unix libraries.

April 14, 2016

Copperhead is a new Android-based mobile operating system with an appealing design brief. They’ve implemented an array of sensible-sounding security improvements, and the technical explanations for these changes are reassuringly lucid. They’re also open-source focused and not tied to any proprietary cloud services, which is exactly what I want. I might have to give this thing a try; I might hate my phone less.

While there are hundreds of C compilers in existence – it’s even possible, though highly unlikely, that the terrible C compiler I wrote back in 1997 is still out there somewhere – there are only two (and a half) of them which actually matter. The maintainers of these compilers increasingly subscribe to a pedantic, user-hostile interpretation of the C language which, as a user of the language, has become rather troubling:

Recently we have seen spectacular advances in compiler optimisation. Spectacular in that large swathes of existing previously-working code have been discovered, by diligent compilers, to be contrary to the published C standard, and `optimised’ into non-working machine code.

In fact, it turns out that there is practically no existing C code which is correct according to said standards (including C compilers themselves).

Real existing code does not conform to the rules now being enforced by compilers. Indeed often it can be very hard to write new code which does conform to the rules, even if you know what the rules are and take great care.

It’s an interesting post by Ian Jackson of the Debian project which some additional links that are worth reading if you have an interest in this sort of thing.

April 1, 2016

I CAN’T STOP: it’s a Mackie SWA1801 for $650, JUST WHAT I ALWAYS WANTED (since the last time I got what I always wanted, temporarily(?) sating my PA-equipment lust with a pair of SWA1501s).

No, I’m not going to buy it. I’m not. Where would I PUT it? What about the fact that the gear I’ve collected will already max out a 20-amp circuit? For that matter, would this beast even fit through my basement door? Why do I keep looking at these things?

Well, I guess that one’s obvious: MORE BASS.

March 23, 2016

Why Microservices:

Lately, some of these SCS turned out to be still too large, so we decomposed them by extracting several microservices. Because we are already running a distributed system, cutting applications into smaller pieces is now a rather easy exercise. One of the reasons, why I agree with Stephan Tilkov that you should not start with a monolith, when your goal is a microservices architecture.

This article is not about the pros and cons of microservice architectures. This article is mostly about the pros. Not because they do not have downsides, but because I’m biased and completely convinced that microservices are a great idea.

March 22, 2016

My response to this article is a resounding “well, yeah,” but it’s interesting to see someone saying it, and in a social-science research context, at that:
People Want Power Because They Want Autonomy:

All told, this research indicates that the desire for power may be somewhat misplaced: Generally, when people say they want power, what they really want is autonomy. And when they get that autonomy, they tend to stop wanting power.

March 19, 2016

I don’t hate aggravator any more and may eventually have to change its name. I blamed the hardware, since it’s running the same Ubuntu 14.04 I’ve used successfully on three previous Thinkpads, but installing a different video driver fixed the “crash on resume” issue, which was really a “crash when the machine discovers you’ve plugged in its power cord” issue, and uninstalling whatever the hell “apport” is made the relentless barrage of “the machine has crashed, do you want to send us all your personal data so we can pretend we care” messages stop accosting me at startup.

With a second battery replacing the DVD player, I feel comfortable leaving the power supply at home, now, too.

Yay.

March 15, 2016

Archive of Interesting Code is a long list of clean, readable, well-commented implementations of useful algorithms.

March 12, 2016

I recorded my DJ set last night – dubby, gritty downtempo – and have just finished uploading it for your enjoyment:

March 2016 at the Dungeon

March 8, 2016

Notes about getting better power management out of Linux when running on a Thinkpad X300, like the one I have been so frustrated with recently that I gave it the hostname “aggravator”. I get maybe 90 minutes out of its battery, currently. I was thinking about replacing its DVD player with a second battery, but I’m reluctant to throw even more money at this thing when its reliability has been so underwhelming thus far.

March 6, 2016

KreativeKorp Relay Fonts has a glorious array of retrocomputing typefaces pulled from sources I haven’t thought about in years.

March 5, 2016

The junk removal people have come and gone. With them went all the random objects that have been sitting in the backyard at Rock Star Parking, unclaimed and unloved, since last summer’s Floodland truck unloading. In addition, they departed with an assortment of mysterious objects that had taken up residence under the porch for no apparent reason. Why, exactly, did we have two portable charcoal barbecues? Where did that knit bag of rubber stamps come from? Who put that big black plastic bag next to my motorcycle, and what was inside it? (None of us were brave enough to open it and find out.) And what kind of person brings a goddamn swamp cooler to a party and then forgets to take it home again afterward?

Also gone: the massive slab of seating furniture which would have been the first component of the now-cancelled couch-fortress project, the coffee table I’ve disliked since the day my ex-wife surprised me with it, and some bags of clothing which did not belong to any of this house’s past or present occupants. Alas: the cans of paint, piles of lead-acid batteries, and terrifyingly rusty propane cylinder were problems they were unwilling to solve. I guess it’s time to break out the box of thermite.

March 4, 2016

Resources for Amateur Compiler Writers

March 1, 2016

After sixteen years of membership, I just closed my Metafilter account. End of an era: it was the last piece of the old Web, the one I was excited about and emotionally invested in, the one that felt like a community – or a community of communities, really. No more; Metafilter’s culture has been growing steadily crankier over the years, less fun and more nitpicky, and has finally sunk below the “never read the comments” threshold.

The web is over; it’s basically just television, now, with a few exceptions. Reddit is still fun, though it’s more of a playground than a community. Blogs are over, message boards all seem to have gotten swallowed up by Facebook, and Facebook itself doesn’t have very much going on anymore.

I’m not sure what comes next, but I’m looking forward to finding out about it.

February 29, 2016

I didn’t get any photos, but the verdict was clear after Saturday night’s impromptu gathering: you totally can make delicious grilled-cheese sandwiches with a flamethrower. Use a soft cheese and keep the slices thin (so it’ll melt quickly), don’t butter the outside of the bread (it catches fire), and smash the sandwich between a couple of cast iron skillets (or use a panini press) if you’d rather not char the outside. 15-20 seconds of heat will do it, but you can go longer if you want the bread to get super crunchy.

Also, don’t assume that those propane cylinders you haven’t touched since last summer are still full.

February 16, 2016

A nice solid side-table lamp made of oak and epoxy, with step-by-step photos.

February 8, 2016

Colorization using optimization:

In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise: neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.

O-expressions are a new syntax for Lisp-like languages, with an implementation called Liso which is based on Racket.

I’ve been playing with similar ideas recently, from a slightly more functional-programming perspective. Everyone agrees, for the most part, that delimiter pairs create structure, and that commas have higher precedence than semicolons. The idea that any consecutive sequence of non-delimiter punctuation characters constitutes an “operator” token is useful, and I like the use of INDENT/DEDENT as another form of bracketed delimiters. Is it possible, given the additional constraint of “no reserved identifiers”, to construct a language sufficiently general that a language like Python, Javascript, or (my own) Radian could be constructed on top of it as a macro library?

February 1, 2016

Aluminum extrusion channels for LED strip lighting.

January 29, 2016

I’ve just discovered a language called Sisal, developed in the early ’80s, which was based on the same ideas that inspired Radian. It was quickly forgotten, as most languages are, but a group of students resurrected it in 2010 and built a fine-grained parallelism backend for it, based on pthreads. Which was, again, the whole point of Radian. It looks like they pulled it off about a year before I got Radian doing the same tricks, but the project seems to have lapsed back into obscurity.

January 24, 2016

bookmark: useful notes on writing makefiles.

January 10, 2016

A straightforward and well-presented article about the hopelessness of the currently-dominant computing model with some sensible suggestions for a better way.

January 4, 2016

I had a startling thought on the bus this morning which led to a pretty exciting idea. What if I combined the work I’ve been doing on lightweight unikernels with all the work I did on Radian, turning Radian into a tool for building lightweight services?

I still think the Radian language is a good one; it’s clean and neat, and its semantics are constrained in just such a way that it’s easy to interact with asynchronous processes, and thus to split work up among multiple workers. As an experienced developer, I could do a lot of productive work with that language – but I think it’d be just as easy for a newbie to pick up as would python or javascript.

As a new language, though, it suffers from the unavoidable defect of being new: there is no language community, there are no libraries, there are no example programs, there are no tutorials, there is barely any documentation. Not if I made it my full-time job and worked the kind of hours I put in when I was doing Object Basic could I make up for that lack myself.

As a tool, it also has significant limitations. The constraints of its immutable dataflow semantics make it very difficult to interact with code written in other, less-constrained languages. That makes it very difficult to interact with the operating system and all of its libraries. Its parallel execution model and its dataflow semantics make it very difficult to debug using a conventional debugger: the traditional mode of stepping through line by line and watching the variables change is basically impossible, because there is nothing in the compiled code which cleanly matches up with the original source lines or variable names.

Finally, and worst of all, I was never able to find a problem for which Radian was a uniquely powerful solution. I looked into datamining, and then into numerics, but in each case I got bogged down in the immense amount of work necessary to get the core libraries up to the point that someone not already dedicated to the language would care to use it.

So, the new idea.

Unikernel-based services are still quite new, and *lightweight* unikernel services are still quite a cutting-edge activity. I’ve been having a great time poking around in the gritty guts of the x86 architecture but that’s because it reminds me of coding in the wild west of the late ’80s and early ’90s. Quite a barrier to entry!

What if I turned radian into a dedicated tool for building standalone microservices? The output would not be an executable program, but a bootable disk image you could start up inside a VM.

This would neatly turn all of radian’s disadvantages into strengths – or inevitable constraints of the target environment, at least. Can’t interact with system libraries? Well, isolation is the whole point of a lightweight unikernel service. Can’t step through with a debugger? Who cares, it’s probably running in the cloud anyway, so you debug it via log files – which is pretty much exactly the way I used to debug Radian programs anyway. Can’t link your favorite library into Radian? Doesn’t matter, just boot it up in a separate microservice, then make them sling packets at each other!

Radian’s strengths, meanwhile, would remain strengths. It’s all about slinging data between parallel asynchronous processes, and what else are microservices? Lightweight tasks are built right into the language, as is a synchronization mechanism which would fit perfectly on top of a socket-based communication mechanism.

Furthermore, there’s basically nothing else out there that does this. The only comparable system is MirageOS, which basically wraps OCaml and a bunch of unix libraries into a bootable VM. And that’s great, but it’s OCaml, and it’s a *much* heavier solution than I’d be presenting.

Radian, under this vision, would become shell scripting for the cloud: letting anyone with the tiniest trace of programming knowledge go from idea to running microservice in a matter of seconds. I am undoubtedly feeling unreasonably excited about this idea right now, but if I could accomplish this, it seems like it might be the most valuable thing I ever did with my career.

So. What would I have to do in order to bring this idea into reality? The radian compiler wouldn’t need any changes; all the work would be on the runtime side. I would certainly have to write some better documentation, though. I’d then replace the fleet kernel’s C runtime with the Radian runtime and rework the radian allocator and threads accordingly. I’d also have to write a little bit of code for emitting a disk image, and for invoking qemu on it.

As far as the radian standard library goes, I’d scrap the fancy but unfinished bignums package and bang out something simple and reliable based on good ol’ int64 and float64. The major piece of new work I’d have to do would be the development of a socket interface. Since I’d be going for an all-in-one runtime library, though, I could scrap the clunky ffi workarounds that used to make this sort of thing so laborious, and just build it right in! Call straight into the kernel, why not.

At that point… we’ve got sockets, tools for working with data going in and out of sockets, tools for managing complex asynchronous processes involving multiple sockets, and a tool that turns source code into a running VM in a snap of the fingers. What’s next? Time to play.

This would be an insanely formidable task if I were thinking about starting it from scratch, but I’ve already done all the hard parts! What I’ve just described sounds more like… assembling pieces.

Wow. Could I actually make all those years of effort pay off? I think I might try it.

January 1, 2016

everything wrong with the 2015 web

The Website Obesity Crisis: starts off pointing out just how absurdly large websites have become, then continues – in remarkably straightforward, plain-spoken terms – to highlight everything I hate about modern web design. Such a satisfying read.

December 31, 2015

two projects:
– router sets up first VM, exposes libvirtd proxy, launches boot process; implements firewall policies between fleet VMs
– shell implementation (bash fork? just posix sh? something new?) uses DNS-SD to locate libvirtd proxy, implements process management in terms of VMs

then, port/compile the programs I want to use as unikernel VM images.

I’ve made significant progress toward the completion of a new, non-POSIX-oriented unikernel library in the form of a baremetal C99 runtime, but even as I’m cruising down the home stretch I think I will put the project aside. Sure, my unikernel library would be tighter and lighter than the rest, if I finished it, but it really doesn’t add all that much, and working on it means I haven’t been making any progress on the stuff that is truly unique to my vision.

IncludeOS is marketed as a cloud-service tool, but when I look at what they’re actually building, it looks very much like what I’m doing. Their project is C++, mine is C; their architecture is a bit more full-featured, where mine is deliberately minimal… but it’s still basically the same idea, and I generally agree with all of their architectural choices. So… why not join forces? I can always dust off my fleet-kernel later, if there’s some problem it would solve better; one of the great things about this VM/unikernel architecture is that you can have as many kernel architectures as you like running simultaneously.

The part I really need to build, in order to get my vision for fleet off the ground, is the router/shell system that will make unikernel-based programs useful as everyday workstation tools. I’ve grown increasingly antsy as development of my unikernel library has dragged on, preventing me from getting to the shell; it’s time to refocus and get on with it.

Maybe I can contribute some of my stuff to IncludeOS along the way, too.

December 14, 2015

Useful information about electric bicycle commuting in Seattle, including links to relevant RCW statutes and recommendations for equipment.

December 9, 2015

I miss the web. I didn’t expect it to be over so quickly. What’s next, I wonder? Should we try to build another one, or is the concept fundamentally flawed?

November 25, 2015

Thanksgiving is a science-fiction story

The proper genre for Thanksgiving is science-fiction:

It has come to my attention that people are woefully uninformed about certain episodes in the Thanksgiving narrative. For example, almost no one mentions the part where Squanto threatens to release a bioweapon buried under Plymouth Rock that will bring about the apocalypse.

Mr. S, an ordinary American, is minding his own business outside his East Coast home when he is suddenly abducted by short large-headed creatures like none he has ever seen before. They bring him to their ship and voyage across unimaginable distances to an alien world both grander and more horrible than he could imagine. The aliens have godlike technologies, but their society is dystopian and hivelike. Enslaved at first, then displayed as a curiosity, he finally wins his freedom through pluck and intelligence. Despite the luxuries he enjoys in his new life, he longs for his homeworld.

Next Page »