Building Kobako with AI: Will It Eventually Crash?

This article is translated by AI, if have any corrections please let me know.

Last week I published Kobako: Letting Agents Safely Operate Rails, introducing the goals of the Kobako gem. Since then I’ve kept pushing development forward with Claude Code—but I quickly ran into a situation that demanded major changes. Is this simply the fate of developing with AI?

This is a question worth discussing: when using AI to assist development, is the problem that AI isn’t capable enough, or that the design we humans handed it was too poor?

Uncertainty

Before the AI era, we already had many ways to run software development teams. Agile, for example, is one approach to handling uncertainty.

Why do we need to handle uncertainty? Because before we ship software that’s actually usable, it’s hard to know whether users genuinely like it. Likewise, before our implementation and design are actually run and modified, it’s hard to know whether the technology and architecture we chose can truly keep running.

That’s why software architecture theories (such as Clean Architecture) were proposed—so that, under the right conditions, we can modify or adjust things easily.

In the AI era, we still face the same problems, but development moves ten times faster—and we hit problems ten times faster too. In just one week, I immediately ran into a situation where my initial design hadn’t accounted for what would happen when adding features later.

Kobako took about a week to reach what I’d consider a releasable 0.1.0—maybe not that fast. But there was also half a month of preparation beforehand: technical research, proof of concept, prototyping, and so on. On the foundation and experience of traditional software development, I’d done what needed to be done well.

Preconceptions

As I mentioned in the previous post introducing Kobako, I drew inspiration from druby and adopted RPC for communication—but this ultimately led me to a problem that was hard to deal with.

Because WASI Preview 1 (WebAssembly System Interface) in WebAssembly doesn’t offer a normal I/O mechanism, building an I/O Pipe to communicate based on the druby or RPC concept is fairly complex. Based on the material Claude Code gathered for evaluation, using a Host Linker mechanism is simpler.

First you set up an Import Function so that code inside WebAssembly can call this C method; to pass a return value back, you use an Export Function to call into WASM, allocate a block of memory, and place the return value there. With this round trip, I implemented the mechanism for binding Rails objects so that mruby inside the WebAssembly environment can use them.

But this runs into a problem. A normal RPC connection would look roughly like this.

 1# Client-side
 2conn = RPC::Client.connect("remote_addr")
 3
 4# Server-side
 5server = RPC::Server.bind("address")
 6server.accept do |conn|
 7  req = conn.read
 8  # ...
 9  res = server.dispatch(req)
10  # ...
11  conn.write res
12end

On the Host side it plays the role of the Server, where both “read” and “write” happen over the single connection the Server obtained—that is, one I/O Pipe handles everything.

But when Kobako uses the WebAssembly mechanism to implement binding, the situation becomes like this.

1sandbox = Sandbox.new
2sandbox.on_dispatch do |req|
3  res = sandbox.server.dispatch(req)
4  sandbox.reply res
5end

In reality, the one holding the I/O Pipe is the Sandbox itself (originally designed as Kobako::Wasm::Instance), yet the Sandbox also owns a Kobako::RPC::Server instance. To forward requests, you’d have to implement it the way shown above—but designing it as Sandbox#dispatch actually makes more sense, since the RPC Server is essentially just a Registry and doesn’t have the nature of a Server.

On top of that, the RPC Server isn’t enough to handle all the Dispatch work, because there’s also the Kobako::Handle mechanism, which lets Host-side objects be passed into the Guest like pointers. It’s handy when a bound object returns an object that can’t be encoded by msgpack.

By this point you realize the relationships between objects have basically become a mess: responsibilities are hard to separate, the concepts don’t match the object names, and there’s a high chance everything is coupled together. This is hard to notice early on, but it’s a problem you must face.

What finally made me hit this problem was wanting to support Block—an important Ruby language feature. Even though I can’t support it 100%, limited support already covers many scenarios—yet the problem above directly blocked this mechanism from being implemented.

Starting Over

To solve this problem, the intuitive old approach would be to scrap it and start over—after all, once you know the new constraints, it’s easy to rewrite without any baggage. But the AI era has its own strategy to draw on: re-analysis.

I first had Claude Code set aside the definitions in the SPEC.md spec and re-analyze from the angle of “Kobako wants to design a Ruby language sandbox.” Assuming the current implementation looks like this, and given my goal set up this way, what role do these implementations play?

In a few hours or so, I re-distinguished a set of new concepts (modules):

• Codec - resolves the differences between Ruby and mruby; basically unchanged, still using msgpack
• Runtime - the environment that runs WebAssembly
• Transport - the mechanism for Ruby and mruby to communicate
• Catalog - the resources provided during execution (such as Binding, Handle, Snippet, etc.)

In the end, the Sandbox object handles all the orchestration. The dependencies among these objects are almost flat, and I deliberately kept them one-directional (no circular dependencies) to keep things simple.

When redesigning, I locked the Sandbox interface to the last released version, narrowing the impact of Breaking Changes as much as possible—even though no stable version has been released yet.

This still brings a new challenge: a change like this counts as a “major refactor,” which in the past would have met a lot of resistance and dragged on for a very long time. With the help of AI tools, though, the basic adjustments were done in about three days—by the time this post is published, I may well have already finished tidying up and released the latest version.

Coming back to the original question: rather than AI not being capable enough, it’s more that AI amplifies the “design” variable—good design lets AI run further, while bad design makes the project crash faster.

In conclusion, developing with AI makes you hit the challenges that always existed in traditional software development faster. If you keep ignoring them, things will eventually devolve into a Big Ball of Mud. Traditional development theory is still very useful—and you’ll reach the point where you need it faster than before.