My next mini-experiment is to explore different ways to getting data into Kinetic diagrams.

Kinetic diagrams are stored in plain text files. Initially, my idea described in earlier posts was to create a custom domain-specific-language (DSL) using Ragel, to define all the objects in the diagram. Each object can have metadata attributes. For example, suppose there’s a “Salary” box in my finance diagram. “Salary” might have the following attributes: label=Work Salary, vendor=Coal Mines, and amount=3580.  Here’s how the DSL would describe the objects in the finance diagram:

+ Salary    | label="Work salary",    vendor="Coal Mines",  amt=3580
+ Checking  | label="Checking",       vendor="BigBank",     amt=400
+ Emergency | label="Emergency fund", vendor=" ",           amt=150
+ Savings1  | label="Savings",        vendor="MegaSavings", amt=720
+ Savings2  | label="Savings",        vendor="BigBank",     amt=214
+ Stocks    | label="Stocks",         vendor="Tradetek",    amt=1196
+ Wallet    | label="Wallet",         vendor=" ",           amt=143
+ Purchases | label="Purchases",      vendor="stores",      amt=212

But something about this approach just feels wrong. After all, can I realistically expect users to type vendor=”Coal Mines” in order to set an attribute value? Of course not. Most users aren’t programmers, nor care to be. So my next idea was to allow the user to type a textual data table:

-------------------------------------------------------------
name           | label              | vendor         | amt
-------------------------------------------------------------
Salary         | Work salary        | Coal Mines     | 3580
Checking       | Checking           | BigBank        | 400
Emergency      | Emergency fund     |                | 150
Savings1       | Savings            | MegaSavings    | 720
Savings2       | Savings            | BigBank        | 214
Stocks         | Stocks             | Tradetek       | 1196
Wallet         | Wallet             |                | 143
Purchases      | Purchases          | stores         | 212
----------------------------------------------------------- 

This textual table is much clearer and readable. The gotcha, however, is that this requires the user to format that textual table perfectly. I’d need to define some required format in order to distinguish the header values from the rest of the table, and separate columns from each other. So while readable, this approach is still too brittle for real-world use.

The new approach I’m exploring now is to just read the data from a spreadsheet or database directly. After all, wouldn’t it be great if the user can just edit the data in a spreadsheet?

This will be a fun mini-experiment…stay tuned.

(Taking the cue from Charles Dickens’ weekly serials, I’ll share my thoughts on this topic over a series of posts, with one post each week. On June 16, I briefly touched on the motivation behind Kinetic. Now, let’s dig even deeper into the underlying motivations.)

The central premise behind Kinetic is: Useful diagrams must reflect increasingly-complex ideas and systems. Since Kinetic’s goal is to explore better approaches to diagramming complex systems, let’s first answer the question:

“Well, what’s wrong with the today’s tools to create complex diagrams? Why can’t we just use Microsoft Viso for diagramming complex systems?”

First let me be clear: I absolutely love Visio. In fact, as a Solutions Architect for Motricity (note: Kinetic is personally-funded and is not associated with Motricity in any way), I create and review hundreds of Visio diagrams each year. Anything from high-level technical architecture diagrams to low-level software component diagrams to business process flows. My good friend Sean C. (and the best database engineer I know) jokes that I’m utterly useless without Visio. And he’s right. Visit my desk on any workday, and chances are, you’ll probably see some Visio diagram on my monitor.

Diagramming software such as Microsoft Visio or Omnigraffle allow users to draw boxes, and then drag-and-drop arrows to link the boxes. For simpler diagrams, this intuitive approach works well.

But where tools such as Visio fall short is in dealing with complex diagrams. What do I mean? When diagramming systems that contain hundreds of interacting components, the Visio connector lines start to criss-cross in a spaghetti-like mess. Just for fun, here’s a particularly atrocious example:

Image from: http://thedailywtf.com/Articles/The_Customer-Friendly_System.aspx

Too many times, soon as the diagrammed systems or processes cross a certain threshold of complexity, the Visio diagrams become unwieldy and unmanageable.

In the next post (next Tuesday), I’ll share my thinking on the unwieldiness, and how Kinetic deals with it.

When it comes to web apps, the Ruby community gets it. Rack is a great example of this. What is it? In a nutshell:

Any Ruby web framework now jives great with any Ruby web server. I can scale down from a fully load-balanced production system, to a local Apache, to a quick in-process web server without even thinking about it. I can also add components to the HTTP chain in a snap. Thanks to Rack’s amazing simplicity, it took less than one year to move from Rack 1.0 to widespread Rack compliance. As Sam Ruby put it: I love it when a plan comes together.

.http://www.flickr.com/photos/russmorris/202752043/

Kinetic’s web servers run Phusion Passenger, which is Rack-compliant. Since both Rails and Sinatra are Rack-compliant, I can hook either web framework up to Passenger. Today, my choice is Sinatra for its bare-bones simplicity…perfect for prototyping! But let’s say, six months from now, I also hop on the Rails bandwagon. Thanks to Rack, that hop will be painless.

Or maybe, just maybe, I decide to swap out Phusion Passenger for Unicorn. Or add a caching server to speed up the site. With Rack, I can freely mix-and-match. I love it, it’s plain brilliant.

Now that I’ve decided Kinetic should have a web interface, the next question is: What features should it have?

I’m learning towards just creating a Minimum Viable Product. I’ll be able to get early feedback on my ideas. Plus it’ll force me to resist my natural tendency to be a perfectionist. Ok, so what might a bare-bones, no-fluff app look like?

1. The user enter texts which describes a diagram.
2. Kinetic then generates the diagram (in PNG image format) from that textual description.
3. Kinetic then displays the generated PNG diagram to the user.

That’s as basic as it can get, and it all fits on a single web page:

So that’s my plan for the next several weeks–to get version 0.0001 up-and-running. It’ll be pathetic and primitive, but at least it’ll be something. What do you think?

What’s the spark behind Kinetic? One day last June, while working on an architecture diagram using Microsoft Visio, several of my neurons misfired, and it’s an itch that I’ve been trying to scratch ever since. The idea is simple: How can we use words to describe complex diagrams? For example, is it possible to have a tool that will allow me to update an organizational chart or a network diagram just by changing a few lines of text?

Granted, this isn’t exactly a new idea. Google and you’ll find dozens of programs. But I’ve yet to find one that satisfies my twin requirements of: 1) usability and 2) flexibility.

1. The tool needs to be simple to use. The litmus test is always, “Yeah, but will my grandma be able to use it?” It should be dead simple to update the organization chart.  AT&T Research’s Graphviz rocks, but in my opinion, the syntax might be a bit too arcane for the average user.
2. The tool needs to be flexible. Can the software handle a variety of diagrams? Ideally, I’d like to be able to use the same software to generate both organizational charts as well as network diagrams. Tools such as Diagrammr are limited in the types of diagrams they can generate.

As it turns out, meeting both requirements is a hard problem. It’s a design challenge, a tug-of-war. Make the tool flexible, and usability suffers. And vice versa.

http://www.flickr.com/photos/kitone/2478029256/

I know it’s a challenging problem. The scary thing is–it might not even be solvable using current technologies. And that’s the very reason I’m drawn to it. Paul Graham calls this “running upstairs.”

Use difficulty as a guide not just in selecting the overall aim of your company, but also at decision points along the way. At Viaweb one of our rules of thumb was run upstairs. Suppose you are a little, nimble guy being chased by a big, fat, bully. You open a door and find yourself in a staircase. Do you go up or down? I say up. The bully can probably run downstairs as fast as you can. Going upstairs his bulk will be more of a disadvantage. Running upstairs is hard for you but even harder for him.

What this meant in practice was that we deliberately sought hard problems. If there were two features we could add to our software, both equally valuable in proportion to their difficulty, we’d always take the harder one. Not just because it was more valuable, but because it was harder. We delighted in forcing bigger, slower competitors to follow us over difficult ground. Like guerillas, startups prefer the difficult terrain of the mountains, where the troops of the central government can’t follow. I can remember times when we were just exhausted after wrestling all day with some horrible technical problem. And I’d be delighted, because something that was hard for us would be impossible for our competitors.

Thanks for joining me on this stair-run.