Lightweight Continuous Documentation Build

You know this fixation we all have with the "document"? Yeah, that's the one, the one where we get all precious and don't want to show anyone the "document" until it's all nicely polished.
Well nuts to that!! I've been casting around for some time now to get a nice simple workflow that combines a Git repo with a lightweight wiki engine. I've finally got it up and working and happy. It's a bit of a sweet thing.
Here's how it works:

1. Write your stuff - you guessed it - in Markdown. Store the files in a Git repo.
2. Don't bother creating the "document". Instead break up what you are writing into manageable chunks and put each of those chunks into a file on its own.
3. Every now and then when you reach a point of wanting to go for a walk or running out of puff on what you are writing, do a commit in your repo. If you're not ready to show the world all of it then do a stash of the not-ready bits. But don't be too precious, this is about the conversation.
4. When you commit a post-commit hook runs. It does two things:
1. It issues a command "markdoc build". More on Markdoc in a minute.
2. It copies the resulting html files to the htdocs folder on the webserver.
5. The webserver serves your wiki content to the web.

It's really simple, really lightweight and really robust. Markdoc is the key component. It is a small Python script that takes the Markdown files and converts them to html and provides a tiny bit of structure. It can cope with a directory structure. It gives you back your html files along with a very simple directory listing "home page". Markdoc does this all with a two word command issued in Terminal on the Mac. It has an equally lightweight web server as part of the distro. Simply type "markdoc serve" and it will tell you where you can find the web pages with your browser.

I've chosen to ignore the built in server and instead I just copy the files across to the Mac server. That has a number of advantages. I can put the files for multiple wikis, each in their own directory, within the web root directory and serve from the web root directory. When I browse to the page I just get a directory listing, each of which is a wiki in its own right. When I open a wiki I get Markdoc's lightweight home page. It's very nice. Secondly I can use the Mac server's web hosting system to create a realm that requires access control. That means the wikis can be public or private as I please.

If I want to collaborate with someone on the writing, I don't do it on the wiki. That's only static html. Instead I get them to clone the Git repo and then push their changes back to the remote repo.

So why is this important? Well it means that we can share our progress with our team, our stakeholders and our collaborators. We don't get locked into "you'll have to wait for the next version of the document to see what's in it". Instead our team can see where we are up to, what we are thinking and where we have issues. It fosters discussion, collaboration and team work. It means we can't go off down a blind alley for too long before somebody notices; it means that people have a chance to see where we are going with our writing (and hence with the project) and to offer thoughts and ideas about how we might progress next.

It brings us one step closer to real-time collaboration. One step closer to breaking down the barriers that we've created, inadvertently, with the "document" and with our cumbersome tools. It allows us to step back in time using Git to reach a previous place if we realise we've drifted off track. It allows us to create a branch and work on an idea without fear of losing our work or getting confused. And all the while we are sharing our progress and sharing our ideas.

Very neat, it creates a very good working environment.

If you want some more details of the scripts and hooks let me know and I'll put them somewhere.

Formatting Doesn't Matter

Tools like Word and FrameMaker, InDesign...they're great. You can tweak and twiddle with the formatting to your heart's content. You can make the content look just like you want it to, down to the smallest little thing.

But here's the rub: It doesn't matter a damn. The reason you are creating this content is to communicate with others. The people you are trying to communicate with don't give a damn about all the tweaking and twiddling that you've done. They simply want something that's clear and simple to understand. So you spend a very high proportion of your content time fiddling with formatting - for no good reason whatsoever.

Before you come chasing after me and telling me I'm wrong, I know that what I've just said doesn't apply to marketing and advertising material. If you are going to put an ad in a colour magazine, then yes you should tweak it to within an inch of its life.

That kind of content however only accounts for a very small proportion of the content that companies create. By far the largest proportion of the content is what could generally be called "technical documentation". My contention is that formatting - beyond the most basic - only matters to one person: That person is you. It doesn't matter to the consumers of that content. The consumers of that content don't care about all the time that you spend tweaking that content. They only care that the content is clear and simple to understand.

I heard an alarming statistic recently: A company had a documentation process where the engineers wrote the documents in Word and then the writers came along and edited it and tweaked it until it looked right. How long do you think it took to create each page of documentation? Eight hours! Eight long hours.

That company changed to a process where the engineers input the content into an XML editor and then the writers edited it for sense. The content was output to publication through a standardised process. The new documentation approach took 1 hour per page of content.

That is a huge difference. It is a huge saving for any company, particularly when companies report that documentation costs around 6% of revenue.

I think there is a lifecycle with writers. At some point in their career they become obsessed with presentation. They love the tools that deliver that presentation to them. Whole departments become obsessed with presentation, and indeed whole companies can become equally obsessed with the "company look and feel". Fine if you are delivering a web site. Forget it for the rest.

Later writers and companies move beyond the attraction of presentation and begin to realise that most of their content is functional content. Functional content is just that: content with a function. If it fulfils that function and does it well, then that's what you are after.

That's why companies move to DITA, S1000D and other similar standards. Because they realise that they cannot afford to be fulfilling the fantasies of their staff who are obsessed with presentation. Presentation without relative benefit.

That doesn't mean that you cannot deliver presentation using XML standards. It does mean that the design and codification of presentation is a Write Once and Once Only process, just like with content preparation. Then you just run it when you need it.

The worst possible situation is to move to one of the XML documentation approaches and to try and drag your presentation fetishes with you. Then you really are taking it too far!

So for all those people who cannot free themselves from the tyranny of presentation my message is to Just Do It. I promise you'll feel better.