Saturday, March 9, 2013

Markdoc as a Task Manager!

If you are familiar with Extreme Programming, you will be familiar with Backlog and Sprint as concepts. Otherwise a simple description is that Backlog is all the bits you have to build and Sprint is those bits from the Backlog that you are currently focusing on. These are usually expressed as short "user stories". Please don't tell me that I've over-generalised...I already know that!
So if I generalise further there's a lot to be said for those same sort of concepts applied to other kinds of projects - not just software development. I wrote previously about the excellent Markdoc wiki script and this use case uses that same script.
I first initialised a Markdoc wiki. The order is important here so do this first. I was intending to make it a Git repo so I used the "--vcs-ignore git" flag so it created its own .gitignore. Then I initialised it as a Git repo. The actual directory is inside Dropbox. Inside Markdoc's wiki directory I created 4 sub-directories:
  • backlog
  • active
  • blocked
  • done
Then I set up a remote repo on our server. These repos are managed by the wonderful Gitolite and that will later be important. This step allows others to clone, work on and then push their changes. Alternatively for users that don't work in Git we can share the Dropbox wiki directory with them so they can work in there.
Then on another user on the server I cloned the remote repo. This repo is the one where the Markdoc build takes place and where the web server serves from. The web root directory is the /.html directory in the repo. A launchd job runs a "git pull" at set intervals. If the remote repo has changed then the pull results in a merge and that in turn kicks off a post-merge hook which issues the "markdoc build" command which populates the /.html directory.

Web access is managed by a web server realm that limits access to group members and requires authentication.
So a couple of explanations are in order. Firstly why have both the remote repo and the repo on the web server? Why not just combine them? Two reasons. Firstly, by default, the remote repo is a bare repo and I think it's good practice to keep it like that. That means that there is no working directory and therefore nothing to run "markdoc build" against. Secondly gitolite fairly tightly manages ssh access to that user so it's not practical to configure and remotely manage the web server from that account.
The wasteful bit is the launchd job which runs, on the web server user, at fixed intervals regardless of whether there has been an update to the remote repo or not. I'm toying with ways of fixing that at the moment.
To use the system you populate your backlog directory with Markdown documents, one for each task. We use a fixed template so that we get all the info we need in each "task story". The first level 1 heading is a descriptive title for the task and this is shown on the web as the document's title.
As tasks become active they are moved to the /active directory and info about who is acting on the task and what the dates are is added.
If a task becomes blocked it's moved to blocked and a narrative explaining things is added. Finished tasks are moved to done.
Many task management systems rely on one line task descriptions. For complex jobs these descriptions are often not enough to keep everyone on the same page and to communicate intent. This system changes that. It also allows all of the team to see where the project is up to and it allows stakeholders to see exactly what's happening in the project.
If one of the directories - for instance /blocked - doesn't have anything in it then it's not shown on the wiki page on the web.
Lightweight, fast and flexible. There are dozens of other web systems for Scrum and XP but I've seen none as simple as this and as flexible for other projects. Tell me what you think.

Sunday, March 3, 2013

Lightweight Continuous Documentation Build

You know this fixation we all have with the "document"? Yeah, that's the one, the one where we get all precious and don't want to show anyone the "document" until it's all nicely polished.
Well nuts to that!! I've been casting around for some time now to get a nice simple workflow that combines a Git repo with a lightweight wiki engine. I've finally got it up and working and happy. It's a bit of a sweet thing.
Here's how it works:

  1. Write your stuff - you guessed it - in Markdown. Store the files in a Git repo.
  2. Don't bother creating the "document". Instead break up what you are writing into manageable chunks and put each of those chunks into a file on its own.
  3. Every now and then when you reach a point of wanting to go for a walk or running out of puff on what you are writing, do a commit in your repo. If you're not ready to show the world all of it then do a stash of the not-ready bits. But don't be too precious, this is about the conversation.
  4. When you commit a post-commit hook runs. It does two things:
    1. It issues a command "markdoc build". More on Markdoc in a minute.
    2. It copies the resulting html files to the htdocs folder on the webserver.
  5. The webserver serves your wiki content to the web.
It's really simple, really lightweight and really robust. Markdoc is the key component. It is a small Python script that takes the Markdown files and converts them to html and provides a tiny bit of structure. It can cope with a directory structure. It gives you back your html files along with a very simple directory listing "home page". Markdoc does this all with a two word command issued in Terminal on the Mac. It has an equally lightweight web server as part of the distro. Simply type "markdoc serve" and it will tell you where you can find the web pages with your browser.
I've chosen to ignore the built in server and instead I just copy the files across to the Mac server. That has a number of advantages. I can put the files for multiple wikis, each in their own directory, within the web root directory and serve from the web root directory. When I browse to the page I just get a directory listing, each of which is a wiki in its own right. When I open a wiki I get Markdoc's lightweight home page. It's very nice. Secondly I can use the Mac server's web hosting system to create a realm that requires access control. That means the wikis can be public or private as I please.
If I want to collaborate with someone on the writing, I don't do it on the wiki. That's only static html. Instead I get them to clone the Git repo and then push their changes back to the remote repo.
So why is this important? Well it means that we can share our progress with our team, our stakeholders and our collaborators. We don't get locked into "you'll have to wait for the next version of the document to see what's in it". Instead our team can see where we are up to, what we are thinking and where we have issues. It fosters discussion, collaboration and team work. It means we can't go off down a blind alley for too long before somebody notices; it means that people have a chance to see where we are going with our writing (and hence with the project) and to offer thoughts and ideas about how we might progress next.
It brings us one step closer to real-time collaboration. One step closer to breaking down the barriers that we've created, inadvertently, with the "document" and with our cumbersome tools. It allows us to step back in time using Git to reach a previous place if we realise we've drifted off track. It allows us to create a branch and work on an idea without fear of losing our work or getting confused. And all the while we are sharing our progress and sharing our ideas.
Very neat, it creates a very good working environment.
If you want some more details of the scripts and hooks let me know and I'll put them somewhere.

Thursday, February 14, 2013

iPad at Work - Writing

This is the first of a series of posts about using the iPad at work. I'm not going to be writing about the iPad as a calendar tool or as a way to look at your email. That's just so fundamental that we'll take it as read that you already do that.
What I want to talk about are the more interesting uses of iPad at work.


This is pretty fundamental as well. We nearly all need to write at work and many users of iPad will be heavy users of the writing capability. So here are some thoughts about writing with the iPad.

Get a Keyboard!

Seriously you can't write more than a quick note without a keyboard. There are lots of options out there from keyboard cases to Apple's very nice wireless keyboard. Whatever your choice get it and make sure that you are comfortable touch typing on it before you shell out your hard earned.

Go Markdown!

On the iPad most of the tools for working with Word are, in a word, kludgy. They are overweight and limited in their functionality. It's just not worth it.
Markdown is the only sensible way to write anything other than notes on your iPad. Don't be put off by the first paragraph saying that "Markdown is a text-to-HTML conversion tool..." it's much more than that. It's the way you want to do your documents because:
  • It's trivially simple to learn. It will take you 10 minutes to master it if you are as stupid as me. Most people take less time than that. There are cheat sheets available too. Here's a visual one. Download one and carry it with you for a while. Soon you won't need it at all.
  • It is brilliant for re-purposing what you write. You can write in Markdown and output to docx, odt, html, LaTeX, ePub...the choices are endless.
  • It is lightweight and human-readable. You can understand a Markdown document really simply.
  • It's text so even as fads and file formats come and go it's text and you can open a Markdown file in almost any editor since the beginning of the computer age. That also means it has probably the best chance of future proofing and it frees you to use the tools that you want, not some heavyweight, expensive, proprietary tool.

Get Dropbox

It doesn't have to be Dropbox, however Dropbox support is becoming pervasive amongst the tools you are going to want to use. Whatever you choose you need a way to simply get things into a position where they are accessible on your iPad, on any web browser and on your desktop/laptop. Dropbox ticks all of those boxes.

Choose a Markdown Editor...or three

There are lots of good Markdown editors for iPad. I like Writeup for most stuff (I particularly like its version management and preview of document contents) and I also like Drafts, particularly because of its Actions and URL Actions. I use Drafts for quick notes and the like.
In addition you'll probably want to get TextExpander, it will improve your productivity on iPad.
On Mac you can choose anything you like, as long as it supports UTF-8. I use a variety of tools including Textwrangler, Markdown Pro and Byword. Textwrangler isn't a Markdown editor, but there is a language module available for Markdown and if I'm using this I use the very capable Marked to preview and output. Windows has similar tools available. Just don't use doesn't save as UTF-8.
For me, part of the beauty of using Markdown is that I can pick and choose tools for the particular writing task. If I'm writing something with lots of code snippets or code listings in it I use Textwrangler because its programmers' tools make jobs like that easy. If I'm writing a short note I use Drafts...The results are interchangeable.


Most of the tools we've talked about provide output to at least PDF and HTML. Some of them have many more tricks than that.
What we are going to talk about in the next post is a trivially simple way to output anything you like from Markdown. But before that...


Let's just be clear about one thing when you are using Markdown: Formatting doesn't matter...bear with me and don't start getting grumpy because...formatting doesn't matter. Here's why.
Word and other tools like it have fundamentally stuffed up the way we do documents. They have encouraged us to conflate two unrelated concepts: What a document element is; and How that document element should look. Those are two separate and unrelated concepts.
The only time that those two concepts come together is when we choose a particular presentation mode and a particular presentation tool. For example in Markdown this is a level one heading: #Heading One
Now obviously that's not how we want it to look. But how do we want it to look? What will it look like in a Word doc? What about on a web page? What about in that LaTeX document for our Uni assignment where it has to be in accordance with the APA style sheet? Each one of those will be different and can change as we modify the style for that element. The only thing that's important is that that element is a level one heading.
So having got that out of the way and having already talked about the way Markdown is human-readable...that means that we don't convert from Markdown to anything else until the end of our workflow. At least not until the end of our iPad workflow. We do all our revisions in Markdown, we share the document around in Markdown and when we are happy about it, then and only then, we transform it to whatever our desired output mode(s) are.
OK, I'm with you already: What about track changes you say. Well for me this is a beautiful thing. I hate Word's track changes functionality with a passion. It is kludgy and dense and after you've been modifying a document a few times it gets plain horrible. We don't do any of that with Markdown. We don't have track changes...and we don't need it. I just say to somebody who is reviewing a document for me "just make whatever changes you like". That's because I use Git to version all of my documents and I use Git's diff tools to tell me what's changed and I then decide to accept, reject or modify those proposed changes. Now most people don't use Git, but they can use the very beautiful Kaleidoscope tool or one of a number of other tools around to do the same job...better.
The only other thing to talk about is tables. These are a bit of a pain in Markdown. There is syntax for tables but I find them a bit painful to do manually. So you have three options: either you create your tables as objects (pictures) and embed them in Markdown using the image syntax; or you do them right at the end in Word; or if you can type and you can find the Terminal app in Mac or it's equivalent on other platforms you can use these tools. More on those and on output in general in a future post though.

Monday, February 11, 2013

Markdown to Word

So here's the use case:
You are increasingly working on your iPad at work. It means you don't have to transcribe hand written notes into some other document and you can write documents completely on your iPad. The problem is that the iPad and Microsoft Word are poor partners. The best way to write on the iPad is to use a Markdown editor. It's a simple markup language, anyone can learn it in 10 minutes and it allows the documents to be re-purposed to Word, HTML, LaTeX, ePub, PDF or whatever. And everybody in the business world wants something in MS Word.
The further problem is that whilst Markdown is the bees knees, it's not trivial to do tables in Markdown. In fact writing tables by hand in Markdown is a pain.
This is a real scenario that's been troubling a couple of people I know. So I've been working on a solution. Here's what the solution and tool chain look like:
  1.  iPad combined with a Bluetooth keyboard. If you are going to write on the iPad then do yourself a favour and get a Bluetooth keyboard so you can touch type.
  2. A Markdown editor. I like Writeup on the iPad but there are heaps of others.
  3. iSSH on the iPad to access the server via ssh.
  4. Dropbox. You have to get stuff on and off  your iPad and there's no better way than Dropbox.
  5. Pandoc on an accessible Mac or Linux or Windows server. More on this below.
  6. R statistical analysis package. Stay in your will be alright I promise.
  7. These scripts.
Here's the workflow.
  1. You create a Markdown document on your iPad and store it in Dropbox. Where you want a table in the document you put a marker of the form @filename.extension@ into the document on a line of its own.
  2. Create the table data in MS Excel and export it as tab delimited into a separate named file for each table. The filenames must align with your markers from the step above, but without the @ at either end. Any strings with spaces in them in the table need to be quoted: "Jane Smith". Put these files in the same Dropbox folder as the document with the markers in it.
  3. Use iSSH to generate and transfer a key to your server and set up the server to accept that key. That way you can create a session on the server without passwords. Set up iSSH to run a script on login that calls the script from step 7 above. This is a one time setup.
  4. Login using iSSH and choose and run through selecting your table data and the output file name once for each table.
  5. Now from choose and tell it which file has the markers in it and what you want the document called when it has the tables in it. This will replace the markers with the actual tables.
  6. Now from choose and it will process your document into a Word document.
If this sounds complex it's not. One of the people using this has never used the Terminal and never wants to but they happily generate Word docs with tables in them. It is very quick. The table insertion process takes seconds to run and the conversion to Word takes under a second to run.
The package of scripts has a detailed file with it and you can ask any questions on that site or in comments here. That link takes you to the overview page where you can read the documentation and learn about the very simple pre-requisites. From there you can either do a git clone if you are that way inclined or simply download the files as a zip.

Please give me feedback and suggestions in the comments.

Note 1: Dropbox doesn't update on a computer unless you are logged in. Therefore you need to have a user logged in to the console of the remote computer and that user has to at least have shared the Dropbox folders that you want to use. Otherwise you save a file to Dropbox on the iPad and when you ssh into the server it isn't in Dropbox.
Note 2: Setting up keys for ssh can take a moment but it's worth doing. If you have trouble say so in the comments and I'll write a post on it.

Wednesday, January 16, 2013


The short summary: Ranjini is a Sri Lankan Tamil refugee - she has been assessed as a refugee by Australia. She has also been assessed by Australia as a security risk. Because of these two interlocking assessments she can neither be returned to her home nor released into the Australian community. She, and her 3 children - a baby boy was born last night, are detained indefinitely. You can see more about Ranjini's story here.
The Australian legislation has ASIO making security assessments of asylum applicants. However neither the applicant nor anyone else (with some limits around a new review process) is allowed to see the assessment or know the reasons for it.
This is the crux of the matter. We have a government and a minister who have so badly botched the issue of asylum that the public no longer trust anything they say on the matter. Yet at the same time the public is expected to believe them when they, in a legislated framework, essentially say "trust us this woman is a risk to the community at large". This is the core issue of secrecy in government. It is also the reason that there must always, in a democracy, be a tension around secrecy.
I have not yet reached the point where I can say "there is never a reason for government secrecy'. What I do believe is that secrecy, in and of itself, is a danger to democracy. In the hands of the inept - and that's where I place this government - or the despotic, it is truly dangerous.
When we do not have trust in the sound administration of government why would we trust the executive when it says to us, in effect, "trust us this woman is such a danger to all of us that she must be locked up indefinitely without trial, without public scrutiny of the evidence and without capacity for appeal". That seems neither acceptable, reasonable, democratic nor humane.
So will Ranjini ever be deemed able to be released? Will successive governments continue to incarcerate her without trial or procedural fairness until she is so old and feeble as to be deemed a risk by nobody? What will happen to Ranjini? Will she ever be released? Will this or successive governments ever accept that they have an obligation to disclose to the public and to the affected person the matters that they say are enough to indefinitely detain them?
We release bombers, rapists, murderers and paedophiles into the community at the end of their sentence. Some at least of those people have been shown by research to be a high risk of re-offending. Yet we, rightly, say that they have served their time and we release them.
Ranjini has never been convicted of anything. Ranjini has never seen the evidence that purportedly shows her to be a risk, Ranjini has never seen the negative security assessment. No fairness, no natural justice and no scrutiny of the actions of secret government.
In the absence of the capacity to review, debate and scrutinise these matters in a competent court, we are left with only one way of viewing this: The decision to indefinitely detain Ranjini, and many others in her position, must simply represent complete paranoia on the part of the secret arms of the government and on the part of the executive. Why else would appropriate scrutiny of the assessment be barred? Why else would there be no basis for appeal?
We as a country should be scared for Ranjini and others like her. We should also be scared for the implications for legal fairness and democracy for the rest of us.
A brief addition to provide some more articles to help understanding of the situation Ranjini finds herself in.
Julian Burnside
Michael Gordon

Thursday, January 10, 2013

Being a Git!!

To paraphrase Linus Torvalds, he’s an egotistical bastard and so he always names his software projects after himself. Hence the name Git!
This post is not for coders, hackers or other nefarious creatures who write software for a living. This post is aimed at the common (wo)man in an ordinary old business.
Why would that sort of person be interested in Git? The Git that’s a distributed version control system (DVCS)? The Git that’s written for software development teams, like the one that builds and maintains the Linux kernel?
Let’s look at Git a little bit first (please don’t go away…I’m getting to the point). Git is:
  • A file system based version control system. That means it keeps its files in the file system of your computer, accessible to you with all the tools you might normally use to manipulate and work on those files.
  • A distributed system. That means that there doesn’t have to be a centralised repository although there can be. It also means that you can keep a copy of the repository on your local machine - desktop or laptop or both - and you can share it with other users without a central repository…or with a central repository or a number of central repositories.
  • A version control system. That means it keeps track of every change to every file that is added to Git for the life of the repository.
  • Capable of initialising and using remote repositories to enable repositories to be backed up and shared.
  • Lightweight, in the sense that it is trivial to install and manage, but deceptively powerful.
Imagine a scenario where you are an executive in a company, you travel a lot and you are currently working on a major rebranding of a product line. You are using an external graphic designer to create the logo and the “look” for the rebranding.
You want to be able to work on the rebranding project - all the documents and supporting materials - on the plane when you are travelling and in your hotel rooms. Meanwhile the designer is working on the logo, the general branding, the templates for the brochures and product information. She needs your feedback regularly. You’ve tried doing this by email and saving files to your local machine. However your boss is very picky, he wants to make sure this is exactly right. That means that you and the designer have been through multiple iterations and you are both losing track of the file names which are starting to look like logo_draft13_pete_v0.23.
This is the sort of situation where Git excels. It is trivially easy[1] to set up a Git project that saves you all the hassle. It would look something like the following picture.

Here’s how it works:
  • One of you would set up a local repository and, because of your specific needs, that person would also set up a remote repository[2]. It might be possible to operate in this scenario without a remote repository but it will be simpler with one.
  • That person would then add[3] and commit all their relevant files to their local repository.
  • That same person would then push their local repository to the remote repository.
  • Person two would now clone the remote repository to their local machine.
  • Finally person two would add and commit any files that they have that are relevant and that are not yet in the repository. They would then push their local repository to the remote repository. Person one would pull the remote repository. Now both people have all the files in their local repository and all the files are also in the remote repository.
Now the small fly in the ointment is that Git, by design, is a command line tool. Most of us can’t be bothered with tools like that. We are so used to GUI tools that we simply don’t want to work any other way. So we want a setup like this:

Every platform has various Git GUIs available, I use Sourcetree on the Mac but there are lots of choices.
As each person works, they work on the files in their local repository. They add new files to that repository and commit those files. At each commit Git takes a snapshot of the project and gives that snapshot a unique name. You can wind back to any commit at any time. It’s like endless versioning.
When either of you is ready they can push their local changes to the remote repository. This merges those changes into the remote repository. The other person, when next on line, is warned by their GUI tool that they are “behind” the remote repository so they can pull the remote repository to their local repository. That merges those changes in with their work and they can then push their changes back to the remote repository. Then the other person can pull…and so the process goes on[4].
Of course the next thing that happens is that your boss wants to see where the whole project is up to…well they can just clone the remote repository onto their machine and they have all the files to hand. The boss then thinks you need more help, the project’s growing, so he assigns two more members to the team. They just need access to the remote repository, they clone it and they’re ready to go.
This brief run through has barely scratched the surface of how Git can be useful to “non-coders”. For more information start with the following references.
Git homepage
A great Git book freely available on-line.
Github is a hosted Git service with both free and paid plans.
Git is also used as the “back end” for blogs and wikis and a whole range of other tasks. Google around and see what you find.
Oh, did I mention that Git is Free Open Source Software?

  1. Installing Git is a very simple and quick process, indeed if you have a Mac or a Linux machine it may come already installed, if not it’s the work of 30 minutes. Installing Gitolite (see other footnotes) is the work of maybe an hour, mostly getting keys set up. Anybody who can type in the console and can read instructions can do this from the very good directions available. Once installed setting up a repository takes less than 5 minutes.  ↩
  2. Remote in this sense means on a server accessible to the internet. Typically this repository would have its access controlled with something simple like Gitolite. Gitolite enforces extremely fine grained access control and requires all traffic to occur over SSH which is a secure link. Alternatively you could use a hosted service like Github.  ↩
  3. Text formatted like this denotes an actual Git command line command. To add a file called test.txt to Git the command would be entered like this: git add test.txt with add being the key word. You can do it quickly and simply from the command line, or more usually, you can use a GUI tool.  ↩
  4. Git can diff files between commits to show you what has changed. In general this capability is limited to text files of all sorts. Binary files such as PowerPoint, Word, Illustrator…can’t be diffed. Git knows that they have been changed, but can’t tell you visually what has been changed. Nevertheless Git keeps a version of every change that you have committed.  ↩

Thursday, January 3, 2013

The Unexpectedness of Loss

Googling around tonight, as no doubt we all do from time to time, I was hit with a sudden sense of loss. I found that a bloke I knew briefly, but well, had died just over 3 years ago.

Ray Lynskey was a young bloke in the Royal New Zealand Airforce in 1978 when I met him. Ray was also a glider pilot. I was younger by a few years, a first year Uni student, far from home and finding life tough.

My escape was to fly gliders. That's where I met Ray. After a long day flying from a freezing airstrip at Wigram in Canterbury, we would retire to the bar. From there it was too far, too late and too cold to hitch a ride back to Uni. Instead Ray would give me a bed in his immaculate but very plain Airforce house on the base.

Ray was a quiet, upright sort of bloke. He looked after people and was a quiet leader.

I moved on, left Uni and went back to the north of the country where it was at least warm. Ray moved on too. Later leaving the Airforce and becoming a commercial pilot. Along the way he became the World Gliding Champion in 1995 and becoming the first pilot to fly a glider 2,000 km non-stop in 1990.

Ray died in 2009, of an inoperable brain tumour, after a short illness. I didn't know that until tonight. It took me aback with a strong feeling of loss. Ray was a great bloke, quiet and kind. He supported me when I needed it and for that I will always be grateful.

Wednesday, January 2, 2013

The Distributed Social Web

It's 30 years since the Internet began with the roll out of TCP/IP. Earlier than that the basic design of the Internet was generated in the ARPANET. The Internet and its progenitors had as a key design goal a distributed and resilient infrastructure. Poor reliability of links and nodes meant that the network needed to continue to operate even if links or nodes went AWOL.
Those requirements remain and much of the infrastructure that runs on the Internet demonstrates similar design. Two good examples are email and the XMPP protocol, both of which use a federated model. My mail and XMPP servers don't have "hard wired" connections to other mail or XMPP servers. When I send mail or message someone on another network I don't create an explicit connection between my server and theirs. Instead the traffic is routed through servers which federate with mine as required. In the case of XMPP the federation is generated either when I log on to an XMPP account not on my server or someone on another remote server messages me. In the case of email it happens when I send or receive email. In either case there are often multiple routes available and the connections live only as long as needed.
It's a good system that requires little human interaction. I don't have to determine how to get my email to you. Instead I enter your address and the various components of the email system work out where you are and how to get to you. Ditto with XMPP. The more infrastructure nodes that are "out there" on the Internet, the better the system works.
Now to a change in the way the Internet works. I'm not talking about the underpinnings...rather I'm talking about what sits on top of them. We are seeing the rise of the internet monoliths. Google, Facebook, Twitter...the business model of these and other Internet monoliths requires that you go to their "monolithic" location (I know that they are certainly using distributed infrastructure, its the presentation I'm talking about).
The business model relies on monetising either your presence, your traffic or increasingly it seems, your data. At the heart of all this is your data. We've seen the "privacy" shuffling going on at Facebook for some time, we saw a recent Instragram furore about data (and no I don't buy the hysterics over that but it was nonetheless troubling). Facebook was recently reported to have 900 million members. Facebook in this post suggests it's 1,000 million people. Those are phenomenal numbers. If you are one of those people, then your data is deeply embedded in Facebook. How would you migrate to another platform? What do Facebook's terms of service say about ownership and copyright of your data? Do you know?
These are all pretty valid questions given that, in simplistic terms, you and Facebook have different goals. You want to create and maintain connections with your friends. Facebook wants to make money out of your presence on their site and your data on their site. Please note: I am not saying that Facebook wants to sell your (explicit) data. I do believe that it's clear however, that their game plan is to make money from the fact that your data is on their site. That's good and proper that they should want to make money. It's your choice about your involvement however.
Data is also explicit (your posts, your pictures...) and implicit (who you talk to, what you say you do, where you say you visit...). The implicit data is probably more important than the explicit...hence the issue of Facebook following users beyond Facebook that arose last year.
Given these divergent goals it's likely at some point that someone is going to be unhappy about the deal. More to the point it comes to some basic questions about what we want from the we want large monoliths that can arbitrarily decide how we use the internet and what, to a large extent happens to our data?
The problem is fundamentally that the infrastructure of the "social web", unlike the infrastructure of the older parts of the web, is these monoliths. It isn't distributed - you aren't choosing a "Facebook provider" you're choosing Facebook.We are putting a lot of our eggs in one basket with our use of the likes of Facebook and Twitter.
The other option that's emerging from these concerns are distributed social networks: you choose which provider you use, or you create your own, and then you federate with other providers so that you remain connected and can interact and collaborate with others. One good example of this is Diaspora which grew out of exactly the concerns I've expressed. It's an early stage project which is just on the verge of being generally available.
I'm hoping that the next wave of the internet sees the rise of the Distributed Social Web. That way we begin to have choices. We can continue to use the big monoliths or we can begin to take control of our presence and our data and connect with our friends, family and colleagues in new ways and with new capabilities.

Which Blogging Platform?

I’m looking for a new blogging platform without a lot of success. So, here are the criteria:
  • Must store posts in a user-accessible directory structure as text files. I’m not going to be stuck again unable to get posts out if I want to migrate somewhere else.
  • Must be able to use Markdown as the preferred markup language.
  • Must be self-hosted.
  • Technical requirements:
    • Any of PHP, Ruby (but I’m stuck at 1.8.7 on the server so no requirements for 1.9.x), Python…Mac OS 10.6.8 is the server version and we’re stuck there for various reasons for the foreseeable future
    • Simple to install - I don’t want to have to spend days installing and then have a heavy management load. So installation should be fully scripted or installer based
    • Git backend would be a big bonus. We’re running Git with Gitolite on the server. It’s magically easy to install and use and I’m a big fan. No hosted Git solutions though please…so no Github or Heroku
  • Should provide existing CSS, templates and blog structure. I don’t want to have to fight for days to set up a site that looks OK and has some basic structures such as menus
  • Preferably with its own on-board commenting system
  • Must be FOSS (Open Source) software
If you know of anything that fits the bill please talk about it in the comments or head over to Twitter and tell me about it…I’ll be eternally grateful.