Introduction

About a month ago, I embarked on a complete rewrite of the glu documentation (here is the result). The first iteration of the documentation was created using the wiki hosted on github, alongside the source code.

I was not very happy with this solution for several reasons:

• I knew that writing the documentation was going to be a pretty big effort and I did not want to have the wiki broken as I was writing it
  
  You can argue that the wiki on github is managed, under the cover, as a git repository and that you can check it out locally, even create branches, and push it all at once. But this is where it falls apart:

  • the wiki is not wysiwyg (far from it), and since github uses its own processing pipeline (which to my knowledge is not available as a local command line), there is no way to actually see what you get until you push it live (how would you see a branch for example ?)
  • github seems to be using different pipelines depending on where you are on the website (for example the rendering of a gist is different from the exact same input in the wiki given a file with the same extension!)

Choosing a different solution

I settled for using a markup based solution like what I have done for this blog. The main reasons are:

• it is easy to write
• it can be checked in with the rest of the source code
• it can have its own build process (treat the documentation as code!)

Choosing a solution was not easy as there seems to be a lot of choices. jekyll was actually not a practical solution because it is really well suited for blogs, but not really for heavy documentation with cross references.

I ended but choosing sphinx. sphinx was originally created to write the new python documentation. The sphinx documentation is written with sphinx and it is beautiful (as they mention it themselves) and very comprehensive. It has a lot of bells and whistles like:

• changing the theme
• generating different outputs (html, pdf, man pages, etc…)
• setting up the project for you (Makefile, configuration file, directory structure, etc…)
• generating a self contained web site
• …

In the end I am really happy with the choice as it has been fairly easy to concentrate on the documentation itself rather than the tool. I am also very happy with the final result: it is using the agogo theme with some customizations (which are fairly easy to do).

Integrating sphinx in the build process

glu uses gradle for its build framework and in order to integrate the documentation generation as part of the build, I simply treated the documentation as another project in the source tree. You can check it out on github.

sphinx generates a Makefile as well as a configuration file (conf.py) for you. I simply had to create a gradle task to do the following:

1. copy/inject the version (coming from the build)
2. copy the release notes in the sphinx directory structure
3. invoke the Makefile

This is what it looks like (you can view the full build file)

sphinxDir = new File(buildDir, "sphinx")
htmlDocDir = new File(sphinxDir, "build/html")

task doc << {
  // copy/replace @version@
  copy {
    from("src/main/sphinx") {
      filter(tokens: ["version": version], ReplaceTokens)
    }
    into sphinxDir
    exclude "**/*.png"
    exclude "**/*.gif"
  }
  // copy images (no replacement)
  copy {
    from("src/main/sphinx")
    into sphinxDir
    include "**/*.png"
    include "**/*.gif"
  }
  // copy the release notes at the root of the tree
  copy {
    from new File(rootDir, "RELEASE.rst")
    into new File(sphinxDir, "source")
  }
  // execute the makefile
  ant.exec(executable: "make", dir: sphinxDir, failonerror: true) {
    arg(line: "html")
  }
}

Finally, the documentation gets copied in the final product: in the end it is a static web site with a self contained set of html pages, css and images.

To generate the tables, I used the table-capture mode from emacs :)

Conclusion

Treating documentation as code has a lot of advantages:

• it can be checked in with the rest of the source code thus sharing the same lifecycle, branching strategy, etc…
• using a markup language (text based) allows for easy source control management of the documentation (you can do diffs, and whatever else you can on regular source code)
• it can have its own build process allowing you to inject information from the build itself (in this case I am injecting only the build version, but it opens up the door to a lot more (like test results, etc…))
• you can build any kind of automation around it

Note about github

I may seem to be ranting about github, but really I love what they have done. It is an amazing idea and if you are a software developer, you should have an account there! From my point of view, github is for developers what a portfolio is for artists!