Outlining in OPML in Linux

Posted on Mon 10 August 2015 in Tutorials • 8 min read

A couple of posts ago, I wrote about how I've moved entirely to a markdown-based, Dropbox centric workflow. To clarify, I've used markdown for a quite a while with new projects but relied on Scrivener for legacy stuff. Even with new projects I was using Json, CSV and I even experimented with OmniOutliner for outlining. Since then I've decided that I have all the tools at my disposal to migrate those legacy projects including my current project, the redraft of Weaver of Dreams.

In the last post, I noted that I had created a purely markdown-based outlining template based on Matt Gemmell's excellent work with OmniOutliner. Since then I've created a workflow that takes that outline and turns it into the files and folders that where I'll actually write the novel.

MultiMarkdown

The key ingredient is MultiMarkdown, Fletcher Penny's extension to the original markdown specification. Not only is MultiMarkdown a dialect of Markdown, it's also a compiler and critically for me:

  • It's free software
  • The compiler runs on Mac OS X and Linux
  • The compiler supports CriticMarkup
  • The compiler supports export to OPML
  • There's support for metadata, transclusions and other things not in John Gruber's original markdown spec

OPML

I never really paid much attention to OPML until I started playing with OmniOutliner a few weeks ago. OPML is an open XML-based standard for describing outlines and is a well supported medium of exchange between many popular applications. The MultiMarkdown compiler can convert a markdown file into OPML while still respecting the heading-based hierarchy.

OPML is just XML, that means that not only is it a great format for exchange, it's also great for processing using scripting languages and that piqued my interest. Instead of relying on a proprietary application to import, process and export my files, I can write my own script that works the way I want.

Big deal, why should I care?

Well, you shouldn't. If you are happy using proprietary software and formats, then great. If you are happy having someone else determine your workflow, then that's great too. You probably spend more time writing as a result than I do!

I have no conceptual problem with proprietary software. However, I do have a problem with being beholden to others. In the past I've bought apps that are effectively dead or haven't changed in years including VoodooPad for the Mac and Fractal Mapper on Windows, both of which were important world building tools for me.

I do not like proprietary file formats. I do not like watching good apps become bloated with features I do not use or waiting for the developer to introduce something that I really do need or support one of the platforms I use.

I do not like being told how to work; no one workflow or way of doing things will suit everyone. That is true of me and that is true of everyone. My point is not to tell you to use my workflow, rather it's to encourage you to develop your own.

So I care because my writing is too important to me to trust to a closed format that can change or needs a proprietary application to open. I care because, my workflow is only as strong as the applications I use and if those developers fail or lose interest then my productivity can suffer.

How I work

I spent enough time with word processors in my formative years as a writer that I write rather linealy. However, Scrivener and my time as a technical writer have taught me the value of breaking a large work into smaller units. Even though I like to draft from the beginning and even though I still like to organise my project into chapters; dividing a large project into small pieces is best practice.

So with that in mind, my outline template looks like this:

# Title of Project
## Prologue
### Scene
* Notes
* Notes
## Chapter
### Scene
* Notes
* Notes
### Scene
* Notes
* Notes
## Epilogue
### Scene
* Notes
* Notes

That's great for creating an outline, but when I go to write the book, I want to do so in a hierarchical structure of files and directories on my computer's filesystem. For example:

- Title of Project/
    - title of project.md
    - Prologue/
        - Prologue.md
        - scene 1.md
    - Chapter 1/
        - chapter 1.md
        - scene 1.1.md
        - scene 1.2.md
    - Chapter 2/
        - chapter 2.md
        - scene 2.1.md
        - scene 2.2.md
    - Epilogue/
        - epilogue.md
        - scene 1.md

Nothing radical here in my thinking; instead of using somebody else's filesystem and file format, I'm using the one that ships with the operating system and plain text.

So the question is, how to turn my markdown outline, into my project structure? The answer is by processing OPML.

The OPML version of my template, converted with MultiMarkdown, looks like this:

<?xml version="1.0" encoding="utf-8"?>
<opml version="1.0">
<body>
<outline text="Title of Project" _note="">
    <outline text="Prologue" _note="">
        <outline text="Scene" _note="* Notes&#10;* Notes&#10;"></outline>
    </outline>
    <outline text="Chapter" _note="">
        <outline text="Scene" _note="* Notes&#10;* Notes&#10;"></outline>
        <outline text="Scene" _note="* Notes&#10;* Notes&#10;"></outline>
    </outline>
    <outline text="Epilogue" _note="">
        <outline text="Scene" _note="* Notes&#10;* Notes&#10;"></outline>
    </outline>
</outline>
</body>
</opml>

You'll note that the document is hierarchical with each markdown heading converted to an node. Under the Title of Project element we have the prologue, chapter and epilogue and within those we have their respective scenes. The _note attribute actually contains all non-header text and this can store anything from notes to the actual content of the scene.

Now, I grant you that this XML isn't exactly writing friendly, but the good news is that I only use it for processing and I don't directly edit the file. In fact I only displayed it here to show you what you can expect.

The actual processing is done by a script I created to run on the command line. Its job is to take the outline.xml file and turn it into my project, that is the nested files and folders. But that's really just the beginning as the script also generates content, chapter and scene numbers and crucially, the transclusion tags needed so that MultiMarkdown can compile the project to a single HTML or OPML file. The folders also get appropriately numbered to keep them in human-readable order.

So what the script produces is:

- Title of Project/
    - title of project.md
    - 00 Prologue/
        - Prologue.md
        - scene 0.1.md
    - Chapter 001/
        - chapter 1.md
        - scene 1.1.md
        - scene 1.2.md
    - Chapter 002/
        - chapter 2.md
        - scene 2.1.md
        - scene 2.2.md
    - Epilogue/
        - epilogue.md
        - scene 3.1.md

The script does not number the prologue or epilogue by design, however I add the 00 prefix to the name so that it appears before the chapters. Since Epilogue comes after Chapter, I don't need to number it.

All my writing gets done at the scene level. title of project.md only contains the project metadata (title, author, stylesheet, etc) and links to each chapter file. Likewise chapter x.md only contains the title and links to each scene.

While I'm talking about the structure and writing I should probably note here what I use for actually working on the project.

To work at the project level, I typically use Sublime Text, which is actually a text editor with some very powerful processing features. Like Scrivener, it provides a binder (the sidebar), split screens, spell checking, a distraction free mode and compiler (build system). However unlike Scrivener it has:

  • A very rich plugin architecture and hundreds of freely available plugins
  • Multi-cursor selectors
  • Better markdown support
  • Tabbed editing
  • Python console

Sublime Text is proprietary but I can live with that because it is just a text editor; there's no lock in. I could also use Gedit/Pluma or even emacs if I wanted. On the Mac I sometimes use MultiMardown Composer, which again is proprietary but again there's no lock-in and I wanted to support Fletcher Penny, who develops the app and MultiMarkdown itself.

I also like distraction free writing in something like Pyroom on the 11 inch screen of my laptop. Pyroom is so bareboned, it doesn't have a spell checker or even any UI elements that a mouse can click. Keyboard-driven, green-on-black bliss.

Hang on, mate, what happens when you want to move scenes?

Good question, and you're probably thinking with no small degree of smugness that Scrivener and similar applications, let you drag and drop elements of the project around willy-nilly.

I will concede that this system doesn't give you that ease of use. However, there's nothing to stop you moving and renaming scenes or even chapters and updating their link reference in the chapter x.md or the title of project.md file. That's fine for simple edits but what if you want to do a major restructure?

Well, that's where our friend OPML comes back into the picture. You can export your whole project structure back to a single OPML file, make your changes using a graphical program and then rebuild the structure.

Alternatively, I could build a basic drag-n-drop web UI if I really wanted to but in truth, I really don't need the feature myself. Why?

Well, I'm of the opinion that's it's better to get your story's structure right before wasting too much on the first draft. I've learnt this the hard way and I just don't have the time in my life to be a productive pantser. That's why I like the snowflake method and in fact my outlining template and processes is really just my take on Step 8 but where Randy suggests using a spread sheet, I prefer an outline.

Scrivener's great strength of not imposing a design structure is also its greatest weakness. It leaves the onus on the writer and for many people, especially those new to the craft, that kind of freedom I think can drop you in a hole that's tough to climb out of.

What about comments, notes and tags?

MultiMarkdown ships with CriticMarkup built in meaning you can mark up content to be highlighted, deleted or included. These will pass through the conversion from the outline to OPML and into the project structure nicely so I can tag scenes with things like {==POV Character Name==} and it will be there waiting for me in x.x Scene.md.

MultiMarkdown also supports metadata tags in the document header using basic key-value pair syntax. In my example of characters I could simply include the following at the top of each scene:

pov: Character name
characters: list,of,characters

And why not add things like location and a description too while you are at it. You can extract metadata with the multimarkdown compiler which is a nice bonus if you don't want to write your own parser. By using Sublime Text's Goto Symbol in Project command, you could list every scene by POV character with just a key key strokes.

As for things like file-level tagging and folder colours, most operating systems worth talking about actually support that at the file system level either natively or using a plugin.

But I like the corkboard?

Good for you, I never found it useful myself. There's plenty of corkboard clones out there that will import OPML. Pick your preferred option and add to your workflow.

What about compiling?

Markdown is an ideal candidate for processing into formats for exchange or publishing. The compiler on Scrivener is one of the poorer features of the program; by trying to do everything for everyone, it's become too complex and bloated. It's also pretty slow.

Instead, I've created my own utility that processes markdown into HTML and PDF using Multimarkdown and WKHtmltopdf. It's faster than Scrivener's compiler and it works with any markdown file on my computer. The output is styled using CSS and I can swap out or share stylesheets anyway I want.

Share on: Diaspora*TwitterFacebookGoogle+Email