One Directory to Rule Them All

Posted on Tue 08 April 2014 in Articles • 8 min read

There's something something appealing about using your computer's file system to manage projects. In fact it has a lot going for it. I'll list the obvious below:

  • It's free (as in beer and in speech)
  • It works well with your operating system
  • It's transparent
  • It's portable

This will be obvious to anyone who's a power user of their preferred computing platform but for those who aren't I'll quickly outline what I mean.

Freedom: When you use a third-party application to create or manage your content odds on it will be a proprietary system that locks you into a particular way of working and a particular (often proprietary) filing format. Using plain text files and a directory structure won't cost you a cent and won't lock you in to a particular vendor.

Operating System Support: Computers have been managing files and folders long before the first GUIs and Desktop Environments became common place. They're pretty good at it and most modern operating systems (and many older ones) have enhanced file management features such as the ability to:

  • Tag content with metadata
  • Perform searches with boolean operators
  • Preview files without opening them (this seems to be unique to OS X)
  • Sort files according to different criteria or metadata tags
  • Built in scripting and automation capabilities

Portability: Plain text files (.txt, .md, .csv, .html, .xml etc) can be viewed and edited in any application or operating system that support good old fashioned ASCII text. ASCII text is so ubiquitous that I reckon that every PC made after 1980 can support it. Plain text files are lightweight, human readable and can be marked up to make them structurally meaningful and/or presentational. They are searchable and easy to manipulated as part of an automated workflow.

Files and folders are easy to copy, backup and move. If I want to carry this around on a USB stick and edit documents in PortableApps, I can. If I want to use a GIT server I can. DropBox, Google Docs, OwnCloud syncing between my desktop, laptop, phone or tablet - no problem!

Why Not a Database, Wiki or Application X?

I've wrestled with this question for a number of years now and issue came up when I recently, and briefly, considered building this as a web app. A relational database system is a powerful tool for organising and managing large amounts of data. I've decided not to use one however for several reasons I'll list here:

  • Databases lock you into a particular taxonomy or schema which must be designed before you create content
  • Using SQL to CRUD data is tedious at best
  • Databases need a database engine or app to run

Since I'm 'drafting' this app at the same time as I'm developing content, I need a system that is flexible enough to change as I go. Using a database would mean doing all the design work upfront and designing the taxonomy (including all the tables, relationships etc) for something as potentially complicated as a fantasy or sci-fi setting could take years.

Setting aside databases, the next obvious option is a wiki. World-building is by it's nature an encyclopaedic endeavour and wikis, as illustrated by Wikipedia, are outstanding for writing encyclopaedias. In short a wiki is actually a pretty good choice for world-building and I've used both desktop and web-based wikis for this purpose. At first I used MediaWiki but then I migrated to VoodooPad.

MediaWiki, like any web platform is great for a collaborative project and it's used to power the Il Bethidad project. Alas I work alone so many of the benefits of a web app do not overcome the resulting shortfalls and risks of taking your content online; besides when I used MediaWiki it just felt like overkill.

VoodooPad, a personal desktop wiki for the Mac, is an outstanding program. It's very stable, feature rich, flexible, scriptable and has good export features. However, when I used it for world-building I started to run into limitations. Some of those have been resolved but many have not. Despite it's good export features, it is still a proprietary application and one with an unpredictable future (it's original developer sold it to another company).

VoodooPad is also only available on the Mac and I've made it my goal to only use formats that can be read natively on all platforms and there's no better format for this than plain text. For that reason I similarly discounted Microsoft Office, Filemaker Pro, and many other applications even if they have good export capabilities to open formats.

Directory Structure

My world building app begins it's life as a structured systems of files and folders that I'll automate with scripts.

/world
    +app
    +tools
    -stories
        -story 1
            +content
            +plan
            +output
        +story 2
    -characters
        +character 1
        +character 2
        +character 3
    -countries
        +country 1
        +country 2
    +organisations
    +religions
    +articles       

world: I've called this folder world but it would be more accurate to call it setting because I'll name it after the setting. Most of my stories occur on a single world and therefore the name could be same as the world.

However, if I was writing stories set on multiple worlds (but occupying the same narrative domain) then I'd include a top-level world/planet directory too (or even Star System if you prefer). Multi-world settings are most common in sci-fi stories but they do occur in fantasy epics too, most notably Raymond E. Feist's Riftwar Cycle. One could also argue too that settings with different dimensions could be treated as multiple world settings.

The point is, this system is flexible and will allow for that; creating a database or XML schema to cope with multiple hierarchical setting layers is simply not something I want to waste my time with.

app: This folder will eventually contain the application itself once I decide to move on from simple scripts. Depending on how I choose to develop the app, this folder may sit outside the world folder so I can use the same app to develop multiple settings from the same application source code.

tools: This folder will contain the scripts that will form the basis of my workflow and the 'first draft' of my application. This will give me the opportunity to create my content and my app ideas at the same time without getting bogged down in code development.

stories: This folder will contain all the stories that are set in the world I create; each folder will contain everything I need to plan, write and publish a single story. Note that in the case of multi-volume stories I'll still have one folder represent one story however fee free to create a top-level folder for trilogies or other related works.

characters: Each character in the world will be stored here. I'll talk about the sort of information I'll include (and how I'll cross-reference it to stories) in a later post.

countries: This folder will contain every country in the world stored in a dedicated sub-folder. This will likely be one of the largest and most complicated folders because of:

  1. the amount of countries and
  2. the amount of information I'll document (cities, towns, histories, timelines, geography, economy etc).

Countries will be heavily scripted to break down and standardise the complexity of creating settlements and economics. I'll document this at length in later posts.

Note that I'm using country as a catch all here. Empires are technically collections of countries/states and can contain multiple administrative or sovereign regions with varying levels of autonomy (the Roman Empire is a great example of this). Again the aim of this structure is to be flexible, I'll treat all sovereign countries as equals be they empires, kingdoms, city-states or republics but you are free to do as you please.

organisations: This folder will contain every organisation in the world (that I choose to document). Organisations are like people and they can exist across national borders so they get their own folder. I will document this in a later post.

religions: This folder will contain religions in the world; religions can occur across country-borders hence they get their own folder. I will document this in a later post.

articles: This folder will contain general articles about the world that don't fit easily into one of the main folders above. This could include subjects like:

  • Technology levels and articles about specific technology
  • The nature and structure of magic
  • Flora and fauna
  • Races (human and non-human)
  • Geography, geology and climate (at global or regional levels)
  • Social, intellectual or artistic movements (like Humanism, the Renaissance or the Enlightenment)
  • Major world events affecting multiple countries or regions (i.e. World War 2) and probably a timeline too

Depending on how many articles I end up writing, these will probably be grouped by subject into subfolders.

Scripts

I noted above that the earliest iteration of my world building app will be powered by scripts. A script is a automation utility written in a scripting language. Scripting languages are relatively easy to learn and run (when compared to a compiled language like C or C++) and most operating systems support at least one scripting language out of the box. Following is a list of the most popular operating systems and the scripting languages installed by default::

  • Microsoft Windows (VB Script, Visual Basic for Applications, Shell scripting with com.exe)
  • Mac OS X (Python, PHP, Perl, Ruby, AppleScript, Shell scripting with BASH)
  • Ubuntu (Python, Perl, Shell Scripting with BASH)

Note that these are just installed by default; you can install any scripting language interpreter that has an installer for your operating system.

Scripts run in an interpreter and that interpreter must be installed for the script to work. Most scripts run in the command line but many can also run in GUI applications or in web servers.

My scripting language of choice is PHP however much of what I do will be easy to adapt into other languages.

At first I'll keep the scripts basic (no classes or dependencies just procedural goodness) with each script essentially performing a single task. This means I can concentrate on getting the workflow right before I refactor the code into a coherent application. For example, I'll have a world.php, character.php and country.php file that will generate worlds, characters and countries respectively.

Once I get the workflow running how I want, I'll refractor the code into an application class and use composer.phar to manage any dependencies I need (i.e. markdown, symphony console etc).

Windows Users

I'm a *nix guy and do most of my creative work on Mac OS X; I also have a great fondness for various GNU/Linux distributions. To show you how little you need in terms of computing power, I may even develop this app on a Raspberry Pi running Raspian (a variant of Debian optimised for the little $35 computer).

If you are a Windows user and want to follow along to the letter, don't despair because you can install PHP and even a Linux command line emulator and package manager such as Cygwin. Otherwise, you are free to modify, hack and adapt what I do to run in Windows using VBScript.

Share on: Diaspora*TwitterFacebookGoogle+Email