How Jira boards helped me organize my new book: Parallels between software development and creative writing
Hello, folks! In this short article I will be going through a realization that came to me when I was planning the writing of my new book. Quick intro: I am a Machine Learning Engineer but, in the free time, one of my favorite hobbies since a child is writing. I already have three e-books published through Amazon KDP and I will leave the link to them at the end (written in Portuguese, though).
Anyway, the goal of this piece is to show how we can draw parallels between software engineering/agile methods and the creative writing process. More specifically, I will try to show how the organization of my day-to-day tasks in a Jira board helped me translate this knowledge into the structure of the new book I am writing. First, I will go a bit into the terminology I am using at my current company (just because there are a lot of methods and terms around there for similar things); then, I’ll go through how my creative process works to finally wrap it up linking everything together. Let’s go!
Jira Terminology
I have used several agile methods and software throughout my career, but in my opinion they kinda serve the same purpose with minor differences and extra functionalities. Currently, I am working in a hybrid/custom form of Scrum, less strict than the original framework. We follow two-week sprints and have most of the rituals although with a different cadence and more flexibility, and we also don’t have a Scrum Master — instead, we share this responsibility. Regarding the organization of the tasks, we use a Jira board with the following breakdowns:
- Initiative: Big, long-term goal. My team is a platform team, so we mainly create frameworks to help Data Scientists and/or Data Analysts or fix/improve the current ML infrastructure. So a Initiative is something more generic and broad, that usually takes at least a semester and is very connected to the company/business unit’s key results;
- Epic: Inside each Initiative, we get a little more specific and use Epics to map tasks that should take about a quarter. Usually the epics are mapped at quarterly meetings and are not created frequently;
- Story: Then, inside each Epic we have the stories, which are the structure we most often create during the sprints. They should be relatively specific and well-defined, usually taking somewhere between 1 to 2 sprints;
- Subtask: The most granular structure. A subtask is associated with a story and maps a very specific task that needs to be done during the sprint. We are not super strict on creating boards with dozens of tasks, so each developer is free to create their subtasks as they see fit. However, we do control the stories.
Using examples, we could have as Initiatives something like “Improve the performance of the current models” or “Reduce the time-to-market of new models”. Inside the second Initiative, we could have Epics like “Improve the process of feature selection” or “Reduce training time”, for instance. For the last Epic, we could have stories like “Optimize the current data pipeline” or “Improve development environment”. Finally, for the subtasks we would have more specific activities related to the day-to-day development.
My Creative Process
I am still searching for the framework that works best for me, but until now all my books were created on a “freestyle” development based on dreams. Well, I am a person with very vivid dreams and that’s where I get most of my ideas from. Most of the times I don’t have to stand still thinking about the plot and next steps, but rather the opposite: the ideas come to me in my dreams and then I have to figure out a way to connect them all into a single story. I have a backlog of “dream ideas” (most of them are very raw) to write about someday that would translate to around 20 books.
As for the writing per se, I usually start writing without too much compromise (additional comment: I HATE and always hated hand-writing; my hands sweat and hurt, even I don’t understand my written characters sometimes and I often break the pen or pencil because I think my grip is too strong. So in this sense I am not old school at all, I’m all in on typing in a computer) and focus on the main parts of the story, re-writing and changing/cutting/adding parts after the book is technically finished for the first time (or even going back and forth as the ideas come by). Although this approach allows me to get to the “exciting” parts (i.e. the climax and the end) faster, I often have to rewrite a lot of stuff and go through the whole thing multiple times. Also, I noticed that I always have a hard time to start the books, since the ideas and characters are not so well-built and I have to kinda fast forward to the most crucial parts and then reconstruct backwards.
As I got more experienced both in life and in my professional career, I realized that I might be relying too much on my “crazy free-will”, dream-driven development. So, for the book I am currently writing, I chose to try something different.
Current Book Planning
First of all, I think there is no right or wrong way of writing a book; also, there is no silver bullet, so I really believe everyone should experiment and learn what works best for them.
That being said, I was struggling in this new book using my old ways because my dreams were kinda disconnected this time: I had the main idea, the ending and the protagonist crystal clear, but the other characters and the middle portion of the book were just not coming. I could start writing it and hope it would come eventually as it did the other times, but I got really scared that the start and end might become disconnected and the characters don’t receive the proper development — I mean, in a way that would be beyond repair once I finished for the first time. TLDR: Doing the iterative approach seemed like a very risky proposition with a lot of effort involved.
After some initial struggling I came to the realization that maybe I could draw a parallel between the way I was developing software and breaking down my tasks, and the way I need to develop my book and break down each of its parts. It is not an exact analogy, but it helped me a lot to organize my thoughts and kickoff the writing.
The whole book is the single Initiative, so we go straight to the Epics. In my mind, each Epic is a very important fact that I am certain must happen sometime during the book. I will once again use Star Wars: Revenge of the Sith as an example. Epics could be: Anakin discovers Palpatine is Darth Sidious; Palpatine kills Mace Windu and turns Anakin to Darth Vader; Order 66 is executed, bringing chaos everywhere; Anakin and Obi-Wan have an epic duel.
Generally speaking, Epics should resume the biggest moments of your book (important deaths, conflicts, turning points, change of scenery, plot twists, etc) and build the spinal cord that binds the beginning and the end.
Then, the Stories should try to cover the ground between each one of the Epics, serving as a “glue” that gives the story meaning and allows for character and plot development. The way I think, each Story is a Chapter in the book, but I didn’t have all the chapters set in stone before starting. After all, even in software development we have changing requirements, unforeseen situations and re-prioritization that makes us leave some buffers to accommodate and change plans in the middle of the course. I think this is no different here, as new ideas come up when you are writing, even though they are not (usually) major plot-altering ones.
One important caveat that I must emphasize is that you must plan each one of your character’s timelines as well. What I mean is: you should try to connect the Epics and Stories to where each major character is at each moment, so you can have a better feel for how you should develop them so their trajectory makes sense at each point of the story. There is no clear-cut answer here as to the extent to which you must detail this, but at least for me I did it for the protagonist and the main supporting characters, detailing their main characteristics, desires, conflicts and shortcomings and how those are affected at the most important parts of the plot. Of course the definition of “major” character and the amount you need varies greatly from book to book (one that immediately comes to mind is Lathe of Heaven from the brilliant Ursula K. Le Guin, which only has three characters and is still a masterpiece), but the bottom line is that you must have a very clear definition of this before starting to write.
And finally, even though I am a fan of creating at least a few Substasks per Story, that is not the case for my book development strategy. I think this is because I still rely on my dreams to some extent as the main source for creativity, and also because I think I work better figuring out the “small details” on-the-fly. However, having the main parts and the connection between them well-mapped before starting allowed me to start and finish the first chapter much faster and with much more fluidity than before. Also, I have a lot more confidence that the story will make sense and the start will connect with the end, leading to less refactoring. I spent more time planning and proactively thinking about each chapter and that “delayed” the start of writing, but so far it’s paying off.
Of course, there is a tradeoff between planning and execution (that is the same in software development): Too much planning, and you are too late to start developing and miss out on quick wins; freestyle development might get the quick wins vary fast, but what usually happens is that if you don’t have the minimal planning you will end up with a lot of refactoring and technical debt (in Brazil we have a saying for this: “barato que sai caro”, which is something like “a cheap thing that ends up being expensive”).
And that’s it! By no means I wanted to make it sound that this is the “correct framework” for creative writing — in fact, it might not even be the one I settle with, as I am still young and figuring out the sweet spot; I just wanted to write this story to show how knowledge is much more generalizable than it is often thought, and there are many opportunities right in front of our faces to transfer and apply it to other domains.
Before we finish, since you already came all the way here, here is the link to my e-books in Amazon, if you want to take a look on it. Thank you and happy writing ;)
