My research and writing workflow

This is a work in progress and might be heavily expanded upon in the near or far future.

I’m pretty happy with my research and writing setup and I always recommend people to do this or that. So for once, I try to write-up my current workflow and the tools involved. There is only simple rule in place, that pretty much defines what tools I’m using: Own your data!

Data that you are producing needs to be valued as work in the current capitalist environment. So if you didn’t get paid for it, you still (have to) own it. It also makes it easier to shift tools and setups if you own your data. Owning data means being freely able to read, move and copy, transform, or delete it.

OK, on to an overview of my tools and their basic functions. The basic functions also outline the general workflow.

Tool and workflow outline

In research, you always collect, curate, and consume material such as scientific papers, other textual material, images and videos etc. Guided by your inquiry, research question or project, you produce something new out of the collected and worked on material and your own thoughts. In the last stage, you want to hand your new thing over to other people.

The two main tools in my setup are Obsidian and Zotero. Both are free, but only Zotero is open-source. Both offer paid services that you generally don’t need, except if you want to make your life easier or become a power user. Obsidian’s raw material are markdown files, which are basically glorified plain text files that you have on your own computer. Zotero works with a database hosted somewhere else, but has superior export abilities. are.na is an image-collection service as a platform. You can use something else here, I just happen to like are.na a lot. hypothes.is is also a service, but free and the platform’s code is open-source and has a vibrant community, as well as some great export functionalities. pandoc is open-source and a bit nerdy, and just transforms documents. So, all in all, most of these tools meet my initial criteria of enabling you to own your data.

Let us go over the setups of each tool and then some further setup specialities to glue everything together.

Firefox

With Firefox, you’ll surf the web and visit websites, platforms with scientific papers, podcasts and video-sites and what not. You can, of course, use any other browser, but why would you do?

Zotero Documentation: https://www.zotero.org/support/

In Zotero you collect references to things you want to keep, read, work through, cite in your texts, etc. The references can either just point to another place, like a bookmark, but they can also contain attachments. These attachments are usually either a locally saved snapshot of a website, in case it ever goes offline, but you still need the content, or in most cases, a PDF of the text you want to read.

When you do academic writing, it is best practice to reference your sources that you built upon. As is typical with an academic process, this can be tedious and painful. Zotero tries to take the pain out of it. You can have a folder with all your sources, for example, and with a few clicks have a proper formatted list of sources that you can paste into your document.

Zotero is also a wonderful tool to read and annotate PDFs. Have a look at the PDF Reader and Note Editor guide. It syncs with mobile, so you could also read on your phone or tablet.

Obsidian

Documentation: https://help.obsidian.html/Obsidian/Index

This is where the magic happens. Alternatively, you could have a look at Zettlr. Compared to Obsidian, it is completely open-source, and the developer is super sympathetic. But it’s too sluggish for me and the plugin ecosystem is practically non-existent, making it hard to extend. Obsidian is basically a text-editor with superpowers. One of the basic ideas is, that you can write in something and easily link to another file, over time building up a strong knowledge network.

I use the following plugins for writing:

All these plugins are installable through the built-in plugin-manager. You see that the goal is actually to get as much material as possible into Obsidian. This is where you spent most of your productive time. Going over your notes and annotations, your resources, piecing them all together and at last, come up with your own material.

How exactly you work in Obsidian is up to you and a whole other rabbit whole I fell in several times…

pandoc

Just install this one and forget about it. Installation guide can be found here: https://pandoc.org/installing.html.

Further setup

The last step is to make Obsidian be able to talk to Zotero (for the references) and to pandoc (for the nice exports) because that’s the shitty part of academic writing.

First, you need to export a Zotero collection or your whole library (not recommended if you have many items).

The settings you should have ticked are:

Remember where you’ve saved that export! You’ll need it in Obsidian. Go to the plugin-setting of the Citations plugin and adjust them.

Use the path to the Zotero BibTeX export file and put it into Citation database path. You’re free in using whatever you want for Literature note folder. That is just the place where the notes end up that you import from Zotero.

As a last step, go into the pandoc plugin settings and put the following preferences:

--bibliography=/Users/adrian/path/to/Export.bib --citeproc --csl=/Users/adrian/path/to/Citation-Style.csl

If you don’t know what a citation style is yet, just leave out the --csl=/Users/adrian/path/to/Citation-Style.csl part. This little piece of information will let pandoc know, that you used references in your text, and tries to replace them accordingly during export.

How to

Use references in Obsidian

Import notes and annotation


        ---
        title: {{title}}
        authors: {{authorString}}
        year: {{year}}
        ---

        {{note}}

And now?

It sounds like a lot, but you basically just do it once and then forget about it more or less. And it really really makes life easier. Or so I tend to believe.