One big file, or many small ones?
Table of Contents
It’s a very frequently asked question nowadays:
"Should I use one big Org file or many small ones?"
It’s become especially common since the release of Org-Roam (repository), which provides indexing and other tools that help when using the many-small-files paradigm.
**Bottom line:…
One big file, or many small ones?
Table of Contents
It’s a very frequently asked question nowadays:
"Should I use one big Org file or many small ones?"
It’s become especially common since the release of Org-Roam (repository), which provides indexing and other tools that help when using the many-small-files paradigm.
Bottom line: Use whatever combination of small and large files that works for you. It’s not uncommon for Org users to have relatively large files with tens or even hundreds of thousands of lines, accumulated over many years.
Considerations
Performance
Emacs
Although Emacs has historically had some performance issues with very large files, this has mainly been an issue of line length rather than file size. (As well, recent developments in Emacs have improved performance with long lines, such as so-long-mode.)
Org files generally do not suffer from such long lines, since they are intended to be read and written by humans.
Org
- File size
Org has historically performed well even with large files having many lines. However, with the use of overlays to fold outline trees, at some point, the performance of folding commands could eventually suffer.
Recent developments in Org have seen the change from overlays to text properties as the default folding backend, which has improved performance. As well, Emacs 29.1 gained an improved overlay implementation which greatly improves performance.
- Initialization
When using many files, a constant for each file is the time it takes to initialize org-mode in each Org file’s buffer. Once that initialization is complete, the price is paid, and operations are fast; but when performing an operation on many Org files that are not yet open, the initialization cost can add up, delaying the completion of the overall operation.
However, recent developments in Org have reduced per-buffer initialization time, and efforts in this regard continue.
Usability
Agenda
The Org Agenda works by searching files individually in Emacs. This requires each file searched to be open in Emacs. As mentioned, initialization time can be a factor when searching files that aren’t yet open, but this can be mitigated by, e.g. using various tools like desktop-save-mode to automatically open relevant buffers before they are needed.
Searching
Searching Org files from within Emacs (using tools like Org’s built-in agenda commands, as well as third-party tools like Org QL) requires the files to be open in Emacs, and therefore initialization time is a factor.
However, being ultimately plain text files, Org files can also be searched using external, line-based utilities, such as grep, ripgrep, etc. And since these do not require the files to be open in Emacs, initialization time is not a concern.
Configuration
Several Org-mode settings can be configured either globally in your init file or locally (per file or per outline tree). See the manual for more details. For an example of local customizations see this FAQ.
Organization
Organization is largely a personal matter. Emacs and Org are tools designed to help users develop workflows to meet their needs, whatever they may be. There’s virtually no limit to creativity in this regard.
A user may find that, for some use cases, one file may serve as a kind of "notebook" to collect all the information and tasks related to a topic. For others, a hierarchy of various directories and files may be more useful (e.g. Worg).
Methodology
Here are some ideas for organizing Org-mode files:
- One file for all your projects and notes.
- One file per project.
- One file per client.
- One file per area of responsibility or type of work (e.g., household, health, work, etc.).
- One file for projects, one for appointments, one for reference material, one for someday/maybe items, etc.
- A wiki of hyperlinked files that grows and adapts to meet your needs.
For more ideas, see some of the links on the org-tutorial index or this page on org-mode and GTD.
Resources
Articles
Karl Voit: My Current Org Mode Files and Heading StructureThis long article describes my current Org mode files, how they are named, what main headings they contain and some statistics. Karl Voit: The Right Way to Use Org ModeMany people wonder how Emacs Org mode is used in "the right way". Here is my short answer to that: There isn’t one right way for using Org mode features. If this is obvious to you, you can skip the rest of this article. If not, you really should read this until the end. It may answer many questions you probably do have in your head. Especially when you’re rather new with the Org mode universe.
Colophon
This question is asked very frequently on Reddit’s r/orgmode and other forums. This page is intended to become a comprehensive resource that can be shared when the question is asked. Please feel free to add relevant blog articles to the "Articles" section, and relevant discussions to the "Discussions" one. When adding content, please follow the patterns established in each section.