Introduction to Denote (simple note-taking)

This looks like a wonderful tool, and I'm curious to try it! Do you do any note taking related to reading papers or other sources, and try to link your notes to that source? I've spent some time trying to integrate org-roam with Zotero, and while I've more-or-less got it working, I'm still fighting with it a bit. I'm curious as to how you use Denote to link your notes with materials that you've read (if you do).

I like your ideas about simplicity of note taking, the naming sceme for a note and being dependent on just basic technology. These are all good ideas. But I don't understand why are you doing one file per note? It seems like very inneficient way of using your long term storage. Depending on what your basic allocation size is (4kb often), it means a lots of disk is wasted for very short notes. I think you could achieve very similar overview with very low technology if you just put all your notes in a single org note file. Can't you just format your titles appropriately and add your custom software for backlinks, denote lonks and what you like? Org files are just plain text files, so it is still a sufficiently low tech. Admittedly you would need maybe a simple grep-based one-liner to display nice listing off notes in a command line, and graphical file explorer (dolphin, nautilius etc) wouldn't play nicely with it, but that would be the only downside I can see. Anyway, some cool ideas you present; nice to see a new video from you. Cheers.

He explains this in the manual. Large files are harder to use with existing tools, such as terminal search tools iirc. I dont think most users are going to care about the storage impact of a few kb per file of using one file per note.

I love this package. Thanks for all you contribute to Emacs! While I think this should work OOTB with [consult-notes](https://github.com/mclear-tools/consult-notes) please let me know if there is any further functionality I could add that would be helpful.

Very nice! I loved org mode initially but could not use it very effectively as it was a pain to create new files every time. Started using org-roam with templates and my workflow changed completely. I think your package has a few advantages over roam (the view and hierarchical search). I have been using Deft for searching. I would recommend some sort of capture templates because those have been extremely useful for me.

There’s info about how to set up denote capture templates in the README.

> Large files are harder to use with existing tools, such as terminal search tools iirc. In which way are large files harder to use with search tools? In general, in case of many files, there is an overhead of file I/O per file. Every time a new note is created, the OS will have to make (unnecessary) check to see if the file exists or not, which will be slower and slower over time as the number of files grows, while one file means just a single file access, and appending text at the end of the file. And it is not just saving, it is also searching through because each file has to be accessed, fetched and opened, instead of just one file opened once by Emacs. People usually use indexes to speed up things when there are many files. The number of files on a file system is implementation dependent, there might be issues with encoding of some characters one maybe would like to use in a note (file) name on some file systems. I.e, when sharing or transferring to other file systems, names might not be portable, there might be limit in number of characters in a name etc. The only thing I can think of in favor of using lots of small files instead of one big file is that one can process files in parallel by running a shell script with something like GNU/Parallel or xargs or make -jN. Maybe there are some other pros of small files vs one (or few) big files, but I don't see them at the moment. Also, some tools and file systems might degrade when there are thousands of files present in a folder. I have helped to patch Dired (directory-files function) and Wdired to work faster, exactly because I experienced Emacs degrading in performance when working with folders with big number of files. Other tools might (or might not) have similar issues; no idea, depends on the tool, of course. I don't want to sound negative, and I don't really think it matters, because there are only so many notes a person can produce, and modern computers are quite fast, especially with M.2 ssds, so the sheer volumes of data here probably do not matter regardless of the approach.

> I have read that Org favours the latter method. If true, I strongly disagree with it because of the implicit dependency it introduces and the way it favours machine-friendliness over human-readability in terms of accessing information. You already get what I mean, based on the aforementioned. >Good luck using less on a generic TTY to read a file with a zillion words, headings, sub-headings, sub-sub-headings, property drawers, and other constructs! My point is that notes should be atomic to help the user make sense of them. The more program-agnostic your file is, the better for you and/or everyone else you might share your writings with.

> In which way are large files harder to use with search tools? > Good luck using less on a generic TTY to read a file with a zillion words, headings, sub-headings, sub-sub-headings, property drawers, and other constructs! You said search-tools; which in terminal are usually grep & co; not about consuming (reading) your notes in terminal. These are two different things. > My point is that notes should be atomic to help the user make sense of them. Sure, I agree, that was the part about Prots presentation that I liked. He has seemingly done a good research on note-taking, and it shines through. However, there is nothing inherent that says you have to have "zillions" of headings, subheading etc in your org file. On contrary, you can have each note appear exactly, and I mean *exactly* as in a single file. You can define a capture template to put a note at the end of the new file in exactly the same format as Prot does, but instead of creating a file with timestamp and tags in the name you would put it as a title of the note (first line), You could even let-bind regex that says what is an outline heading to recognize lines starting with an identifier as headings, and use folding so the org file would look like the file listing in dired as shown in the video.

I use Org Roam and love it, so Denote adds little value in note-taking, but the file naming scheme and the function to rename arbitrary files to conform to the naming scheme is very promising, will install this package just for that function.

> But I don't understand why are you doing one file per note? It seems like very inneficient way of using your long term storage. Depending on what your basic allocation size is (4kb often), it means a lots of disk is wasted for very short notes. If someone wrote 100 notes every day of the year for 25 years, the overhead of the wasted block space would be ~3.7 GB—less than 1% of a current disk's capacity. I wouldn't worry about that too much. On the other hand, having multiple files mean that some operations work very well with standard unix tools; the equivalent with a single large file would either require more involved operations or a custom tool. For example, say you want to find the notes from the last year that contain the words "foo" or "bar". With Prot's current model, that's easy enough: grep -Hn 'foo|bar' /path/to/notes/2022* (The flags `-H` and `-n` instruct grep to print out the filename and line number of the matches. If using ripgrep, that'll happen automatically.) Replicating this functionality with a single file is more difficult. Here's an attempt with awk, but I wouldn't be surprised if it contained bugs. awk '/^\* 2022/ { current_heading = $0 } /foo|bar/ { print current_heading, NR, $0 }' notes.org Since a personal note system is never going to be Big Data™, I think that paying a couple extra kilobytes per file for greater convenience is the right trade-off.

> Replicating this functionality with a single file is more difficult. Are you serious? > awk '/^\* 2022/ { current_heading = $0 } /foo|bar/ { print current_heading, NR, $0 }' notes.org Sorry, but you awk example is completey irrelevant. Why do you search for headings and stuff? You seem to assume that for some reason every org file must contain headings and what not. There is no reason for a single file to have different data structure and layout compared to multiple files. You don't have to use org headings with stars and what not at all. You don't even need to make it an org file, can be plain text file. You can use exactly the same format as Prot uses for a single note, you just put it in a single file and put the first line of a note to be what you would name the note file. I thought it was self-obvious that single file would keep the same data. As a bonus one could create an org-capture template and call it on a shortcut, and you could let-bind the expression org mode uses to find headings to this new format, put it in a minor mode or something, and you have a lot of functionality you can use automatically instead of re-inventing and re-writing everything. But it does not even need to be an org file. Also, the entire idea seem to be centric around the use of terminal; which in a case of a prolific Emacs user as Prot seems to be, is an exceptional case. Centering entire system around exceptional case seems a bit overkill and unnecessary complication. > Since a personal note system is never going to be Big Data™ Sure, and modern computers are faster and faster, so the filesystem innefficiency is hidden more and more, but I would still be worry when it comes to thosands of files. There are other considerations than just space and speed that might come in play as I mentioned to the other poster. > paying a couple extra kilobytes per file for greater convenience is the right trade-off I don't know, you are free to use whatever you like of course, but I strongly doubt that hundreds or thousands of small files will add any extra convenience. As experience shows file systems tend to scale poorly with lots and lots of small records. It is just a tip, feel free to ignore of course.

> I don't know, you are free to use whatever you like of course Thanks! And same to you, hope you find a note-taking system that matches your needs and values.

I am just saying that what seems as a convinencr might ot be very convenient in a long run. There is no need for sarcasns nor dovnvoting my comment for discussing technical issues, and pointing out your erroronous assumption with awk exempel.

In the larger computing world, after many applications have been developed that either used 1 note/message per file or multiple notes/messages per file and evaluating the pros and cons of both, the multiple notes/messages format has predominated. This may not affect your individual use case but is relevant to consider if developing a tool for broader use, as the issue will continue to arise in discussions regardless of available disk size and speed. A common issue that arises is that people do browse directory structures manually and in dired and if a directory has 2,000+ files, it becomes unreadable/difficult to browse.

Very nice! Looking forward to the release of the package. Also, I often add TODOs into these files, would adding these to agenda files make the reload of org-agenda slow? I do not have too many files in org-roam as of now, so agenda works OK, but I have heard of cases where it might grow crazily in time.

I'm not doubting you, but I am curious if you could name some examples of these applications that tended towards the multi-note paradigm. Thanks!

Almost all email storage formats and associated mail readers than mh, which is not widely used. Org mode, HyRolo and Evernote (and many associated clones) all structure multiple notes in single files and allow quick searching and retrieval. I would argue that authors have considered per item file formats and decided against it after weighing tradeoffs. But if a major goal of your system is to leverage POSIX file-level search and filtering tools rather than creating your own set, then that is the tradeoff desired.

im not the one who made the package, but org-agenda did become unusable for me when i tried to insert 300 org roam files into it. so maybe not a good idea if you’re using it in a zettelkasten way with tons and tons of small files, but it doesn’t seem like denote is made specifically for that purpose.