UOMF: Path-Independent Links to Local Files Via Locate

Hum, honestly I'm not much convinced: ok, it does work, locate is reasonably fast etc, but for what? I mean, in a sense I'm caring much, witch is the need of surpassing classic file+directory filesystem approach, that's another push in the same direction, but in practical terms why feeling the need to *keep* the classic storage + a way to access it independent of the hierarchy? Why not just org-attach anything and forget about on-disk location of files?

[My Org heading structure](https://karl-voit.at/folder-hierarchy/) does not reflect [my file management hierarchy](https://karl-voit.at/folder-hierarchy/). While this could be discussed separately (there are good arguments for both sides), this is my situation for now. Besides, in my business life, I do have to use shared hierarchies which I can not influence. This way, org attachments do not provide me much advantage to this simple method of linking local files while it does give me freedom to move around things without breaking stuff. One thing I did not mention in my article is that with `tsfile:` links, I even do have fuzzy search in place: I may add file tags via my [filetags](https://karl-voit.at/managing-digital-photographs/) tools and still do not get broken links (as long as the start of the file is still unique). Furthermore, in case you do refer to moving files to the public cloud, you should read [my opinions on the cloud](https://karl-voit.at/cloud/) and [my opinion when you could use those services](https://karl-voit.at/cloud-data-conditions/). I do care a lot on local files. I will continue to curate a local collection that contain my digital photographs, my scanned paper documents, the files I generate for various purposes, and so forth.

I just published an update that also contains "How I Generate Static File Links" to go the full circle with file links in Org-mode.

Thanks for sharing, personally I would prefer implementing a completing read prompt as the interface to find files.

Well, this can be done on top of the retrieval method described here, yes. However, your completion method for creating new links needs to live-index files which is an additional thing to solve. In my current setup, there is no live-indexing in place and I do think that this would result in a substantial delay for creating links to newly created or downloaded files. Or do you have an idea to solve this in a clever fashion? For a limited scope, I actually do use a completion prompt for inserting file links: `my-webarchive-tsfile` from [my config](https://github.com/novoid/dot-emacs/blob/master/config.org) does that but just for one single directory.

That's exactly my point :-) you state that people's prefer traversing tree instead of search, I imaging (without having proof) that that's because of the fear of loosing something, being unable to find it again in the future. However you also state that "too much is too much" so trying to have not much files per directory and not much directories is mandatory in this model. Again I'm adding (still without proof) that that's because with too many files/directories you actually loose things simply because consulting them/manually traversing them became too tedious. PIM research is unfortunately curiously "paused" for a certain number of years, at least AFAIK, and these days people's tend to have too much files. GMail success IMVHO is a very good proof of that: most people's before was "filer" while these days are mostly "piler" and do use search to find things. Same thing do happen in UI terms: on desktop we switched from classic hierarchical menus to "dashes"/"quicklaunchers" etc all in the search&narrow model. At the other end of the spectrum library research almost give up with classic classification observing than even faceted ones are simply unable to sustain today demand and most practical access is via full-text/semantic search of some kind. At web scale the same happen: in the past "directories" was common, they and long ago replaced by search engines. I imaging that not all your files can be used via org-mode, some apps are not much usable by humans with cache-like storage, also evolving a legacy BIG set of files and directories is a nightmare, but in the long run to reconnect to the hierarchical model you describe why not giving up with a *complete* home folder taxonomy substituting it with a hybrid taxonomy where at the "first level" you have org-mode/org-attached stuff accessed via search&narrow and under them you have smaller hierarchies you can easily manage once in them, with not too much files + directories? I imaging you do have a *many* level home taxonomy, that's can be transformed to a far less levels where direct "file-manager alike" files access is bound to a "local root", the ones elicited from org-mode. A stupid limited example: I do store my phone bills in a small hierarchy: a common root, a II level of one folder per years, a third level with a bill per month, with the file named after the month (two digit) the invoice ID and the amount. Such small two-level hierarchy of not too much files instead of being "rooted" under `~/docs/invoices/company-name/yyyy/mm-....pdf` and symlinked under `~/docs/properties/house-name/phone/yyyy/...` and also under `~/docs/contracts/by-company/company-name/...` and `~/docs/contracts/by-kind/TLC/company-name/...` etc I have had in the past is just directly accessible with a simple search&narrow UI (ivy/helm/consult/* style) from many places inside org-mode and for any software I do want to use on it I just need to launch it with the relevant attached dired buffer opened, it's PWD became the local root and I can easily see all files in a hierarchy. The same for git-ed projects, just a common org-attached root and a small hierarchy underneath. I do not be much attentive but doing so I almost respect your suggested "not much than 21" rule and still being able to scale "enough", perhaps not at a cloud provider level scale but hopefully at a personal level scale even 50+ years in the future...

(Oh great: "This field must be under 10000 characters". I'll have to split up my answer then. This is part 1/2.) >you state that people's prefer traversing tree instead of search, I did not state it in the said article but I stated it at other locations because this is the current state of research on that topic, yes. >I imaging (without having proof) that that's because of the fear of loosing something, being unable to find it again in the future. Boah, this is a bold statement. [The Science of Managing Our Digital Stuff](https://mitpress.mit.edu/books/science-managing-our-digital-stuff) has a whole chapter (Chapter 8) on this question. While there is no simple answer as you suggests, there is also good evidence that navigation is also more efficient than search in many cases. >However you also state that "too much is too much" so trying to have not much files per directory and not much directories is mandatory in this model. Again I'm adding (still without proof) that that's because with too many files/directories you actually loose things simply because consulting them/manually traversing them became too tedious. Actually, I don't state that. And I don't think that this is "mandatory in this model". In [articles like that](https://karl-voit.at/2020/01/25/avoid-complex-folder-hierarchies/) I even propose less directories with more files - so quite the opposite of the thing you thought I'm stating. Furthermore, I never had the impression of losing things. ​ >PIM research is unfortunately curiously "paused" for a certain number of years, at least AFAIK, Interesting. What brings you to that conclusion? ​ >and these days people's tend to have too much files. How do you define "too much"? Yes, the average number of files curated by single persons increased over time (Chapter 2 of the book above). But what is "too much"? I *know* that I won't retrieve the majority of files I curate on my disk. Unfortunately, I can't tell *which* part I *will* retrieve in future. As long as cost of keeping does not interfere with retrieval success, the number of items does not matter. And this is an art you can master. Therefore, I'm skeptic that there is a "too much" but only a "not learned to curate and retrieve efficiently enough *yet*". And the latter I try to beat with my blog, my lectures and my book(s) that are in planning stage (for years ;-) ). That's bit literacy which ought to be taught in school but it isn't. ​ >GMail success IMVHO is a very good proof of that: most people's before was "filer" while these days are mostly "piler" and do use search to find things. This very short form, I can not subscribe. And it's not valid to compare email retrieval with local file retrieval. Not at all. [I wrote a paper](https://karl-voit.at/tagstore/en/papers.shtml) on that topic "Creating a Comparative Environment for PIM Evaluation" where I try to make my point that it's not that easy to compare different UIs with each other in PIM research. And comparison between email UX and local file retrieval UX is even less related. ​ >Same thing do happen in UI terms: on desktop we switched from classic hierarchical menus to "dashes"/"quicklaunchers" etc all in the search&narrow model. You might have and that's fine but numbers for the average users do not reflect your assumption. ​ >At the other end of the spectrum library research almost give up with classic classification observing than even faceted ones are simply unable to sustain today demand and most practical access is via full-text/semantic search of some kind. I think this is related to the size of the exploration space of items versus search query length (entropy) and search term quality. My guess: Simple search might scale well until a certain point where you would need to come up with more and more specific search queries that navigation or faceted search interfaces do perform better. If this holds true, search only gives you great results for some time. ​ >At web scale the same happen: in the past "directories" was common, they and long ago replaced by search engines. Yes, there were manually curated hierarchies from Yahoo! and others but quite frankly, I never used them myself and I never saw one of my friends using them. Search took over instantly as there were basic search engines available. Indexing was easier as well, back in the old days.

(Part 2/2) >I imaging that not all your files can be used via org-mode, Depends on the definition of "used via". With my Memacs system, I may link almost any file within Org-mode. However, this is a weak argument here. &#x200B; >some apps are not much usable by humans with cache-like storage, also evolving a legacy BIG set of files and directories is a nightmare, So far, I don't feel like having a nightmare and I certainly am on the larger side of private data hoarders. &#x200B; >but in the long run to reconnect to the hierarchical model you describe why not giving up Oh, when it comes to giving up the current concepts and developing computer UX that is really leaving our stone-age systems behind, I do have ideas far beyond hierarchies and such. You can get a sneak preview in articles like [this](https://karl-voit.at/2017/02/10/evolution-of-systems/), [this](https://karl-voit.at/2018/10/17/ted-nelson-interview/), [this](https://karl-voit.at/2019/10/05/innovation/) or [particularly that](https://karl-voit.at/2020/06/21/simplified-to-standstill/). [WinFS](https://en.wikipedia.org/wiki/Winfs) would have been a first small step in the right direction. &#x200B; >with a complete home folder taxonomy substituting it with a hybrid taxonomy where at the "first level" you have org-mode/org-attached stuff accessed via search&narrow and under them you have smaller hierarchies you can easily manage once in them, with not too much files + directories? I imaging you do have a many level home taxonomy, that's can be transformed to a far less levels where direct "file-manager alike" files access is bound to a "local root", the ones elicited from org-mode. A stupid limited example: I do store my phone bills in a small hierarchy: a common root, a II level of one folder per years, a third level with a bill per month, with the file named after the month (two digit) the invoice ID and the amount. Such small two-level hierarchy of not too much files instead of being "rooted" under \~/docs/invoices/company-name/yyyy/mm-....pdf and symlinked under \~/docs/properties/house-name/phone/yyyy/... and also under \~/docs/contracts/by-company/company-name/... and \~/docs/contracts/by-kind/TLC/company-name/... etc I have had in the past is just directly accessible with a simple search&narrow UI (ivy/helm/consult/\* style) from many places inside org-mode and for any software I do want to use on it I just need to launch it with the relevant attached dired buffer opened, it's PWD became the local root and I can easily see all files in a hierarchy. OK, speaking of our current computer concepts again: I think we both do follow similar approaches here. [My folder hierarchy](https://karl-voit.at/folder-hierarchy/) consists mostly of `archive/<YEAR>/<FILE OR EVENT>` "layers". Starting a navigation process by first teleporting to a specific path is the way to go also according to my own perspective. Concepts like [frecency](https://en.wikipedia.org/wiki/Frecency) should be much more popular as they are at the moment. As far as I understand, the main difference is that you do have a large set of "entry points" for directories that do reflect your Org-mode hierarchy. I do have a unified archive, not related to Org-mode hierarchy elements. However, I do see advantages here as well. But I would not be able to follow that pattern completely since I do have to stick to certain shared sub-hierarchies where I would need to adapt a different concept for Org headings to match the path structure. So far, I do not follow that path. &#x200B; >The same for git-ed projects, just a common org-attached root and a small hierarchy underneath. I do not be much attentive but doing so I almost respect your suggested "not much than 21" rule It's not my rule. It's a rule mentioned in the PIM literature. I don't follow that rule at all. &#x200B; >and still being able to scale "enough", perhaps not at a cloud provider level scale but hopefully at a personal level scale even 50+ years in the future... ... which is a promising situation. In general, there are no single approaches that do work for all people. Furthermore, there are many different approaches that are valid in parallel without much difference in quality. Our discussion here is something the vast majority would not understand at all. This shows that you and I have thought about the *how* and *why* a lot. This makes us a member of a tiny fraction of users who try to come up with concepts that are able to carry them through decades, not just months or years. For the large majority, we need to come up with easy to follow cooking recipes to avoid common mistakes and provide at least a file curation and retrieval experience way above average. This is the view point you should have when reading my blog articles. When I do write about recommendations that doesn't necessarily mean that there is no other way to do things in an efficient and effective way. Conscious people who read my articles may get input to slightly modify or extend their methods but they should also realize where my recommendations differ from theirs without judging their method of being a less perfect one. People who did not think that conscious about their PIM decisions in the past are probably happy about any recommendation they can get. In the end, everything is fine as long as its working. If you bet on the wrong horse in the long run and if you find out that something is not working any more, you should be able to realize this and change that for a better workflow. I try to bet on the horses that support me for the very long run but I might be wrong here and there as well. Time will tell. *Ceterum autem censeo* [*don't contribute anything relevant in web forums like Reddit only*](https://karl-voit.at/2020/10/23/avoid-web-forums)

Thanks for the detailed reply :-) About the 1k chars, well, that's can be a nice point of departure to discuss both the need of "new" (actually old) computer systems and interface "simplification" (actually complication) but while IMVHO super-interesting a bit too much off topic here. To answer I have to precise: I do not study, nor I've done that in the past, PIM subject in a scientific manner, I'm a sysadmin, so a computer user inhabited to see scale from a personal to bigger ones, witch have time to observe, curiosity and automation as one of the main everyday topic. So most of my conclusion are short-cut observations in a certain timeline. From that I've elicited few things, that hopefully can be summarized as much as: - casual users, yes, do not care much about *anything* in "digital" and "documental" terms, for most such activities and topics are just "an unfortunate need of our society", something to be faced as less as possible with the least possible investment; - computer systems do have evolved with no unique purpose, nor with a crafted path, almost anyone seems to have had a real future vision AND being able ad least to demoed that with something tangible. That last part perhaps can be a good point of departure: the first modern computer systems as information systems probably is Paul Otlet/Henry La Fontaine "Mundaneum" from the '30s, was not tangible at that time, but was essentially the web 1.0 with Minitel-alike terminals. The second revolutionary step happen at Xerox with SmallTalk workstations where the "UI" start to be defined and the way to work on information in an automated manner arrive. From that era I just see a long middle age with a vicious circle: people do not care, can't even imaging what they eventually see from a demo, companies are not interesting in teaching, they try to sell something matching users desire and their own profit desire (see General Magic as an example of the outcome, hyper-raw systems that try to mimic the physical world in a virtual one, avoiding the real evolution) so we enter in a kind-of limited and limiting "mechanical UI" era, the erotic dream of nazi-school managements from IBM systems to "now classic" widgets-based UI. They suffice since the society actually try to keep up anyway, no matter how inefficient or uncomfortable a system can be as long as it work enough to survive, and most people's prefer very slow and limited evolution while fear the change. Now after decades of limited and limiting systems we start to see a need of change no one can keep quiet, Notebook UI start to appear, "the notes movement" (not just ZK mania but also classic Evernote, Google Keep etc to name some for the large public), desktop software start to came back to document-alike concept proper to the web etc, for me simply because actual slow, inefficient, limited digitization reach a point where the IBM&co approach can't work anymore. We are at the point of "from menu to search&narrow" witch is not much "my" evolution, but an almost-imposed one, try to see desktop UI: the classic was Windows XP menu, Gnome Menu, KMenu down to fluxbox/windowmaker "right click and use the menu" approach to hit-a-key and search of *all* modern DE "menus", including Android that start to offer a search bar for pretty anything, to the web with classic search engine "command line at middle of the page". That's might be not thought as I describe, but in the end we are there. For file management is not that different: most *really new* users works on "Google Drive alike" systems dropping files without even understand the meaning of "file" [1] and using "topologic memory" [2] and search to retrieve them. I do not really have statistics but that's what I observe. So in model terms I see: - the filing cabinet peoples who try to organize as best as they can, some just in a disposophobic pathological manner some with reasoning and knowledge - the just search & hope new one - many in between In academic-alike environment many use hybrid systems like Zotero, not just as reference managers but as real file management solution, in enterprise-alike environment many use DMS of various kind for giants like Nuxeo to TagSpace "abusing" them in similar way. ALL struggle in integrating them. There is an entire industry who try to integrate mails, calendars, files, tickets, PM-things, people, ... and all are bloated and crappy limited monsters. Emacs do offer a vague base to do the same AND succeed. Both because in "technical terms" is a classic system where the UI is a "2D UI" [3], an user-programming environment and because org-mode is a nice glue to anything, since it can link mails, be an agenda, act as a CRM, ... and from that my target is reach "my PIM", my fully-integrated desktop environment to store, work on my information. Sorry for the "bold and raw" comparisons of systems before, is not intended as a judgment but as a POC, something to base a reasoning on, with the above described target. The essence probably is: from mere observation I see a feeling of a need, witch is also mine, but I feel it's far broader than that, of integration, files themselves are limiting stuff, the mere fact of being "different" for "different" software is an issue, so are "modern" software, anyone self-contained, at maximum with API to interact with the rest of the world. What I propose is something that can IMO/IME scale enough and is also effective to use... [1] https://www.theverge.com/22684730/students-file-folder-directory-structure-education-gen-z [2] https://www.researchgate.net/publication/266595198_Text_is_not_the_enemy_How_illiterates%27_use_their_mobile_phones [3] https://hongchao.me/cli-and-emacs/