reddit / emacs
can org-capture-ref replace zotero?
paarulakan · 2022-10-27 04:35
is anyone using [org-capture-ref](https://github.com/yantar92/org-capture-ref) ? I am planning to replace my zotero in my workflow with emacs. so far capturing content from browsers and adding more details by hand which is tedious. Zotero auto-discovers such metadata automatically adaptive to the site. org-capture-ref seems to be doing a similar thing, org-ref does this but its main UX is from emacs, but when I read i mostly live on firefox and only capture when necessary. it helps me explore more, and would the method till get used to browsing via emacs, which may take a long time, because my bad I have huge resistance. I am going to try out org-capture-ref, but would like to know the experience of existing users.
nonreligious · 2022-10-27 06:03
Thanks for bringing this up! I don't have an answer, but one issue I would have is with the downloading and saving of .pdf files for arXiv entries (and also of journal articles you have access to on Elsevier/Nature/JSTOR). But it looks like `org-protocol` takes care of this?
paarulakan · 2022-10-27 06:30
I think org-ref can take of downloading pdf once it has bibliography. I haven't used that feature yet. Only recently I moved to write papers in org-mode entirely, i mean from starting to generating a pdf. previously I just used orgmode to write down contents like a brain dump and then use overleaf(for collaborative from covid times, and most people from my team do not use emacs) to format according to paper. Even right now I use orgmode to write paper included references and generate tex file and then use a very simple shell script to generate draft pdf. Then use overleaf for journal or conference specific formatting.
nonreligious · 2022-10-27 10:52
> I think org-ref can take of downloading .pdf once it has bibliography.
Huh, did not know this. I use `helm-bibtex` + `org-ref` + `org-roam-bibtex` with a bibliography file generated by Zotero. My workflow for reading a paper is:
1. Find paper on arXiv/journal web page.
2. Add to library using the Zotero connector for Firefox. The `Better BibTeX` connector automatically updates a master .bib file for me, that `org-ref` is setup to use.
3. If Zotero is already running in the background, the paper is automatically downloaded; if not, I find the item in the Zotero library afterwards (the bib/reference for the item has been stored to my online library) and then click "Find Available PDF". Zotero renames and stores the .pdf in a folder that is synced across my devices. The .bib file is updated accordingly with a link to the file location.
4. In Emacs, I use `helm-bibtex` and `org-noter` to search for the paper, and then take notes in an .org file. `org-roam-bibtex` ensures that the notes file is associated with the .pdf, and is part of my `org-roam` file of notes.
For me, part 3. would be the thing I'd like to ensure works correctly with `org-ref`/`org-capture-ref` instead of Zotero. I see that in `helm-bibtex`, once I find the reference to the paper I'm interested in (from the master .bib file), there is an `org-ref` option to `Add PDF to library`, but this requires me to paste a URL into the mini-buffer, which I don't have to hand. I'll keep digging around.
As for writing proper papers (instead of working notes), I haven't done that so much since switching to `org-mode` for my workflow -- just used `AucTeX` and Emacs standard LaTeX handling -- but now that I'm more used to it, I will try doing so in `.org` mode. Now, if only I spent more time writing...
paarulakan · 2022-10-27 11:32
I am facing the same issues as you for part 3. For part 2, simple org-capture works for me. it is the downloading associated pdfs. may be I can tweak my workflow a bit to use it from org-ref pov.
thriveth · 2022-10-27 11:45
I don't use org-capture-ref, but I have replaced Zotero with a completely Emacs-based workflow for managing bibliography and citations, like this:
- I use Ebib to manage the "database" itself, which is really just a large BibTeX file.
- Org-ref or org-cite can handle citations in org-documents, I have set them up to use the Ebib database as default bibiliography.
- I use org-roam-bibtex integration to let org-roam handle my notes about each paper. That means that when I click a citation, a hydra asks me what I want, I press 'n', and it takes me to an existing note or opens an org-roam-capture buffer, pre-filled with title, keywords, author etc.
- Org-ref has functionality to get PDFs if you provide the DOI, if I recall correctly, but I usually don't search by DOI
- Biblio (dependency of org-ref) can be used to look up a paper on e.g. ArXiv and retrieve the BibTeX entry.
- Biblio can also download the PDF for you , either direct from ArXiv or it can look for open access versions of the paper. You need to provide a folder path for your bibliography.
- Having this folder path correctly set up, the hydra mentioned before will also offer to open the PDF. It must be named the same as the cite key. If this is correctly set up, it is a breeze to download PDFs and open them for a quick look in PDF-Tools or even org-noter, read and annotate and save your note in the org-roam system, and close it again.
- Citar can search your BibTeX database and insert org-cite formatted citations, BibTeX entries, or formatted references. It can also be used to find and insert multiple cite keys (I don't think it can be made to insert org-ref formatted citations, but once the keys are there, that is easy).
- With Citar + Embark, I can search for a reference , narrow it down with consult/vertico, and with the Embark-citar menu I can either open the PDF or the org-roam notes for the entry that way.
It took me a bit to find the right setup, and there are still a few rough edges, but I still think it is way nicer than any reference manager I have used before.
nonreligious · 2022-10-27 13:03
Cool! Do you have a config file or portion you're willing to share so that ~~I can steal your setup~~ the community can learn from your experience?
contenyo · 2022-10-27 14:02
I've wanted to replace Zotero with an emacs-based solution for a while, but the major pain point has been finding tools that can grab references that don't provide a proper DOI. A lot of the major journals of my field are very behind the times. For me, Zotero's translation server is indispensable.
[Zotra](https://github.com/mpedramfar/zotra) looks like a promising way to use the Zotero translation server from emacs without using the Zotero client, but I haven't had proper time to tinker with it in my config yet.
bdarcus · 2022-10-27 17:02
>I don't think it can be made to insert org-ref formatted citations ...
Per summary [here](https://github.com/emacs-citar/citar/wiki/Comparisons), it cannot and will not by default (citar will only ever support org-cite), but one can configure it to do so if you'd like.
thriveth · 2022-10-27 17:34
No I know that they won't support org-ref but I have also failed to figure out any way to coax it to do it anyway. But I'm no elisp wizard so I've probably missed something. I'd be very interested if you happen to know how to do it?
thriveth · 2022-10-27 17:42
For sure, I just need to see tonight if I can disentangle it from my general config which is big and unwieldy and has a certain spaghetti like quality to it...
bdarcus · 2022-10-27 18:01
The functions to insert citations for major modes are configured in the `citar-major-mode-functions` variable. So you'd just replace the default `citar-org-insert-citation` function there with one specific to org-ref.
thriveth · 2022-10-27 18:36
Thanks! I'll try and look into that.
I love how we can teach each other here!
thriveth · 2022-10-27 19:50
You can download associated PDFs using Biblio, which is a dependency for org-ref.
voidee123 · 2022-10-27 23:44
I have nearly the same workflow. I use Ebib exclusively for creating child bib files. So for each paper I'm working on I have a bib file that is a subset of my master bib file and Ebib can be used to add references from the master file to the child file as needed. This means that citar only shows me references in the child file when I'm working on a paper but in org-roam I have access to all the references.
To add to org-roam, the nice thing is, in addition to having a note file for each reference, you can add in-text citations to other notes that link back to the reference. Then, when in a reference's main note file, the org-roam buffer will show all notes that have referenced the paper. I usually read a reference with its note file open and take initial notes in there then move individual concepts to their own node which will include an in-text citation to that paper. If other papers are related to the same concept, I can then update the concept note and add more references so my idea of the concept changes over time as read more papers.
I have moved from org-ref to the official org-cite but still use org-ref functions for grabbing pdfs. If you use an emacs daemon via emacsclient, you can use the emacsclient CLI to evaluate lisp. So I have a shell script that accepts a DOI and passes it to `doi-utils-add-bibtex-entry-from-doi` which will download the bib entry to my master bib file and try to download the pdf. Using qutebrowser, I can add keybindings that call shell scripts so I have a binding that shows hints over each url on a page, if a paper that I'm reading in my browser has a doi-url (which most do) I selecting the url will have everything download for me.
As far as replacing zotero goes, I've never used zotero but I've heard it's ability to find pdfs is good. In my experience the org-ref's doi tools manage to get a pdf only about half the time (even when the paper is freely available for download). For this I end up grabbing the download link on the site and using `citar-add-file-to-library` to download the file from the url. This works fine for me but I'm under the impression zotero has a much better success rate for automatically grabbing pdfs.
thriveth · 2022-10-28 01:43
/u/nonreligious I have made a minimal-ish config with just those tools, and put it up as a gist https://gist.github.com/thriveth/f961e42dea4988f464ea67d2d6febbb1
I think it gives a pretty nice workflow!
yantar92 · 2022-10-28 04:52
org-capture-ref is mainly developed to obtain bibtex for DOI-less URLs. When a page does have DOI, things are trivial. May you provide an example paper link in one of the journals in your field?
yantar92 · 2022-10-28 04:56
> I am planning to replace my zotero in my workflow with emacs. so far capturing content from browsers and adding more details by hand which is tedious.
org-capture-ref is mostly adapted for qutebrowser. Though a bookmarklet should work in firefox. I did not dig much into major browsers yet. Feel free to open an issue on github or contact me via email.
Downloading pdfs is tricky. org-ref provides some functions to do this, but those functions mostly work reliably with ArXiv. Not so much for papers behind the paywall.
nonreligious · 2022-10-28 04:58
Thanks very much! Something to tinker with this weekend.
Face_Motor_Cut · 2022-10-29 11:26
You are the author of org-capture-ref, right? In theory, org-capture-ref would be perfect for me, because I want to obtain bibtex for all kind of websites. However, I am just wondering, isn't it a crazy amount of work to develop this? Web scraping can change a lot... so, why not use the zotero translation server 🤔
yantar92 · 2022-10-29 11:49
> You are the author of org-capture-ref, right?
Yup.
> isn't it a crazy amount of work to develop this
Depends. Many websites actually follow standard protocols for metadata (like OpenGraph). So, things are not too bad and may be even better in future.
> why not use the zotero translation server
That's an option, but I always felt that it is too heavy. To run it locally, you need docker, AFAIR. And Emacs package opens the flexibility once you know Elisp. Adding a new scraper takes 5-10 minutes for me for most websites - people can easily extend org-capture-ref using Elisp.
The downside is when ceratin websites change the metadata scheme. That is what adds to maintenance. But I haven't experienced too much of this issue so far. Maybe twice since I started development and mostly for particulary alien websites that do not provide metadata at all.