Strategies for literate config in org-mode

I am curious what your strategies/approaches are for using a literate org config. Currently I have the following in my `init.el`: ```elisp (require 'org) (org-babel-load-file (expand-file-name "/path/to/config.org")) ``` My whole configuration resides in a single, separate org file, including the early-init part (which is tangled to a separate `early-init.el` file in `user-emacs-directory`). Whilst this mostly works fine, this approach gives me issues when I try to use the latest version of `org-mode` via `straight.el`. I believe this is caused by the fact that I'm loading the built-in `org` version in `init.el`, and then trying to load/use an updated version in `config.org` (which causes some `void variable` errors for me). Alternative strategies are to put something like `(load-file "config.el")` in `init.el`, and tangle `config.org` manually or [on save](https://www.reddit.com/r/orgmode/comments/jy8fuc/orgbabeltangle_on_save/). Just curious how you all are using your literate org config file, and whether there is anyone who was able to solve/avoid the issue with `org` that I'm having (I could stick to the built-in version, but I would like to be able to use the latest version).

In my private early-init ;; Tangle the file if needed (let* ((default-directory user-emacs-directory) (org-file "config.org") (el-file "config.el") (changed-at (file-attribute-modification-time (file-attributes org-file)))) (require 'org-macs) (unless (org-file-newer-than-p el-file changed-at) (require 'ob-tangle) (org-babel-tangle-file org-file el-file "emacs-lisp")) (load-file el-file))

I use a single Org file to tangle into the different .el files (including early-init.el and init.el as you can see in https://github.com/vidbina/dotfiles/tree/reorg-for-lit-prog/emacs#usage for a visual overview). My entire config will optionally load some personal/private config files which are tangled from an org file in a separate private repo. Sticking to the built-in Org as you can see in https://github.com/vidbina/dotfiles/tree/reorg-for-lit-prog/emacs#-org is my current approach. If updates are needed, I'll just update my Emacs which may be easier for me because I manage my Emacs build with Nix which is also covered through my literate config. From my recollection (of earlier suffering), Emacs and Org don't quite seem to be as loosely coupled as I would hope but I may be misremembering. Specific versions of Org may rely on specific Emacs facilities and some facilities in Emacs may in some cases reach for the built-in Org which causes issues when another Org is loaded that is not at parity to your wishlist Org. You may therefore bump into funky compat issues when you manually install a different version to the built-in Org. My knowledge here is limited though.

I like tangling ahead of time with a bash script, this way I can avoid having to use org during my configurations bootstrap. https://github.com/DiamondBond/emacs/blob/master/bin/tangle.sh

I follow a very similar approach, keeping everything related to Emacs in one org file to be tangled to early init, init, or system specific files. I do however install org mode via straight.el by triggering the install in my init.el before tangling the org file. Here is my config: https://github.com/xcapaldi/dotfiles/blob/main/config.org I recommend reading in plain text so you can see all the org details. You might also be interested in how I separate system specific configs by tangling to Mac.el or Linux.el and then loading them conditionally.

I'll study yours to figure out why I bumped into Org version issues when trying to install newer versions with straight because you seem to have cracked that issue. Thanks for sharing.

I don't know if [my config](https://github.com/amno1/.emacs.d/blob/main/init.org) is to be considered as typical "org" config, but it is literate for sure, definitely unorthodox and uses org-mode as one of it building blocks. It is a blend of a code generator, an application on top of org mode and Emacs setup. The idea is to take a setup and generate a machine specific init file from it so no tangling, probing for files, packages, paths, and other stuff has to be done when Emacs starts. It does also some other, little bit of experimental stuff. I also played a bit with org mode and bend it into a gui for managing the init file. For the setup itself check "packages" heading. Each package has it's own subheading, so I am getting same tidy structure as with use-package, but without using it. As a consequence, I can use helm-imenu so I can quickly jump to any package configuration interactivily. I have also hacked package.el so I can update and install from local repos without need to have two copies of the repo (symlinked).

​ I use straight.el and package.el at the same time and have troubles with it several times, after restarting Emacs `use-package` is unable to load packages anymore. I identified that it happens due to straight.el, so I remove the whole `straight` folder and usually it helps. Straight will install Org mode from sources if Org is a dependency to any package that straight.el manages, must have told it not doing that: (use-package org :straight (:type built-in)) I have these lines in `early-init.el` that tangles emacs-lisp source blocks from an Org Roam note into `init.el` file, that's it: (require 'org) (org-babel-tangle-file "~/Documents/Notes/Pavels-Emacs-Configuration.org" (expand-file-name "init.el" user-emacs-directory) "emacs-lisp")

[deleted]

Using straight alongside package.el is not supported.

I do something similar. The advantage is that you don't pay for loading Org mode automatically on every start.

You need to load org mode from melpa before org-babel-load-file, otherwise you'll gett the older version distributed with Emacs. Here's my init.el file (a bit more complicated than otherwise because I want to split my config in multiple org files and still be able to export it to a single html file.): ``` ;;; init.el -*- lexical-binding: t; -*- (require 'subr-x) ; for string-remove-suffix ;;; We will be using no-littering, but we need first to tell straight to keep ;;; repo and build directories out of the way (defun neo/litter-directory (&optional leaf-dir version) (let* ((dir (directory-file-name (file-name-directory user-emacs-directory))) (distribution (string-remove-suffix ".d" dir)) (package-dir (if leaf-dir leaf-dir "")) (version-dir (if version version ""))) (file-name-as-directory (format "%s-save.d/%s/%s" distribution package-dir version-dir)))) (setq straight-base-dir (neo/litter-directory)) (setq straight-build-dir (neo/litter-directory "straight/build" emacs-version)) ;;; streaight.el bootstrap code from https://github.com/raxod502/straight.el#getting-started ;;; NOTE: the directory used to be user-emacs-directory, we are moving packages ;;; out of our configuration and in places compatible with no-littering (defvar bootstrap-version) (let ((bootstrap-file (expand-file-name "straight/repos/straight.el/bootstrap.el" straight-base-dir)) (bootstrap-version 5)) (unless (file-exists-p bootstrap-file) (with-current-buffer (url-retrieve-synchronously "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el" 'silent 'inhibit-cookies) (goto-char (point-max)) (eval-print-last-sexp))) (load bootstrap-file nil 'nomessage)) ;;; Install org from upstream repo before using org-babel, or we would ;;; get whatever old version is distributed with Emacs. (straight-use-package '(org :type git :repo "https://code.orgmode.org/bzg/org-mode.git" :local-repo "org" :depth full :pre-build (straight-recipes-org-elpa--build) :build (:not autoloads) :files (:defaults "lisp/*.el" ("etc/styles/" "etc/styles/*")))) (straight-use-package 'use-package) ;; Configure use-package to use straight.el by default (use-package straight :custom (straight-use-package-by-default t)) (require 'ox-org) (with-temp-buffer (insert-file-contents "~/neo/neo.org") ;; TODO should export to a different file and move to this only if different ;; this would allow us to cache neo-flat.el. (let* ((flat-neo-config-dir (neo/litter-directory "neo")) (flat-neo-config-file (expand-file-name "neo-flat.org" flat-neo-config-dir))) (make-directory flat-neo-config-dir t) (org-export-to-file 'org flat-neo-config-file) (org-babel-load-file flat-neo-config-file))) ```

The process of extracting code blocks from a literate source. In org mode is a simple thing, in more complex literate programming systems like Knuth's Web, it does more like stitching together pieces, creating indexes and cross references and the such.

Currently I do the following: 1. Everything is in my emacs.org and tangles out to all of the various files I need (early-init.el, init.el, etc). I explicitly tangle it when I want to use changes and de-tangle the source code when I need to make changes or debug with all of my code in context. 2. Use sub-trees for all of my package configs. In each sub-tree I use `:noweb-ref` to collect my package list next to a separate code block for that package's config. This way I can swiftly disable packages and autoremove them by commenting out a subtree. I am currently using package.el but my intention later is to package my whole config as a guix package that uses emacs-packages as a dependency for a highly reproducible config :)

This deserves a big warning in the straight.el readme and some guidance for migration from package.el. Having known that straight.el doesn't work alongside package.el I would not even try straight. My usage of it is simple - for packages that are not on Melpa. Now I'm migrating from straight to el-get. I saw your new package manager Elpaca, kudos for the project, but as I got it also doesn't work together with package.el, right?

> This deserves a big warning in the straight.el readme It is noted in the first section of the README: https://github.com/radian-software/straight.el#features As well in the "Guiding principles" section that immediately follows: https://github.com/radian-software/straight.el#guiding-principles > and some guidance for migration from package.el. The third section, "Getting started" provides such guidance: https://github.com/radian-software/straight.el#getting-started IMO the README is too large and that may have contributed to you not noticing those sections. There's an open issue to rewrite straight's documentation to make it simpler/easier to parse. > I saw your new package manager Elpaca, kudos for the project, but as I got it also doesn't work together with package.el, right? I have no plans on supporting the use of package.el alongside Elpaca at the moment. There is some work being done to package.el itself to allow installing from git repositories. I would keep an eye on that for your use case, but wouldn't expect it to land until at least Emacs 29 or later.

It is possible to install custom packages using package.el by maintaining your own registry. See https://github.com/jcs-emacs/jcs-elpa for example

Thank you for the inspiration! From the responses I like your approach the most as it allows me to keep everything in one org file. I guess the only "downside" is that if you change anything in the blocks tangling to init.el you will have to manually tangle the org file first to ensure changes are applied upon restarting emacs. However, this is a downside I can live with as I would expect changes to these blocks to be minimal.

Actually `org-babel-load-file` will take care of tangling the entire file every time you start emacs. Of course if you make a change to your `config.org` and then restart emacs, it uses the previously tangled `early-init.el` and `init.el` first which subsequently calls `org-babel-load-file` tangling new versions at that point. But yeah, I just tangle it if I change those blocks which is pretty infrequent. I actually I am trying to use a similar literate config for configuration and installed applications on my linux box as well as everything that can be installed via `homebrew` on mac. That is still a little messy and a work in progress but you can actually have a bunch of shell command blocks running under sudo for your system setup "script": ``` #+BEGIN_SRC sh :dir /sudo:: dnf install git #+END_SRC ```