deltille · 2022-04-08 15:40
When pasting text from the web into my org documents I frequently get the "These default coding systems were tried to encode text in the buffer [...] However, each of them encountered characters it couldn't encode" warning.
Usually, left/right quotes and em dashes are the culprits. I go in and replace them by hand, but this is annoying. Is there a customizable setting to either automatically strip these unencodable characters on save, or specify (eg. with an alist) what characters to transform them to?
Admittedly I'm not super familiar with different text encoding schema, so I'm asking this partly to bypass having to spend a few hours reading up. Resultantly, I'm not averse to changing my files to use a specific encoding that would bypass this issue entirely, as long as 1) it's reasonably future-proof and 2) I can still reliably isearch for these 'problem' quote marks and em dashes (ie. simple " character would still find them).
Hamilton950B · 2022-04-08 19:48
Unicode can encode pretty much anything any other coding can, so it's unlikely these characters are not codable into Unicode. I think it's more likely that your org mode buffer has somehow been switched to something other than Unicode. It should tell you at the far left of the modeline what the coding is for that buffer. '-' for ascii, '1' for 8859-1, 'U' for Unicode.
I've got this in my ~/.emacs:
(prefer-coding-system 'utf-8)
I've been told it's a "big hammer" and I shouldn't do that. But it works for me.
deltille · 2022-04-08 22:02
I think it doesn't even have an encoding explicitly specified. The [...] I'm omitting is it trying "undecided-unix" and "iso-latin-1-unix" in that order. (Modeline area where encoding would be is just a "-" and says unspecified-unix on mouseover.)
'Big hammers' as you say are one of the things I'm worried about inflicting on myself. I know at least enough to know UTF8 is *reasonably* easy to deal with though so I may cargo cult that bit of config if I don't find a more suitable option. Brief glance at the help for prefer-coding-system looks like it'll have the desired behavior wrt. not clobbering buffers with a more specific encoding defined.
On a little experimentation it also looks like it isn't inflicting permanent changes on my file,and doesn't even switch to that encoding when it no longer contains any unicode characters, which is great on both counts.
Unfortunately the issue of isearch for quotes and dashes I was afraid of seems to be present. :/ I can deal with it, but it's nonideal. Wonder if there's a translation table already lurking somewhere in the source for precisely this situation...