How to make emacs accept UTF-8 from the keyboard

My friends have persuaded me to "try again" (about the 5th time in about 12 years) with emacs. I'm currently suffering a little, and need help with emacs + utf-8.

I'm running the 23.3.1 emacs gui on Windows 7 with my own custom keyboard layout (built with MS Keyboard Layout Creator). The layout has a full ISO-8859-1 (Latin-1) character set, plus some additional characters from ISO-8859-9 (Latin-5, ğış etc for Turkish) and ŵ for Welsh (don't know where that one lives).

In my .emacs, I have (blindly) added these lines:

(Update: here's the latest evolving mess:)

;; set up unicode
;; keyboard / input method settings
(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(set-language-environment 'UTF-8) ; prefer utf-8 for language settings
(set-default-coding-systems 'utf-8)
(setq default-buffer-file-coding-system 'utf-8)
(setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))
(prefer-coding-system       'utf-8)
(setq buffer-file-coding-system 'utf-8-unix)
(setq default-file-name-coding-system 'utf-8-unix)
(setq default-keyboard-coding-system 'utf-8-unix)
(setq default-process-coding-system '(utf-8-unix . utf-8-unix))
(setq default-sendmail-coding-system 'utf-8-unix)
(setq default-terminal-coding-system 'utf-8-unix)

Now, when I enter characters from ISO Latin-1 from the keyboard, they are accepted without problems, but characters from outside Latin-1 are "translated" to an approximate character in Latin-1. Thus, for example, Latin-5 "ğ" gets converted to a plain "g".

Cutting and pasting, however, work fine.

Can anyone tell me what I'm doing wrong? I should like to make everything I do with emacs utf-8 with BOM.

Brent.Longborough

Posted 2012-04-08T20:36:50.563

Reputation: 629

Is this in Emacs GUI, or the console version? – user1686 – 2012-04-08T20:41:46.120

@grawity: The GUI. I've updated the question. – Brent.Longborough – 2012-04-08T20:43:35.733

2Welsh is spoken in and around Wales :) – Der Hochstapler – 2012-04-10T16:55:25.900

2@OliverSalzburg: Yes, I live there; I don't know on which ISO-8859 subpage, if any, the ŵs live... – Brent.Longborough – 2012-04-10T18:18:02.500

Does http://stackoverflow.com/questions/10159693/how-can-i-make-sure-emacs-always-saves-edited-created-plain-text-as-utf8 help?

– N.N. – 2012-04-15T09:11:45.563

@N.N. Hi! Fancy meeting you here! No, I'm afraid it doesn't. – Brent.Longborough – 2012-04-15T09:23:40.260

Answers

This seems like it was a problem of codepage (and hence specific to Windows) because Emacs internally used old Windows APIs that could only return chars that belong to your codepage (hence Windows turned ğ into a plain g before passing it to Emacs).

AFAIK this is not the case any more in newer Emacsen (we now use the newer APIs which use some kind of Unicode, probably UTF-16).

Stefan

Posted 2012-04-08T20:36:50.563

Reputation: 1 052

Thank you; I can confirm that my current emacs doesn't suffer from this affliction! – Brent.Longborough – 2019-03-10T15:55:14.133

My emacs says that set-language-environment takes a string, not a symbol.

(set-language-environment "UTF-8")

Does using utf-16 for the keyboard encoding system work?

fstx

Posted 2012-04-08T20:36:50.563

Reputation: 932

Sorry, set-language-environment "UTF-8" has no effect, while (set-keyboard-coding-system 'utf-16) gives "error: Unsuitable coding system for keyboard: utf-16" – Brent.Longborough – 2012-04-25T09:57:02.670

1I would guess that you would need to patch emacs to get it working then. There is a windows message called WM_UNICHAR that might help. – fstx – 2012-04-27T08:38:21.140

on my linux box, I can type greek, french (accents) and spanish (ñ) on emacs with just these lines in my .emacs:

;; Set encoding
(prefer-coding-system 'utf-8)
(setq coding-system-for-read 'utf-8)
(setq coding-system-for-write 'utf-8)

terdon

Posted 2012-04-08T20:36:50.563

Reputation: 45 216

Just tried it. Thank you for trying, but it still doesn't accept non-ISO-8859-1 characters like ŵ and ş. – Brent.Longborough – 2012-08-16T18:08:02.607

That is strange, I mean greek is certainly not in ISO-8859-1 right? Are you sure you are using a font that supports these characters? – terdon – 2012-08-16T18:23:49.620

Oh, yes. Cut-and-paste works fine; it's just the direct entry from the keyboard that doesn't work correctly. How do you enter your Greek characters? – Brent.Longborough – 2012-08-16T22:20:25.037

I added at the begining of my init.el:

(require 'iso-transl)

And it accepts my spanish inputs.

jgomo3

Posted 2012-04-08T20:36:50.563

Reputation: 281

Thanks for the tip. Unfortunately, my spanish inputs are already working OK (as is all of ISO-9959-1). When I switch to Turkish (ı, İ, ş, Ş, ğ, Ğ) that emacs misbehaves, with or without iso-transl – Brent.Longborough – 2015-05-16T18:26:13.883