Serial console login on OSX

Question

Background

I've got a Wyse WY-50 terminal here that, for various nolstagia and productivity reasons (really!), I'd like to connect to my Macbook and use as a login terminal. I'm using OSX 10.10 El Capitan.

The terminal itself has two ports on the back, one marked MODEM and one marked AUX. From the manual, normally the device you're connecting to uses the MODEM port, and another ancilliary device like a printer uses the AUX port. These are both DB25 ports.

The MODEM port is connected to my mac via a DB25-DB9 converter, and from there to the mac via a USB-DB9 adapter using a Prolific chipset.

I had to install a driver for the converter to become available. After that was done, I got two devices created in /dev/, those are cu.usbserial and tty.usbserial.

I followed the guide here to set up a LaunchDaemon Plist to spawn a getty session attached to the terminal, since apparently you can't just either edit /etc/ttys or run getty by hand in modern OSXes.

The terminal itself is set to VT100 emulation mode at 19200 baud, with DTR flow control on receive and no flow control on send (the only option for send flow control is XON/XOFF)

Difficulty

This setup has proven difficult to troubleshoot, since incorrect communication with the serial device tends to "hang" it, requiring a reboot to become usable again.

In all cases, calls to getty whether in a file or by hand were of the form /usr/libexec/getty std.19200 cu.usbserial

Anytime you see "No output", this means the terminal was slient, showing nothing, and was unresponsive to keypresses.

• Running getty by hand: No output. (Seems to validate the guide)
• Adding a line to /etc/ttys: No output. (Again, validates the guide)

From here, I tried the plist method by adding a file to /Library/LaunchDaemons named console.plist with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/>
    <key>KeepAlive</key>
    <false/>
    <key>Label</key>
    <string>serialconsole</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/libexec/getty</string>
        <string>std.19200</string>
        <string>cu.usbserial</string>
    </array>
    <key>StandardErrorPath</key>
    <string>/tmp/serialconsole.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/serialconsole.out</string>
</dict>
</plist>

After loading and running the job, the following things were tried, each time attempting to stop, edit, unload, reload, and run the job in order:

• Flow control on the terminal set to NONE: No output.
• Different baud rates set on the getty command and the terminal (from 9600 to 19200): No output.
• Using the tty.usbserial device rather than the cu.usbserial device in combination with the above: No output.
• Moving the serial line on the terminal to the AUX port and reconfiguring it to use the AUX port for communication: SOMETHING HAPPENS!

On another port

So, I can get basic communication if the computer is connected to the terminal by way of the AUX port. With settings as follows:

• getty running under launchd with the above plist
• Terminal personality set to VT100
• Flow control disabled

However, this results in a lot of garbage characters on the terminal screen. The login prompt itself is clean, but hitting return always generates a small amount of garbage. After logging in, this garbage tends to get concatenated to the end of commands, making the session almost unreliable and unusable.

With the settings as described above, here's an example of what I mean on the terminal output:

You can see some noise once the port is initialized, and then a "clean" login display. I enter my username and press Return on the terminal keyboard. Rather than dropping down a line and prompting for a password, You see the UTHx appear, and then the Password prompt on the same line.

Entering my password (which is properly not echoed) displays the "last login" message, and then a completely trashed shell afterwards. It's almost as if linefeed characters are being mangled somehow.

If I press Ctrl+l on the terminal, the last line resets itself, and I see a clean prompt (not pictured). However, the garbage returns the next time I press return on the terminal.

My terminal is set to /bin/sh.

Another suggestion I read was to try using the reset command to clear the display to known-good settings. Typing reset and hitting return gives me a clean error "Unknown terminal type: su (-1)". With a prompt for Terminal type?.

Changing the "personality" setting on the terminal, correlated with a reset above, to other promising options only modifies the "kind" of garbage I see - none of it is clean.

• VT100 (as shown)
• UNIX CONSOLE (lots of high-ascii symbols)
• WY-50 (the actual model of the terminal: lots of high-ascii symbols sprayed all over the screen)

Hardware?

I don't think so. The exact same adapter/terminal/etc setup was connected to an Ubuntu Linux machine, with getty configured as described, and it Just Worked®, with no noise or garbage on the screen.

Furthermore, using the third party application Serial to just write plain text to the terminal works without issue - no garbage, and newlines are handled correctly.

Something about Mac OS's configuration with getty is causing issues with this terminal.

Any ideas?

Update 1

I found a way to get a usable terminal session. Run screen, pointed at the serial port like so:

screen /dev/cu.usbserial 57600

And then from within screen, hit ctrl + a, : and then type:

exec ::: /usr/libexec/getty std.57600

Et voila, a clean and usable login shell appears on the terminal!

However, this method still has some drawbacks.

The prompt line is doubled. that is, any time you hit return at the prompt, the prompt, a newline, and another prompt appears. There's also still a ton of noise on the line - any commands that dump a lot of text at once like an ls results in garbled output.

Pictured below:

Programs are convinced that the terminal type is set to screen, even if i do an export term='vt100' - this means that most full screen terminal apps like vim and mutt drop a lot of control codes that the terminal can't handle, usually hanging it to the point a reset is necessary to gain input again.

I'm still searching for a way to have the terminal Just Work on login.

Answer 1

After much fiddling around, I finally figured out the right combination of things to use for a fully usable terminal.

tl;dr: FTDI-based usb/serial adapter, terminal set to VT220 with flow control both ways, special settings in /etc/gettytab

The details

Hardware

I got a new USB/Serial cable. This one has the FTDI, rather than the Prolific chipset. This is important since OSX comes with the FTDI drivers in the box, rather than having to install the third party Prolific drivers. This made troubleshooting much easier, since the serial port stopped getting stuck after each experiment, meaning no more reboots to become usable again.

If you're going to try this yourself, the cable I used was the StarTech Industrial USB RS232 Serial Adapter. This was in conjunction with a random DB9/DB25 adapter, and a gender changer since the ports on the back of the terminal are female.

Note: A previous version of this guide suggested using a different cable - this particular cable will let us use the modem port on the terminal, correcting some various display glitches!

This cable should be connected to the MODEM port on the terminal.

System Configuration

Ignore any guide that suggests you use /etc/ttys, since this file is ignored in modern versions of OS X. This would be the way to go on Linux/BSD, but not here.

I was left wondering how to set the terminal type for the login, since as described above, just using the std.19200 that's provided in /etc/gettytab left the terminal set to su for some reason, and nothing worked.

I added a line as follows:

std.ttyUSB:\
    :np:im=\r\n:sp#57600:tt=vt220:

This sets the login banner to a simple CRLF, sets the line speed to 57600 baud, and most importantly, sets the terminal to vt220. This will become important later.

Now we need to actually start this. As described elsewhere, you can't just run getty, since that's restricted to launchd. We have to make and load a daemon plist.

To /Library/LaunchDaemons, I added a serialconsole.plist with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>KeepAlive</key>
    <true/>
    <key>Label</key>
    <string>serialconsole</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/libexec/getty</string>
        <string>std.ttyUSB</string>
        <string>cu.usbserial-AI02GVBH</string>
    </array>
</dict>
</plist>

If you do this yourself, the third string will almost certainly be different, since this is the serial number of the adapter. The second line references the line we just added to to /etc/gettytab. Also note the setting of KeepAlive to true - this means that when the process exits (say, we log out), a new instance will spawn. Without this, you have to manually start the job.

Terminal Settings

Finally, the terminal itself has to be configured. This is a Wyse WY50, but it can emulate many other terminal types. I find that VT220 mode gives the best experience.

To enter setup mode on the terminal from factory settings, hit Shift + Setup.

Pressing F1 to enter the DISP menu, I set:

• Columns to 132
• lines to 42,
• Autopage OFF
• 80/132 CLR ON. (Clears the screen if the character mode changes)

F2 to the General screen.

• Personality set to VT220-8 important!
• Scrl set to Jump (screen scrolling goes ridiculously slow on other settings)
• RCV CR set to CR (transmits just a CR, rather than a CRLF)
• Enhance set to ON (accepts enhanced control codes)
• Autoscrl set to ON (self explanatory - scrolls the screen rather than overwriting it)
• Monitor set to OFF
• Recognize DEL set to ON

F3 to the KEYCODE screen

• Keycode set to ASCII
• Corner key set to META (also known as Alt)

F4 to the COMM screen

More super important stuff:

• Comm set to FDX (full duplex)
• MDM RC HSK set to Both (receive flow control)
• Send Ack set to On
• MDM XMT HSK set to Xon-Xoff

One more - F5 to the PORTS screen

• MDM BAUD RATE set to 57600
• AUX Data/Parity set to 8/None
• AUX stop bits set to 1

Everything else I left at factory defaults.

Hit F12, then Space to set Save to Yes, and then F12 once more to save settings.

Putting it all together

We're ready to go. Make sure the terminal is on and plugged in to the port, verify the settings are as described above both gettytab and the plist file you created.

We load the created plist into launchd:

launchctl load -w /Library/LaunchDaemons/serialconsole.plist

And finally start it:

launchctl start serialconsole.plist

And in a moment, you should have a login prompt on your terminal! Some photos:

Display glitches

You might notice that certain applications still write garbage onto the screen, especially if they do anything fancy like bold text or certain ASCII characters. Most recent machines, macs included, use UTF-8 for their character set. But this venerable glass terminal has no idea what that is! You'll need to set the locale to the appropriate legacy codepage for your region. In the USA, this is en_US.ISO-8859-1. Have a look at the output of locale, and work the command export LC_ALL='en_US.ISO-8859-1' into a login script that checks your $TERM variable. You really don't want to set this system wide, but only on sessions where your glass terminal is being used.