UNIX
 | IMPORTANT: The content of this page is outdated. If you have checked or updated this page and found the content to be suitable, please remove this notice. |
|---|
"It's a UNIX system! I know this!"—Lex, Jurassic Park
Quirky, often counter-intuitive, but incredibly flexible, UNIX has gone from a little-known research operating system in the 1970s to an entire design philosophy. In its experimental days, UNIX stood in the background, influencing OSes but not making much noise on its own; that changed starting in the mid-1980s, when the first commercial UNIX products appeared, and exploded in the mid-1990s, as OSes either based directly on UNIX or following its principles came to the forefront via the World Wide Web.
UNIX was once considered unfriendly, terse and somewhat elitist, requiring expensive licensing and large, expensive minicomputers to run; now, almost all of UNIX's source code is freely licensed, and UNIX derivatives can be found on things as small as a smart phone [1], or as large as an IBM mainframe. It's also probably the only OS that came into existence specifically because a bored programmer wanted to play a game.
Science-fiction writer Neal Stephenson wrote of Unix, in his essay "In The Beginning Was The Command Line" in January 1999:
"Windows 95 and Mac OS are products, contrived by engineers in the service of specific companies. Unix, by contrast, is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic.
What made old epics like Gilgamesh so powerful and so long-lived was that they were living bodies of narrative that many people knew by heart, and told over and over again--making their own personal embellishments whenever it struck their fancy. The bad embellishments were shouted down, the good ones picked up by others, polished, improved, and, over time, incorporated into the story. Likewise, Unix is known, loved, and understood by so many hackers that it can be re-created from scratch whenever someone needs it."
Multics for the rest of us
Before diving into what UNIX is, it'd make sense to mention the OS that inspired it: Multics. Multics was itself a research OS; it was highly ambitious, was highly secure, required expensive IBM-ish mainframes to run, and was also stuck in Development Hell. (Eventually Honeywell-Bull would commercialize it, but that was several years off.) One of the research partners was Bell Labs, then the experimental division of the Bell System, and in 1969, they decided to leave the project. This left one programmer there, Ken Thompson, with not much else to do.
Thompson had a pet project going on Bell's Multics machine, a game called Space Travel, but at several dollars (in 1969 money) per run, using it was not economically sound. Looking for a cheaper alternative (as well as a new project to work on), he spotted a spare DEC PDP-7 in his lab, and began porting Space Travel to it. As he did, he found himself recreating various parts of Multics around it; eventually, he had a file system, kernel and utilities ready to go. Since it was kind of like Multics, but "neutered", he decided to call the new system "Unics". Eventually, after getting a more powerful system (a DEC PDP-11) to improve Unics and work on the first real application for it (a typesetting package that eventually became nroff), Ken and his development partner Dennis Ritchie started setting the name in small caps, at which point it mutated into "UNIX," which would have been displayed as "Unix" with Ritchie's formatting.
UNIX used to be trademarked by Bell Labs itself, but through several historical accidents it's now controlled by the Open Group, who publishes a "Single UNIX Specification" and grants use of the trademark to OSes that pass a verification suite based on the SUS.
A lot of tropes associated with UNIX from the start began here; Multics had long, self-explanatory commands with much shorter aliases, and UNIX generally just used the shorter forms (like ls, cp, etc.) directly. Back in those days, your terminal was very likely going to be a clunky old Teletype 33 printing terminal, and when the terminal was capable of about 10 characters per second, brevity counted. Also, the PDP-7 in particular didn't have much memory, and Multics' way of handling communications between running programs would have been massive overkill, so Ken and Dennis came up with "pipes", a feature even non-UNIX OSes like Windows NT use now.
Portability, C and the Berkeley connection
Not long after the first edition of UNIX was published inside Bell Labs, Ken and Dennis started work on making the system portable. In the early 1970s, this was a big deal, since almost all OSes up to that point had been written specifically for the machine they were going to run on. The idea was to write most of the kernel in a higher-level language that could be "compiled" into code for different machines, then add small bits of machine-dependent code where needed to handle things like interrupts and memory. The language Dennis invented for the project was called "C", and after he and Brian Kernighan published a book on it, it became a hit outside Bell Labs and even outside the UNIX community (in fact, it's highly likely the Web browser you're reading this in has C or C++ code in it).
Once Unix was ported to C, Bell Labs started allowing researchers at universities to study its insides. Since the Bell System was still a regulated monopoly at the time, and thus couldn't sell computers or OSes, Bell Labs would give university computer science departments access to the UNIX source code for the cost of duplication once they signed a non-disclosure agreement. UNIX became very popular in operating systems classes after this. One class's notes (Lions' Commentary on UNIX 6th Edition, with Source Code, aka the Lions Book) became a widely-bootlegged underground publication and circulated for years this way due to both the Bell System's licensing strictures and the later UNIX Wars (see below); it was, finally, legally published in book form 20 years later, in 1996.
The college with the most influence by far, though, was the University of California at Berkeley, or just "Berkeley" for short. Berkeley's Computer Science Research Group added many new features to the UNIX system, eventually creating their own version of UNIX called the Berkeley Software Distribution, or BSD. Among the many things they did were porting UNIX from the PDP-11 to the VAX (in the process, modernizing the way it handled virtual memory), adding networking support, improving the user input/output functions significantly, and generally cleaning things up. The icing on the cake was the licensing -- Berkeley put their changes under a license that allowed free distribution and modification. This meant that anyone with a Bell UNIX license could use BSD, and this also had bigger implications later on.
In the meantime, Bell Labs released what would be the last important version of "classic" UNIX, the 7th Edition (otherwise known as "Version 7" or "V7" to fans). V7 was considered by many the last version of UNIX ever, and through ports like Xenix found its way onto many machines not powerful enough to run BSD UNIX. There was also a full rewrite of V7 called "Minix", which aimed to help teach UNIX concepts (and, later, advanced concepts such as microkernels) to students and curious hackers that couldn't get a V7 license from Bell. This was many people's first introduction to UNIX concepts, and like BSD, it would have a huge influence later on.
UNIX goes commercial; GNU is born (with the Free Software Movement)
In 1982, AT&T (the Bell System's holding company) lost a long-running lawsuit with the US Department of Justice, and was forced to divest itself of its local telephone service companies (the baby Bells). In return, they were finally allowed to enter the computer industry, and commercializing UNIX was at the top of their priorities. They changed the licensing such that various parts of the system were "unbundled" or a la carte, making a usable UNIX system much more expensive. This annoyed many people, but there was little they could do about it at the time. In the meantime, the UNIX group at Bell continued working on the original UNIX tree, which eventually became UNIX System V, the basis of all of AT&T and its partners' commercial UNIX offerings. Most vendors ended up merging it with their own code bases (often based on Version 7) and creating their own UNIX variants such as HP-UX (Hewlett-Packard) and AIX (IBM). Direct ports of System V were available from outside porting houses like Interactive Systems.
At around the same time, a researcher at MIT's AI Lab named Richard Stallman decided he was fed up with Boston-area startups taking ideas from the AI Lab and "hoarding" them. His main concern was that, once the source to various parts of an OS were "closed" or made unavailable, fixing or improving on them was impossible (or, at least, extremely difficult). He envisioned a complete operating system where all of the parts were "free software" and could be modified and changed at will, but with the caveat that the changes had to be shared with others as well. He called his vision "GNU", a Recursive Acronym for "GNU's Not UNIX!", posted a manifesto describing his intentions to Usenet in 1983, and created the Free Software Foundation to oversee the effort. By 1990, the FSF would have most of the parts of GNU ready, including a compiler, utilities and such, but no kernel to go with it.
On the other side of the US, in Silicon Valley, some of the Berkeley researchers, along with hardware designers from Stanford, got together to make the first computers designed specifically to run UNIX, called workstations. The most famous of these was Sun Microsystems, named after the Stanford University Network, a part of the ARPANET, the ancestor of the Internet. Other companies, like Silicon Graphics, soon followed, and even companies that historically had ignored UNIX before (HP and IBM) got into the fold with the HP 9000 (running HP/UX) and the IBM RT-PC (running either AIX or EOS, a BSD derivative).
The UNIX Wars; Linux and the BSDs appear
Starting around 1988, Sun and AT&T entered into an agreement to develop UNIX-based software together, and called the joint venture "UNIX International". As part of the agreement, Sun (who had been using a custom BSD variant called SunOS up to this point) and AT&T agreed to make a "merged" UNIX that would combine the best parts of BSD and System V; this was released as System V Release 4 in 1990, with Sun's version being released under the name Solaris 2. Other vendors, specifically DEC, IBM, and HP, felt snubbed and formed the Open Software Foundation in protest, and began work on a UNIX derivative called OSF/1, which (like NeXTStep and, much later, Mac OS X), was based on the Mach microkernel and parts from 4.3 BSD.
In the meantime, the IBM PC and its clones were taking over the commercial market by storm, and the Mac was introducing people to a new way of working that many felt no one could touch. UNIX was in a state of disarray; most vendors had proprietary changes to their UNIX builds, none of which were compatible with the others, and porting software was becoming more difficult. This inspired some of the first standards for UNIX, promulgated by the POSIX working groups and by X/Open. Getting everyone to agree would be difficult, but as the PC became more and more powerful and Microsoft began talking about making a version of Windows to compete with UNIX, the impetus to cooperate grew. By the mid-1990s, UNIX International had been disbanded, and OSF stopped development on OSF/1 (leaving DEC to maintain their own branch, renamed "Tru64", for their Alpha machines); OSF merged with X/Open to form the aforementioned Open Group.
At Berkeley, a programmer by the name of Keith Bostic, inspired by Stallman's GNU Project (and Berkeley's own separation of their TCP/IP code in "Networking Release 1"), came up with a large project of his own: purge the BSD codebase of proprietary AT&T code to allow Berkeley-based startups to market it without licenses from AT&T. Involving almost all of the UCB Computer Science Research Group, the project was almost completely successful, with the results being released as "Networking Release 2" or "Net/2", and two "forks" soon emerged; "BSD/OS," from startup BSDi, and "386/BSD," a free software product from Berkeley hacker Bill Jolitz.
Eventually, AT&T decided to wash their hands of the whole thing, and spun off the UNIX group as UNIX System Laboratories (USL). In one final salvo before concluding the war, USL sued Berkeley and BSDi, claiming copyright infringement and restraint of trade. The possibility of a free UNIX appearing in the near future appeared dim.
In 1991, a programmer in Helsinki, Finland named Linus Torvalds, who was inspired by Minix [2] posted his intent to build a "little" UNIX clone (which he intended to be "just a hobby" and "nothing big like GNU") to Usenet, with some of the work already done. Others agreed to help, and by 1992 Linux, as it was dubbed by one of Linus's helpers, was maturing quickly. Linus made the important decision to put the Linux kernel under the same license as GNU's tools and utilities, making it attractive to developers who appreciated GNU's stance. With commercial UNIX still expensive, and BSD's future unclear, Linux grew quickly, and by 1995 had reached version 1.0; already, there were several vendors offering "distributions" or fully-usable OSes based around Linus's kernel and GNU, with the UK-based Ubuntu currently being the most popular.
While all this was going on, Jolitz suddenly dropped support for 386BSD, resulting in a set of unofficial patches to the last version appearing. Eventually, these patches were cleaned up, improved, and became the subject of two forks: NetBSD, which focused on porting the Net/2 tapes to as many platforms as possible, and FreeBSD, which sought to continue where 386BSD left off, and make a quality BSD release intended specifically for PC compatibles. The USL lawsuit still loomed large, though, and progress was slow compared to Linux (which wasn't party to the lawsuit and could move at full speed).
Finally, USL was bought by Novell, and Novell decided to call a truce. It was decided that, with the exception of small parts of code inside the kernel, that BSD was solely under Berkeley's license, and that the UNIX trademark would be given over to X/Open. The decision meant that NetBSD and FreeBSD could return to full development, and by the late 1990s, development speed was on par with Linux. NetBSD itself would later be forked into OpenBSD, which has an emphasis on security, and FreeBSD 4.0 was forked into DragonFlyBSD, due to a disagreement over an architectural change in FreeBSD 5.0.
The Open Source Movement
Not long after this, another development in the software world had a huge impact on UNIX and the Internet. The World Wide Web was finally gaining in popularity after years of development, and the vehicle most people used to access web content was a program called "Netscape Navigator". The company behind it, also called Netscape, had placed the software under a license that allowed free usage for individuals and non-commercial organizations, but required payment for use in corporations; it was hoped that corporations would also invest in their server software, which was top-of-the-line for the time and quite expensive. The problems started when it turned out few people were registering their copies of Netscape; despite what the license said, registration was never mandatory and the program was effectively free to anyone interested. To add to this, Netscape's server software was being severely threatened by the Apache HTTP Server, which was not only free but gaining steadily in market share (and to this day is still the most popular Web server on the Internet by a wide margin, even beating Microsoft's own Internet Information Services). Finally, Microsoft introduced Windows 98 with an "Active Desktop" feature that integrated Web-browser functionality with the Windows desktop.
Strangely, Netscape was actually more upset over what Microsoft did (and the loss of affiliate revenue it would have cost them) than the fact that their server business was dying, and decided to sue Microsoft in a landmark antitrust suit. But that wasn't the only strange thing Netscape did in 1998.
Inspired by the success of Linux and the BSDs, and no doubt knowing that the company itself was in bad shape, Netscape decided to release the sources to the latest version of Navigator (which had been renamed "Communicator" by then) under a free license. Out of the meetings for this came a term called "open source", which attempted to provide the gist of the free software movement's goals without having to go into licensing details, making it easier for people not up on their legalese to understand what was going on. This was a controversial move, with GNU partisans saying it didn't go far enough, while others argued that the word "free" was too easily misunderstood and indicated that the software was worthless. Either way, the term stuck, and "open source" now indicates any software license that conforms to the Open Source Definition, a set of rules based on the Debian Free Software Guidelines (Debian being a highly influential and somewhat conservative distribution of Linux).
As for Netscape, most of the company migrated to mozilla.org, a non-profit founded to handle the development of a new browser based on the Communicator code. After a few false starts (including a decision to rewrite most of the client from scratch, a move that rankled some of the Netscape veterans), the new browser was in usable shape by 2003. A slimmed-down version without the built-in mail/news reader or the Web page editor was developed alongside it, and eventually became Mozilla Firefox, and a slimmed-down version consisting of just a mail/news reader eventually became Mozilla Thunderbird. In the other direction, a fully-featured clone of the Netscape Communicator product suite (including browser, mail/news client, and HTML authoring, among other functions) was released under the name Mozilla SeaMonkey.
UNIX today
Now, in 2012, UNIX and its clones and derivatives are more popular than ever. Apple decided to open-source the kernel and user tools for Darwin, the basis of Mac OS X, and even Solaris, one of the last bastions of old-school commercial UNIX, has since been opened up. Various PC-centric distributions of Linux are giving Microsoft a run for its money for the first time in years, especially on low-end PCs that can't run Vista or 7, and a resurgent Mac (running UNIX itself) is competing with Microsoft on the high end. The BSDs are popular in server, networking and security-related applications. Ubuntu, a distribution of Linux, is quite popular with computer-savvy people who appreciate its ease of use and its excellent hardware support, and is even starting to dent the average userdom. There are also several forks of the official Linux kernel being used on high-end electronic devices and smartphones, as well as in networking gear like Ethernet switches and routers, and in many embedded devices. In the coming decade UNIX systems are likely to spread further into the consumer desktop environment as Microsoft prepares to bring the Windows product line to an end.
The rise of small, simple, low-cost, low-power "netbooks"--basic laptop computers for $300 or less geared towards Internet functionality at the expense of gaming and multimedia--has also worked in UNIX's favor. Many netbooks have discarded the traditional x86 architecture due to power consumption and cost concerns in favor of the Advanced RISC Machine (ARM) architecture, also used by the vast majority of embedded devices such as cell phones. The only version of Microsoft Windows for ARM processors is the largely incompatible "Windows CE" (Microsoft has shown Windows 8 working on ARM, however), and the vast majority of ARM netbooks run some version of UNIX, usually Linux. While ARM is currently used on only a small proportion of netbooks, its share is growing, and one manufacturer believes that ARM machines will control 50% of the netbook market by 2012. Existing Windows netbooks often still run Windows XP, which is now a full three generations out of date, as netbooks' low-performance hardware simply cannot cope with the demands of Windows Vista, although more and more netbooks are now coming with a stripped-down version of Windows 7 called "Windows 7 Starter".
In addition to netbooks, other portable devices nowadays often contain some code from UNIX or UNIX-like operating systems; Apple's iOS, used on the iPhone, iPod Touch, and iPad, contains code from the Darwin base of Mac OS X and iOS's main competitor, Google's Android operating system, contains a fork of the Linux kernel at its core.