Survey of Operating Systems:
§ 6: UNIX Advanced Features

Instructor: M.S. Schmalz

Reading Assignments and Exercises

The development of UNIX has led to the corresponding development of a variety of tools and capabilities that augment the original UNIX system calls, commands, and utilities. In this section of our course notes, we review several advanced features that you are likely to encounter in practice.

In particular, this section reviews the use of UNIX communication features and programming toolkits. This section is organized as follows:

6.1. Communication using mail and talk
6.2. Overview of Communications and Sockets
6.3. Toolkits, including X-Windows and Motif
6.4. Enhanced UNIX Operating Systems, including Mach

Information in this section was compiled from a variety of text- and Web-based sources, and is not to be used for any commercial purpose.

6.1. Communication using mail and talk

Reading Assignments and Exercises

It is convenient to be able to send messages over the Internet via Email, and to "talk" interactively with colleagues or friends using the UNIX text-based conferencing facility called talk.

6.1.1. UNIX mail Program

The UNIX mail program allows you to use the Internet to send email messages. Invocation of the program occurs by typing mail at the prompt. A list of messages in your inbox is then presented, and you choose a message to read by its index tag or number. You can reply to the message and make a carbon copy for others to read. An example mail session follows.

Example. Here is a sample mail session from our hypothetic user bsimpson's account:

eclipse:92% mail
Retrieving mail from POP server mail.cise.ufl.edu...
mailx version 5.0 Tue Jul 15 21:29:48 PDT 1997  Type ? for help.
"/cis/homes/bsimpson/inbox": 263 messages 2 new 108 unread
 U261 ti@info.techonline Tue Apr 13 18:04   48/2061  Texas Instruments Worldwi
>N262 gerd@cise.ufl.edu  Tue Apr 13 19:06  121/5054  Warning: could not send m
 N263 tmch@cise.ufl.edu  Tue Apr 13 19:08   37/1562  Biomimetic Signal process
? 

Note the invoke following the UNIX prompt (eclipse:92%, where "92" denotes the number of that command in the history sequence belonging to the UNIX shell from which the command was invoked).

Also note the version number (mailx version 5.0) and date (Tue Jul 15 21:29:48 PDT 1997).

There are 263 messages total in the inbox file, of which 2 have arrived recently (new), and 108 have not been read.

The newest messages begin with Message 262, which is pointed to by the character ">". Each message has a tag, an index, originating address, date stamp, size, and partial subject header.

For example, Message 261 is from Texas Instruments technical support group (originating address = ti@info.techonline). Its date stamp is Tuesday, 13 April 2000 at 18:04 GMT (Greenwich Mean Time), and there are 2,061 bytes in the message body.

To read a message, one enters the index of that message at the mail program prompt ("?" on the last line of the preceding example), and the message header and message body appear on the screen.

A diverse group of functions can be invoked from the mail prompt. One of the following mail commands is entered:

 mailx Commands
alias,group user ...            declare alias for user names
alternates user                 declare alternate names for your login
cd,chdir [directory]            chdir to directory or home if none given
!command                        shell escape
copy [msglist] file             save messages to file without marking as saved
delete [msglist]                delete messages
discard,ignore header           discard header field when printing message
dp,dt [msglist]                 delete messages and type next message
echo string                     print the string
edit [msglist]                  edit messages
folder,file filename            change mailboxes to filename
folders                         list files in directory of current folder
followup [msglist]              reply to authors of messages and save copy
Followup [message]              reply to all recipients of message and save copy
from [msglist]                  give header lines of messages
header [message]                print page of active message headers
help,?                          print this help message
hold,preserve [msglist]         hold messages in mailbox
inc                             incorporate new messages into current session
list                            list all commands (no explanations)
mail user                       mail to specific user
Mail                            mail to specific user, saving copy
mbox [msglist]                  messages will go to mbox when quitting
next [message]                  goto and type next message
pipe,| [msglist] shell-cmd      pipe the messages to the shell command
print,type [msglist]            print messages
Print,Type [msglist]            print messages with all headers
quit                            quit, preserving unread messages
reply,respond [msglist]         reply to authors (only) of the messages
Reply,Respond [message]         reply to the author and recipients of the msg
save [msglist] file             save (appending) messages to file
Save [msglist]                  save messages to file named after author
set variable[=value]            set variable to value
size [msglist]                  print size of messages
source file                     read commands from file
top [msglist]                   print top 5 lines of messages
touch [msglist]                 force the messages to be saved when quitting
undelete [msglist]              restore deleted messages
undiscard,unignore header       add header field back to list printed
unread,new [msglist]            mark messages unread
version                         print version
visual [msglist]                edit list with $VISUAL editor
write [msglist] file            write messages without headers
xit,exit                        quit, preserving all messages
z [+/-]                         display next [last] page of 10 headers

[msglist] is optional and specifies messages by number, author, subject or
type.  The default is the current message.

In each of the preceding commands the expression denoted by [msglist] is a list of message numbers or indices delimited by spaces, for example the list:

261 263

would specify messages 261 and 263.

This concept of operations on multiple messages can be very powerful, as shown in the following example.

Example. To delete messages 161, 162, 163, 164, and 213, the following command sequence would be entered at the prompt (emboldened):

? d 161-164 213

where the range specifier 161-164 has the usual meaning.

Care should be taken in using range specifiers, as it is possible to inadvertently delete a message you wanted to save. Unless a backup of your inbox file is made, there is no way to recover the deleted message once you have exited the mail program. If, however, you delete a message before exiting mail, then you can use undelete to restore the deleted message.

6.1.2. Interactive Communication with UNIX talk Program

It is occasionally useful to chat interactively with a colleague who is logged into a UNIX machine elsewhere. With the UNIX talk utility, one can "converse" using a typewriter-like display.

To use the talk utility, enter talk address at the UNIX prompt, where address denotes the current address of the person you want to chat with, and is formatted as follows.

The address can be the username, as returned by the who(1) utility. For example, to talk to a user logged in to the same machine as yourself, address is the user's login name (e.g., bsimpson). To talk to a user on another machine (also called a host), address must have one of the following forms:

host!user
host.user
host:user
user@host

In most cases, user@host is preferred.

As soon as it is invoked, talk sends a message to the specified address that looks somewhat like the following:

Message from TalkDaemon@ her_machine at time ...
talk: connection requested by your_address
talk: respond with: talk your_address

The recipient (i.e., the person at address) can establish the talk connection by typing:

talk your_address

As soon as communication is established, the two communicating parties can type simultaneously, with text output displayed on different halves of the screen. Characters are handled as follows:

Typing {Ctrl-L} will refresh the sender's screen, just as in emacs.

The erase and kill characters will affect the sender's terminal in the manner described by the termios(3) interface. You can read about this interface by entering man termios at the UNIX prompt.

The interrupt {Ctrl-C} character terminates talk, and notifies the other user in the talk session that the talk session has been terminated. The other user can do nothing except exit.

Typing non-printable (control) characters will cause them to be written to the recipient's terminal as follows:

Control characters will appear as a caret (^) followed by the appropriate ASCII character (e.g., control-C is displayed as ^C, and

Characters with the high-order bit set will appear in "meta" notation. For example, \003 is displayed as ^C and \372 is displayed as M-z.

You can deny or grant permission to be a recipient of a talk message through using the mesg utility. Thus, a user's privilege setting could constrain the domain of accessibility of other users' terminals. Certain commands, such as pr disallow messages in order to prevent interference with their output. Also, the talk program will fail when the user lacks the appropriate privileges to perform the requested access and communication.

6.2. Overview of Communications and Sockets

Reading Assignments and Exercises

A UNIX socket is a two-way communications channel or pipe, which can be used to communicate in a wide variety of domains. One of the most common domains for socket communication is the Internet. In this brief discussion, we will summarize the simpler case where sockets are used to communicate between processes on a UNIX computer system. UNIX sockets use many of the same function calls as Internet sockets do, so this discussion can be generalized to Internet communications in a relatively straightforward fashion.

6.2.1. Overview of Sockets

With socket communications, all data transfer occurs through the sockets interface, instead of through the file interface. Although UNIX sockets are a special file type in the UNIX file system, instead of calls to open() and read(), socket programming uses calls to socket(), bind(), and recv().

The sockets programmer creates client and server programs. programs. The server listens for incoming connections from clients, then handles those communications. This is very similar to communication with Internet sockets, but with some subtle differences.

For instance, when describing the path to the UNIX file that represents a socket you want to use), the struct sockaddr_un is employed, which has the following fields:

    struct sockaddr_un {
        unsigned short sun_family;  /* AF_UNIX */
        char sun_path[108];
    }

This structure will be passed to the bind() function, which associates a socket descriptor (a file descriptor) with a specific file, whose filename is in the sun_path field.

6.2.2. Server Functionality

Here follows a brief description of server functionality in UNIX sockets programming. This example is couched in terms of a server that simply echoes its input back to the calling program.

Note that all socket calls return -1 on detection or receipt of an error. Then the global variable errno is set to describe whatever went wrong. Thus, your debugging of a sockets program should include comprehensive error checking, and your code should have full error checking routines, as shown below.

Step 1. Call socket(): A call to socket() with the proper arguments creates the Unix socket, as follows:

    unsigned int s, s2;
    struct sockaddr_un local, remote;
    int len;

    s = socket(AF_UNIX, SOCK_STREAM, 0);

In the preceding code fragment, the second argument, SOCK_STREAM, instructs socket() to create a stream socket. Datagram sockets (SOCK_DGRAM) are supported in the Unix domain, but this discussion is restricted to stream sockets, hence the use of SOCK_DGRAM in the preceding code fragment.

Step 2. Call bind(): A socket descriptor was returned by the call to socket(). Now we must bind that descriptor to an address in the Unix domain, which is a special file on disk. An example code fragment follows:

    local.sun_family = AF_UNIX;  /* local is declared before socket() ^ */
    local.sun_path = "/home/bsimpson/mysocket";
    unlink(local.sun_path);
    len = strlen(local.sun_path) + sizeof(local.sun_family);
    bind(s, (struct sockaddr *)&local, len);

This code fragment associates the socket descriptor "s" with the Unix socket address "/home/bsimpson/mysocket". Observe that unlink() was called before bind() to remove the socket prior to binding it to the socket descriptor. If the socket already exists,an EINVAL error would be returned.

Step 3. Call listen(): This call instructs the socket to listen for incoming connections from client programs, as follows:

    listen(s, 5);

The second argument of listen is 5, which is the number of incoming connections that can be queued before you call accept(), as shown in Step 4, below. If there are five many connections waiting to be accepted, calls by additional clients will generate the error ECONNREFUSED.

Step 4. Call accept(): This will accept a connection from a client, and returns an additional socket descriptor. Here, the old descriptor is still listening for new connections, but the new socket descriptor is connected to the client. An example follows:

    len = sizeof(struct sockaddr_un);
    s2 = accept(s, &remote, &len);

When accept() returns, the remote variable will be filled with the remote process' struct sockaddr_un, and len will be set to its length. The descriptor s2 is connected to the client, and is ready for send() and recv().

Step 5. Handle the connection and loop back to accept(): To communicate to the client and echo information sent by the client, we close the connection then accept() a new connection, as follows:

    while (len = recv(s2, &buf, 100, 0), len > 0)
        send(s2, &buf, len, 0);
    /* loop back to accept() from this point */

Step 6. Close the connection: You can close the connection either by calling close(), or by calling Shutdown().

Here follows example source code for a server that echoes its input, and waits for connection to a UNIX socket (named, in this case, "echo_socket").

    #include <stdio.h>
    #include <stdlib.h>
    #include <errno.h>
    #include <string.h>
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <sys/un.h>

    #define SOCK_PATH "echo_socket"

    int main(void)
    {
        int s, s2, t, len;
        struct sockaddr_un local, remote;
        char str[100];

        if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) {
            perror("socket");
            exit(1);
        }

        local.sun_family = AF_UNIX;
        strcpy(local.sun_path, SOCK_PATH);
        unlink(local.sun_path);
        len = strlen(local.sun_path) + sizeof(local.sun_family);
        if (bind(s, (struct sockaddr *)&local, len) == -1) {
            perror("bind");
            exit(1);
        }

        if (listen(s, 5) == -1) {
            perror("listen");
            exit(1);
        }

        for(;;) {
            int done, n;
            printf("Waiting for a connection...\n");
            t = sizeof(remote);
            if ((s2 = accept(s, (struct sockaddr *)&remote, &t)) == -1) {
                perror("accept");
                exit(1);
            }

            printf("Connected.\n");

            done = 0;
            do {
                n = recv(s2, str, 100, 0);
                if (n <= 0) {
                    if (n < 0) perror("recv");
                    done = 1;
                }

                if (!done) 
                    if (send(s2, str, n, 0) < 0) {
                        perror("send");
                        done = 1;
                    }
            } while (!done);

            close(s2);
        }

        return 0;
    }

The preceding example includes calls to socket(), bind(), listen(), accept(), send() and recv().

6.2.3. Client Functionality

The client is a process that communicates with the server. Design of the client in the "echoing server" applications is much easier due to the absence of listen() and accept() calls. Here follows a stepwise description of client functionality:

Step 1. Call socket(): This creates a UNIX domain socket to use for communication.

Step 2. Create a struct sockaddr_un with a remote address (the socket to which the server is listening) and call connect() with the remote address as an argument

Step 3. Assuming that there were no errors in Steps 1 and 2, you will be connected to the remote side. Use send() and recv() calls to communicate with the server.

Example code for a client process in the "echoing server" application follows:

    #include <stdio.h>
    #include <stdlib.h>
    #include <errno.h>
    #include <string.h>
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <sys/un.h>
    
    #define SOCK_PATH "echo_socket"

    int main(void)
    {
        int s, t, len;
        struct sockaddr_un remote;
        char str[100];

        if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) {
            perror("socket");
            exit(1);
        }

        printf("Trying to connect...\n");

        remote.sun_family = AF_UNIX;
        strcpy(remote.sun_path, SOCK_PATH);
        len = strlen(remote.sun_path) + sizeof(remote.sun_family);
        if (connect(s, (struct sockaddr *)&remote, len) == -1) {
            perror("connect");
            exit(1);
        }

        printf("Connected.\n");

        while(printf("> "), fgets(str, 100, stdin), !feof(stdin)) {
            if (send(s, str, strlen(str), 0) == -1) {
                perror("send");
                exit(1);
            }

            if ((t=recv(s, str, 100, 0)) > 0) {
                str[t] = '\0';
                printf("echo> %s", str);
            } else {
                if (t < 0) perror("recv");
                else printf("Server closed connection\n");
                exit(1);
            }
        }

        close(s);

        return 0;
    }

In the client code, observe that one needs only a few system calls for set up, i.e., socket() and connect(). Since the client isn't going to be accept()ing any incoming connections, a call to listen() is not required. However, the client uses send() and recv() for transferring data.

6.2.4. Full Duplex Communication with socketpair()

It is often desireable to be able to send and receive data simultaneously. This is called full duplex communication, in contrast to just being able to send or receive data at any one time, which is called simplex communication. Since pipes are unidirectional (with a few exceptions in UNIX System V), it is not feasible to implement full duplex communication using pipes.

Fortunately, Unix domain sockets support bi-directional communications using a system call known as socketpair(). This call creates and returns a pair of sockets that are already connected. You can immediately use these socket descriptors for interprocess communication.

For example, consider two processes. The first process sends a character (char) to the second, and the second process changes the character to uppercase and returns it. Here follows some example code, where error checking has been omitted for clarity:

    #include <stdio.h>
    #include <stdlib.h>
    #include <ctype.h>
    #include <unistd.h>
    #include <sys/types.h>
    #include <sys/socket.h>

    int main(void)
    {
        int sv[2]; /* the pair of socket descriptors */
        char buf; /* for data exchange between processes */

        socketpair(AF_UNIX, SOCK_STREAM, 0, sv);

        if (!fork()) {  /* child */
            read(sv[1], &buf, 1);
            printf("child: read '%c'\n", buf);
            buf = toupper(buf);  /* make it uppercase */
            write(sv[1], &buf, 1);
            printf("child: sent '%c'\n", buf);

        } else { /* parent */
            write(sv[0], "b", 1);
            printf("parent: sent 'b'\n");
            read(sv[0], &buf, 1);
            printf("parent: read '%c'\n", buf);
        }
        return 0;
    }

Observe that socketpair() requires both a domain (AF_UNIX) and socket type (SOCK_STREAM). These can be any legal variables or constants. However, the choice of these variables depends on which routines in the UNIX kernel you want to handle the specified code, and whether you want stream or datagram sockets. In the preceding example AF_UNIX sockets were chosen because they are slightly faster on some implementations than AF_INET sockets.

Also, by using write() and read() instead of send() and recv(), one can generally avoid having to specify the flags argument that send() and recv() use.

6.3. Toolkits, including X-Windows and Motif

Reading Assignments and Exercises

The Sun workstations at UF/CISE use various programming interfaces and collections of utilities, respectively called windowing systems and toolkits. Interfaces such as X-Windows or Motif use a bitmap display to draw windows used to communicate with programs. You can manipulate windows using a pointer on the screen (controlled by the mouse).

6.3.1. X-Windows Interface

The X Window system, developed at MIT in the late 1980s, has become an industry standard for graphics workstations. The software is freely available, versatile, and suitable for a wide range of platforms, from high-end microcomputers to mainframes.

In the early days of computing, there were few interactive interfaces with the computer. For commercial users, this situation changed in the early 1980s. Graphical user interfaces became available on personal computers such as the Apple Mac, and products such as Microsoft Windows arrived for the IBM PC. Most new micros coming on the market had windowing systems as standard software.

From personal computers, workstations evolved as more powerful platforms for graphics, engineering, and low-end scientific computing applications. There are nearly a dozen leading workstation manufacturers in the computer market. They offer products that utilize different hardware configurations and different operating systems. Some have proprietary window systems available, others don't. The common denominator since the late 1980s has been X-Windows.

X-Windows is a hardware-independent, network-based window system which was developed at MIT. The source code is freely available for virtually any workstation, mainframe or high-end micro.

Rather than summarize in detail the functionality of X-Windows, here follow two Web links that provide a layered presentation of information, including programming hints, new releases, etc.:

Resource guide includes tutorials, Usenet groups, magazines, tools, official and open-group motifs, widgets, and libraries:

Online X Tutorials A comprehensive listing of X tutorials, some in English, others in European or Asian languages.

X-Windows is a standard programming window system that aspiring UNIX gurus should learn and be familiar with. Occasionally, UF's CISE Department offers a class in X-Windows programming. Check the course listings each semester to determine whether or not that class is being offered.

6.3.2. Motif Interface

OSF/Motif (OSF = Open Systems Foundation) has become the major Graphical User Interface (GUI) technology for Open Systems, as well as a de jure standard (IEEE P1295). The previous version of OSF/Motif (Release 1.2) introduced major new features such as internationalization, drag-and-drop (similar to Microsoft Windows), and tear-off menus. Those features were intended to allow application developers to produce interoperable, easy to use applications for a worldwide market. As a result, this technology has been selected to become the basis of the Common Desktop Environment jointly developed by HP, IBM, Novell and SunSoft, proposed to become an X/Open standard.

Every Motif release contains new features that help the end user community (e.g. drag and drop in 1.2) or the developer's community: programming features that are invisible from the end users but make developer's life much easier (e.g. representation types in 1.2). OSF Motif 2.0 includes items for developers such as the extensibility framework and the uniform transfer model, and extension for end users such as virtual screen support and workspace management. And it also contains new widgets that actually serve both the end user community and the programmers.

For programmers, Motif 2.0 presents the following features:

• Virtual screen support

• Workspace management

• New widgets increasing ease of use and providing more direct manipulation of application objects.

Motif tends to be less prevalent than X-Windows in academic environments. Nevertheless, there are several excellent tutorials on Motif, which have the following links:

• Open Group/Motif home page

• The Motif Developer on-line magazine

• Official OSF/Motif 1.2 manuals on-line (courtesy of SCO)

Additional links are available by searching the Net using keywords such as X windows, motif, and open systems.

6.4. Enhanced UNIX Operating Systems, including Mach

Reading Assignments and Exercises

The following discussion of RT-Mach is an extension of Dr. Raj Rajkumar's recent overview of this topic.

In modern applications of microprocessors, computing is done while data is streaming through the computer system. This is called real time processing. In particular, the need for real-time resource management distinguishes multimedia applications from traditional applications.

In response, many commercial OS products today support a basic fixed-priority preemptive scheduling framework. Future extensions to this basic capability can be expected to support real-time capabilities in multimedia filesystems, X-window services, device drivers and network protocols among others. In the longer term, operating systems should support a distributed resource reservation capability with both spatial and temporal firewalls (security processes) between concurrently executing multimedia and non-real-time applications. Additionally, support should exist for powerful, high-bandwidth networks such as ATM, Myrinet, and Switched Fast Ethernet.

The Real-Time Mach microkernel-based operating system under development at Carnegie-Mellon University focuses on the subject of distributed real-time OS mechanisms and services for developing and supporting real-time and multimedia applications in current and future OSs. The philosophy behind RT-Mach is firmly based on real-time scheduling theory and in particular on priority-driven preemptive scheduling. Here, a given process is assigned a priority. Processes with higher priorities can pre-empt existing, lower-priority processes that are currently executing or are waiting to execute.

RT-Mach extends this philosophy by adding a fundamental OS notion of temporal protection that enables the timing behavior of applications to be isolated from one another. The principles behind this resource management philosophy have many implications in the design of OS subsystems. Associated issues include scheduling policies, synchronization primitives, inter-process communication, filesystems, windowing systems, device drivers, network protocols and communication protocol processing. The capabilities of Real-Time Mach span many of these subsystems while current and future planned work will address the other subsystems.

An excellent summary of Real-Time Mach is at Carnegie-Mellon University's Real-Time and Multimedia Laboratory Home Page. This includes a technical overview, a FAQ list, example distribution, etc.

This concludes our overview of basic UNIX control structures. We next discuss the software development process with a UNIX operating system.

References