White Papers:

Use of the X Window System to Meet Special Operator Console Requirements

Eric Wogsberg
Jupiter Systems

3073 Teagarden Street
San Leandro, CA 94577
(510) 667-9000
An earlier version of this white paper was published in ISA proceedings 1994.

X Window System, Open Systems, Operator Interface, Control Systems X Terminal, Workstation, Personal Computer, Ethernet, Interoperability Multiple Displays, Fault Tolerance, Redundancy, Real-Time, Multimedia

Background is provided on the X Window System, its development and adoption by the computing community, the X client/server model and the role of the X Window System in Open Systems. The special Operator Interface requirements of Control Systems are then discussed and it is shown how readily available X Window System hardware and software components can be used to satisfy these requirements.

1.  Introduction

2.  Design Goals of the X Window System

3.  Key Features of the X Window System

4.  Market Activity of the X Window System

5.  Special Requirements of Operators' Consoles

6.  Pixel Real Estate
7.  Group/Collaborative Activities

8.  Redundant Networks

9.  Deterministic Performance

10.  Multimedia - Audio and Video

11.  Conclusions

12.  References

The X Window System was created at MIT as part of Project Athena, a major, multi-year project to evaluate the future of computers in higher education. Several critical problems were delineated and the X Window System's design goals were selected to solve these problems. As a result of design decisions made at MIT and also as a result of policy decisions regarding licensing and maintenance of the X Window System code, the X Window System has had tremendous commercial impact.
This commercial success has led to the availability of a broad and diverse set of products incorporating many features which are especially useful in operator consoles. This paper will explore the characteristics of the X Window System that have given rise to the success and diversity of X products and will show how some of the capabilities available now can be used in meeting the special requirements of operator consoles.

 2.  Design Goals of the X Window System
The impact of the X Window System has grown far beyond what was expected by its original creators. The reasons for this success were the result of excellent problem analysis, excellent design and implementation, and some key policy decisions. This success, of course, would not have occurred if there hadn't been a tremendous need in the computer industry for a comprehensive graphics and windowing standard.

Ten years ago Project Athena analyzed how computers were being used on the MIT campus, and it looked forward to see how computers might be used in the future. There were serious impediments to the use of computers at that time and the researchers could see that these problems would just get worse as computers and applications proliferated. The X Window System was seen as a way to avoid some of these problems. So X was developed to achieve three important goals:

* To allow connection and use of hardware from multiple vendors in a single network;

* To allow the campus computing system to grow incrementally and without limit as applications increase and as the size of the user community increases;

* To preserve the investment in educational software and training in the face of rapidly changing technology.

These goals are just as significant to business and industry as they are to academia.

 3.  Key Features of the X Window System
First a word about client/server terminology: in the PC (personal computer) world client and user are thought to be synonymous but that is not the case with X. In the X world Client refers very specifically to the application program. This application program (Client) needs services to accomplish anything. It needs a compute server to execute it, it needs a file server or database server to store data, and it may need a print server for hard copy. If it must interact with a user then it also needs a user interface server. In a standalone computer all these services will be provided by the same box. In a networked environment, each of these services may be provided by separate devices.

The X Window System is specifically a user interface capability. Thus an X Server is a user interface device typically consisting of a display, a keyboard and a pointing device such as a mouse. PCs and workstations can be used as X Servers and they can also execute X clients (application programs). X terminals are strictly user interface devices (X Servers) and do not execute application programs.

The key features of the X Window System are:

* Network Transparent - the X client does not have to be executing on the same device where the user is sitting. Thus a user, sitting at an X Server, can access application programs running anywhere on the network.

* Hardware Independent - the X Window System runs on any kind of processor and will work with any display hardware.

* Software Independent - X runs under many different operating systems and can be used from most programming languages.

* Network Independent - X can communicate over any mechanism that provides a reliable duplex byte stream. Thus many different media and protocols can be used.

* Comprehensive Display Capabilities - X provides the display capabilities most applications require such as multifont characters (including international fonts), 2D graphics, images, and, of course, windows.

* Extensibility - the X protocol can be extended to meet special needs without affecting the base-level capabilities. PEX for 3D graphics and XIE for image processing are two major extensions provided by the X Consortium but the protocol can also be extended in small ways to solve relatively simple problems like blink.

These features give the X Window System a high degree of functionality and interoperability.
MIT made a key policy decision in 1985 to not charge for the X Window System code and to make it available to anyone who wanted it merely for the cost of distribution. Needless to say, this has certainly helped promote the use of X.
Another key event in X history was the formation, in 1988, of the X Consortium. The Consortium consists of about 40 companies (membership is open, anyone can join) which provide funding to support a small technical team which maintains, enhances and distributes the X Window System. This ensures that X will continue to develop in a vendor-neutral fashion and will be readily available to all parties, whether or not they are members of the Consortium.

 4.  Market Activity of the X Window System
The wide adoption of the X Window System by the computer industry has led to a great deal of market activity. In 1993 over 250,000 X terminals and over 300,000 X Server software licenses for PCs were sold. About 650,000 workstations using X were sold. This represents well over $10 billion in revenue for X platforms and server software. Revenue for application software based on X would be in addition to this as well as revenue for supercomputers, mainframes, minicomputers and PCs acting as compute servers for X clients.

At its most basic level the X Window System is a specification of the interface between an application program and the hardware (keyboard, mouse and monitor) used to interface with the human user of that application program. This interface specification allows computer equipment from different manufacturers to interoperate - one computer executes the application program while another computer or terminal provides the user interactivity for that program. Because the X Window system provides this interoperability it has led to commoditization of user interface hardware in much the same way that the SCSI and IDE interface specifications have led to commoditization of disk drives. This has been demonstrated especially well in the X terminal market. The predecessor market was for graphics terminals which were produced in low volume, had proprietary interfaces and protocols, were expensive, had low general utility and therefore had low perceived value. The X Window System coerced manufacturers to move away from their unique proprietary systems. Because of the high utility of X Window terminals there was strong demand and manufacturers could move into higher levels of mass production and achieve economies of scale in production and distribution that were heretofore unknown. But the interchangeability of product from different vendors forced manufacturers to sell their products at low margins. This gave the user good perceived value.

We are now entering a second phase of market activity where commoditization is giving way to differentiation. Manufacturers have mastered the basic features and now are looking for ways to differentiate their products. This differentiation is made possible by the extensibility of the X protocol and is made necessary by users who have special requirements for their computer/human interface. Luckily, due to the architecture of the X Window System, there is no need to give up basic features and functionality when seeking special capabilities.

 5.  Special Requirements of Operators' Consoles
Ever increasing demands for improved plant operation necessitate continuing evolution of the operator interface. A big step forward was made with the change to the soft console (i.e. computer driven displays) from hard-wired instrument/switch panels. Graphical user interface technology was a big improvement over character-oriented displays. But what are the next steps in the evolution of the operator interface? There are five areas where emerging technology, now readily available under the X Window System, can contribute greatly to improving the operator interface. These areas are:

* Providing adequate screen space to display large amounts of information;

* Sharing of information for collaboration, supervision or training;
* Providing network redundancy for communications fault tolerance;

* Providing deterministic performance;

* Providing multimedia capability - audio and video.

Keep in mind that not all of the technology discussed herein is part of the official X Window System standard. The extensibility of X allows individual vendors and users to expand the capabilities of X without oversight from the X Consortium. Later, extensions which prove to be generally useful can be adopted by the Consortium and made a part of the official X standard.

 6.  Display of Large Amounts of Information
One requirement that is common to a wide variety of operator consoles is the need to view at one time a lot of data - more data than will fit on a single display. Various techniques have been used to deal with this problem.
A raster scan display is made up of a rectangular array of "picture elements" or pixels. The total number of pixels available represents an upper bound to the amount of information that can be displayed. Traditionally (prior to the availability of windowed displays) there were only two choices: either make the displayed information more compact so it would fit into a smaller number of pixels or improve the resolution of the display. With windowed displays additional techniques have become available such as overlapping windows and iconified windows (windows that have been reduced in size to symbolic or icon representation).
But problems exist with all of these techniques. Compaction can make the information too small or too dense to read easily. There is a limit to the resolution (and therefore the number of pixels) you can get on a reasonably priced monitor.
Resolution Pixels Screen size

640x480 0.3M
1024x768 0.8M 15, 17

1152x900 1.0M 17, 19

1280x1024 1.3M 17, 19

Display Resolutions

But even 1.3 megapixels frequently isn't enough screen real estate and the next size larger monitor (20" square monitor with 2Kx2K resolution) costs about $40K - an order of magnitude higher cost per pixel. The problem with overlapping or iconified windows is that operator action is required to make them visible, time to redraw is required, and bringing up an additional window will obscure other information that is already displayed.

Figure 1:  a three-screen X Window terminal

The only good solution to the real estate problem is to take multiple monitors and operate them in parallel. This can be done by setting displays from multiple PCs, workstations or X terminals side-by-side. A better way is to integrate multiple displays on a single platform so one mouse and keyboard can be used to interact with all the displays. Figure 1 shows a three-screen X Window terminal. Each screen has 1280x1024 resolution for a total of 3.9 megapixels. The cursor will move from screen to screen as the mouse is moved and windows can be picked up on one screen and moved to another. Windows can also be resized to overlap screen boundaries if desired. This display works with absolutely standard X Window applications - no special coding is required to take advantage of multiple screens.

 7.  Group/Collaborative Activities
Another requirement that occurs more and more frequently today is the need to share information among operators. Sometimes it may be necessary to share data just between two operators while at other times several operators may be involved. In some situations the shared data may be read-only while in others an operator will be manipulating the data individually or in collaboration with others.

One approach to sharing information visually would be to replicate the data on each operator's console. This works fine for a relatively small amount of data and a small number of operators. If there is a large amount of data to share or if it must be shared among a large number of operators then large-screen displays may be the best solution.

Rear-screen projectors with 67 to 120 inch diagonal measurement can be used to display data for many operators to view simultaneously. Multiple projectors can be used to create massive display walls. Some installations in the transportation and telecommunications sectors have over 100 projectors in a single control room. Figure 2 shows a display wall using six 67" projectors in a 3x2 array. Each projector has 1280x1024 resolution so the total display amounts to eight million pixels.

Figure 2:  a display wall using six 67" projectors in a 3x2 array

Building display walls with projectors is a hardware intensive solution to sharing data. Not only is it expensive but it only works when the people sharing the data are all in the same room. What if it is necessary to share information dynamically with someone in the next room or the next building or even the next continent?

The solution to this problem could be a software capability known as SharedX which allows a user to send a copy of one or more windows from his X Server to another X Server. Then both users can interact with the Client program. This technique can be extended like a party line to include more participants. The Client can be accessed via the LAN for collaborators at the same site, or over a WAN for collaborators who are geographically distant.
Some important characteristics of SharedX1 are:

* No changes are required to standard X applications;

* Running applications can be shared at any time, they do not need to be restarted to be shared;

* Receiving X Servers need no modifications to participate. Any X Server will do.

* Receivers of a shared application can interact with it, as well as the sender.

SharedX is very handy in training situations where the trainer can passively monitor every action the trainee makes without being physically present and looking over the trainee's shoulder.

SharedX can be used in ad hoc situations, too. Say an unusual circumstance arises and the operator is not sure how to handle it. He can share his display with his supervisor or the plant engineer and together they can evaluate the situation and decide on the proper action to take. This can even be done over a phone line connection if the advisor is not on site.

Another possible use for SharedX is to report bugs to the supplier of the control system you are using.
SharedX was developed privately and is currently only available from one company. However a similar capability has been developed independently2 and will be made available to the X community as contributed software.

 8.  Redundant Networks
Each component of a distributed computing system must be designed to continue operating in a safe manner if the LAN goes down and communication with the other components is lost. But that approach provides only a base-level fail-safe operation.

A significant improvement over this can be achieved with minimal effort by using redundant LANs. The hardware required is simply a second Ethernet interface for each unit which is to be included in this scheme and a second Ethernet cable interconnecting these extra interfaces. Figure 3 shows the LAN communications for a control room based on two centralized control computers. Four Ethernets are used - two for redundant paths to the Front End Processors and two for redundant paths to the console clusters and other services within the control room.

Figure 4 shows a detail of one console cluster which consists of a compute server running X clients and seven multi-screen X Window terminals with dual Ethernet connections. Bridges are provided at each connection to the control room nets to isolate local traffic.

The software can be relatively simple if DECnet is available as an option. Although DECnet was developed years ago by Digital Equipment Corporation as a proprietary networking protocol, it has become enough of a standard that it is available as an option for most PCs, workstations and X terminals.

But why DECnet rather than the more ubiquitous TCP/IP? It's very simple. TCP/IP requires a separate logical address for each Ethernet interface on the LAN whereas DECnet uses one logical address for all the Ethernet interfaces on a given node. Furthermore, DECnet software maintains routing tables which show if multiple paths are available between two nodes. Once a path is established between two nodes DECnet continues to use that path. But when that path is interrupted DECnet goes to its routing tables and, if an alternate route is available, will begin communicating over that path. This happens automatically. The user doesn't have to take any action and the Client program doesn't have to take any action. Switchover can occur in a matter of seconds so unless an operator is very alert he probably won't even realize when a switchover occurs.

TCP/IP can also be used in a failover scenario but it is more complex. The Client program has to be aware of the paths between nodes, it has to recognize when the active path has failed, and it has to initiate use of the other path. Thus this approach will not work with naive Clients.

 9.  Deterministic Performance
The X Window System, for all its strong points, does have some shortcomings for mission critical applications such as a process operator's console. The shortcomings arise from the fact that an X Server typically services a number of different Clients. Each Client is autonomous and can place arbitrarily heavy demands on the Server.

Once the Server starts processing a buffer load of commands from a Client it disregards requests for service from other Clients until the first Client's buffer is exhausted. Typically a buffer of graphics rendering commands can be dispatched in a fraction of a second. The Server then checks all other Clients, in round-robin fashion, to see if they have commands to be processed. This technique works fine in a bounded system where all Clients are known to be "polite" and do not, under any circumstances, hog the Server.

But if a Client sends a large buffer of slow graphics commands a delay of several seconds could occur. The worst case scenario is when a Client demands a response from an inattentive operator. In this case the delay can be arbitrarily long.
One solution to this and related problems is RealTimeX, a software product that has been developed privately over a period of years. RealTimeX3 provides priorities, preemption and predictability, concepts that have been available in real-time operating systems for many years but which have only recently become available in a multi-tasking display environment. The key features of RealTimeX are:

* Window priorities - priorities can be assigned to indicate importance of various windows so the important windows cannot be iconified or moved off-screen or obscured by less important windows;

* Client priorities - instead of a simple round-robin scheduler RealTimeX provides a hierarchy of priorities to determine how the Server handles contention among Clients;

* Scheduling control - timeslices can be set to control the amount of time the Server spends on one Client before switching to the next Client;
* Preemption - a lower priority Client can be interrupted by a higher priority Client;

* Predictability - markers are provided to build synchronization and scheduling models;

RealTimeX, as a proprietary offering from a single vendor, is not part of the X Window System standard. There is a comparable implementation4 from another vendor with even broader functionality. This product, named TREX, has been offered to the X Consortium as the starting point for a standard real-time extension to X. The offer is currently under consideration.

 10.  Multimedia - Audio and Video
NETaudio5 is a comprehensive standard for playing, recording and manipulating audio data over a network. It uses the X client/server model to isolate application code from the hardware (and hardware-specific software) used for audio input and output on the PC, workstation or X terminal. NETaudio will soon be made a part of the X standard.
NETaudio provides the following capabilities:

* Audio data can be sent from the application to the desktop device for playback.

* If a microphone is available, audio data can be recorded and transmitted back to the application.

* Audio data can be stored in the Server for later playback. Thus commonly used sounds can be played without sending them over the network each time.

* Multiple sources of audio data can be mixed and manipulated in a variety of ways, including volume adjustment, mixing, and changing sampling rate or precision.

* Various digital audio formats are supported and conversion is performed automatically in the Server.
The data rates required for audio signals range from 8KB/second for telephone-quality monaural to 176KB/second for CD-quality stereo. These data rates are well within the capability of Ethernet and most other LANs.

Video is a more difficult problem to solve. Data rates for full-frame, full-motion color video are over 10MB/second. Even with use of compression/decompression technology video will saturate most LANs available today. Furthermore the asynchronous LANs in widespread use today will not deliver data at a guaranteed smooth flow rate so the resultant image will be jumpy. Newer networking technologies such as ATM (Asynchronous Transfer Mode) provide higher bandwidth and the isochronous services needed for video. It will be a year or two before these are widely deployed due to the current high cost and limited availability of interfaces.

Most control room applications for video involve monitoring closed circuit cameras. In general it is not a problem to bring the analog signal from the camera directly to the console. This obviates the problem of delivering video digitally over the network.

One early method of incorporating live video into a windowed display used an external device to overlay the video into a chroma-keyed window (Figure 5). No changes are required in the X Server's display hardware. The Server puts up a rectangular window of a certain solid color and the video overlay box patches the captured video into that region of the screen. The rest of the pixels generated by the X Server pass through to the screen unchanged. New versions of this type of device can display four live video windows scaled to any size.

Figure 6:  pixels as drawn by the Server on command from the various Clients

A newer technique, based on digital technology within the X Server, has recently become available at very reasonable cost. In a Server's display controller there is usually one frame buffer which holds the pixel data to be displayed on the monitor. These are the pixels as drawn by the Server on command from the various Clients (Figure 6). The pixel data in the frame buffer go into a device called a RAMDAC (Random Access Memory/Digital to Analog Converter) which generates the Red, Green and Blue analog signals that drive the monitor.

Figure 7:  video data and the rendered data are combined by a special RAMDAC
It is difficult to bring digital video into this frame buffer because of the bandwidth required, color table conflicts, and overlapping windows. However it is possible to set up a parallel frame buffer just for the digitized video data. The video data and the rendered data are combined by a special RAMDAC that has separate color tables for the two data streams (Figure 7). This video window can be positioned anywhere on the screen and can be scaled up to fill the whole screen or scaled down to a very small window.
The software which provides the control functions necessary to manipulate and control live video under the X Window System has not yet been made part of the standard. There are two competing proposals (Xv and MVEX). With either of these software packages the video window can be scaled to any size, color parameters (hue, brightness, contrast, saturation) can be adjusted, and different video sources can be selected.

 11.  Conclusions
The key features of the X Window System (hardware and software independence, network transparency, extensibility) make it an ideal environment in which to incorporate emerging technology. The public availability of the X Window System source code has fostered a vendor and user community that actively develops new capabilities which frequently are contributed to the public domain. The X Consortium actively develops new capabilities, establishes standards when appropriate and coordinates dissemination of contributed software.

These activities keep the X Window System at the state-of-the-art in user interface technology and ensure that new capabilities will be available from multiple vendors at reasonable prices.

Special operator interface requirements of control systems can easily be met within the structure of the X Window System and can be met in a manner which is robust, flexible and economical.

12.  References
1. D. Garfinkel, B. C. Welti, and T. W. Yip, "HP SharedX: A Tool for Real-Time Collaboration," Hewlett-Packard Journal, April 1994, pg. 23-36.
2. C. Bormann, and G. Hoffmann, "Xmc and Xy - Scalable Window Sharing and Mobility," The X Resource, Issue 9, January 1994, pp. 205-210.
3. S. Black and R. Chesler, "Real-Time Extensions to X Windows Provide Deterministic Graphics," Computer Technology Review, Volume XIII, Number 6, May 1993, pp. 27-31.

4. J. Malacarne, "Can You Bet Your Life on X? Using the X Window System for Command and Control Displays," The X Resource, Issue 1, Winter 1992, pp. 7-25.

5. J. Fulton, and G. Renda, "NETaudio: Make Your Applications Sing (As Well As Dance)," The X Resource, Issue 9, January 1994, pp. 181-194.

 Top of Page

News Reprints


News Directory


©Copyright 1997-1999, 2000-2001 Jupiter Systems, Inc. All rights reserved.