The ``software crisis,'' now a household phrase, is particularly severe in concurrent computing for several reasons such as those listed below.
Parallel computers have become widely used in industry and government over a short span of time, and so there is pressure to develop a large number of parallel applications over a short period of time. This telescoping of the time available for parallel program development makes productivity of parallel programming even more important than productivity of sequential programming.
Developing correct parallel programs is even more difficult than developing correct sequential programs because programmers may have to deal with nondeterminacy and multiple threads of execution. Nondeterminacy and concurrent threads are difficult to deal with both for formal reasoning and for debugging. The issue of productivity of parallel programs is exacerbated by the difficulty of reasoning about parallel programs.
Optimizing a program to obtain adequate performance is simpler for a sequential program than for a parallel program. The diversity of parallel architectures and the ranges of performance parameters (such as message latency) make optimization of parallel programs difficulty and machine-specific. Since parallel programs are developed because they execute faster than sequential programs, efficiency is critical. The need for efficiency of program execution is another reason that achieving programmer productivity is more difficult for parallel programming than for sequential programming.
There is no standard in hardware. Shared-memory multiprocessors, networks of workstations, and data-parallel machines compete for the attention of applications developers. Though standards for software (such as HPF) are developing, they have been slow in coming. The development of CASE tools proceeds more rapidly when there are a small number of industry standards. The inadequacy and wide diversity of tools hinders productivity.
One approach to improving programmer productivity is reuse of programs. Though program libraries help a great deal, they are inadequate for dealing with the difficulties of parallel programming for several reasons including the following.
A program library is written in one language using one runtime system. Given the range of architectures, and the large number of sequential applications that need to be ported to parallel machines, a library in one language and using one runtime system is inadequate. There is insufficient time to develop libraries in all the parallel programming languages and runtime systems that are available.
Many applications developers are (quite properly) conservative, and do not want to commit to a significant amount of effort to develop a parallel application if their effort is wasted because some machines, languages or runtime systems are no longer supported. The risk of wasted effort is much lower in application development for sequential machines than for parallel machines, especially since de facto industry standards are emerging such as C, C++, Fortran and object interfaces such as OLE, OpenDoc, and CORBA. Commitment to a single parallel library, language and runtime system is a risk.
In sequential programming, there is only one form of composition: sequential composition. Furthermore, all the data of a sequential program is mapped on a single address space. So, all we need to be concerned about in designing a sequential library is whether its routines can be called in some sequence, and whether routines deal with the data structures used in the program.
In parallel programming, we have to be concerned with sequential composition and parallel composition: We may want procedures in the library to cooperate in a concurrent execution. This can be difficult because each parallel procedure may expect a specific distribution of its data across the memory of the machine. The difficulty of composing procedures in a library makes libraries more difficult to use in parallel programming than sequential programming.