Numerical Analysis:
Mathematics of Scientific Computing, 3rd Edition
David Kincaid & Ward Cheney
Appendix: An Overview of Mathematical Software
A tremendous amount of mathematical software is available
on the World Wide Web (www),
and more is being developed each day.
To find the most uptodate information,
one can execute a search of the Internet
for available mathematical software in a particular
application area of interest.
It is helpful to classify mathematical software into three categories:
(a) public domain
(b) freely accessible (some usage restrictions apply)
(c) proprietary (license agreement required)
Public domain has a specific legal meaning, implying that any use is permitted,
including modification, resale, and so on. From the Internet, one can download
either public domain or freely accessible software, and in many cases, one can
obtain free demonstration copies of commercial software packages. Some of the
software on the Internet has usage restrictions imposed by the authors, such
as copyright, that allow unrestricted use for research and educational purposes
only. (An example of this is code from the
ACM Collected Algorithms,
which is subject to the ACM Software Copyright and License Agreement.) On the
other hand, proprietary software must be purchased or leased from the developing individual, company, or from a computer store that sells software. Notice that whether money
changes hands before the software can be used is not addressed in this breakdown
of software into categories.
For example, consider these seemingly contradictory examples:
(a) paying for public domain software (Netlib on CDROM)
(b) paying for accessible software: you can freely download some
software, but you had better send in a check, if your use it for noneducational purposes
(c) software that is given away for free, but is proprietary in the
sense that a license agreement must be signed before you can get it
In the following, we give a quick overview of some available
mathematical software with links to the wwwaddresses where additional
information and some of the software can be found.
Also mentioned are several systems for helping in the search
for mathematical software that solves specific problems.
This is not a complete and comprehensive listing since
developments are proceeding at such a rapid rate that any listing is soon
out of date!
Since mathematical software is written in a variety of different programming
languages and for a wide range of different computer architectures, it is difficult
to know how to organize the discussion. We are guided by an excellent overview
of mathematical special functions by Lozier and Olver [1994]. They organize
software into the categories of
Software Packages,
Intermediate Libraries,
Comprehensive Libraries, and
Interactive Systems.
First, Software Packages
contain one or more subroutines for solving a particular problem in a subfield of numerical mathematics.
Second, Intermediate Libraries are usually collections of subprograms for use on small computers or PCs. Some of these libraries contain a collection of useful mathematical functions in one or more computer languages. They may have been written by the manufacturer of the computer equipment or the developer of the compiler. Also, some intermediate libraries are available for PCs that are subsets of generalpurpose mathematical libraries.
Third, Comprehensive Libraries contain subroutines that have been assimilated into highquality software containing many unifying features such as uniform documentation, uniform style of usage, and errorhandling conditions.
Finally, Interactive Systems are fully interactive computer environments with a powerful set of keyboard commands so that the user can avoid the compilelinkexecute cycle. Usually there is an evolutionary process that mathematical software goes through from the introduction of the original idea in a journal or report to the general acceptance by the scientific community of the algorithm. Eventually, the algorithm is incorporated into a large mathematical library or a sophisticated interactive computer system.
We start by mentioning some searching systems for finding
available mathematical software on the Internet
and then giving general information related to mathematical software
research and developments.
Finally, we discuss a number of comprehensive libraries,
interactive systems, and
some examples of intermediate libraries or mathematical software packages.
Searching Systems

Guide to Available Mathematical Software
(GAMS)
 is a large online and crossindexed virtual repository of mathematical and statistical software for use in computational science and engineering. It was developed as a
National Institute of Standards and Technology (NIST)
project for providing scientists and engineers with improved access to reusable computer software. The user is guided through a problem decision tree to search for appropriate software for the particular problem to be solved or for the package/module name. Rather than providing a physical repository, the guide provides transparent access to multiple repositories maintained by NIST and others. One can obtain abstracts, documentation, and source code.
Also, a Java interface to GAMS
is available with additional search features.
Other sources
of information for GAMS mathematical software includes
directories, journals, repositories, freely available
packages, software vendors, and educational software vendors.

Netlib
 is a repository for mathematical software, documents (papers, reports, etc.), databases (conferences, performance data, etc.), and other useful mathematical information.
There is a keyword searching capability for obtaining mathematical software as well as for searching back issues of the weekly
NADigest
online newsletter. This is a system developed to serve the community of numerical analysts and other researchers.
Also, the NANet Home Page
serves the scientific computing community by providing a White Pages directory service.
Some other related efforts are the following.

HPCNetlib
is a high performance branch of Netlib.
It provides information about high performance mathematical software,
whether research or commercial, as well as giving a roadmap to issues
such as software selection and performance.

Matrix Market
provides a convenient access to a repository of test data for use
in comparative studies of numerical linear algebra algorithms.

PTLib
is a source of information about high quality software
and tools for parallel systems.
It is part of the
National High Performance Computing and Communication Software Exchange
(NHSE)
General Information
 Conferences
 are announced and information on them are available on the Internet. For example, the
Netlib Conference Database
contains information about upcoming conferences, lectures, and other
meetings relevant to the fields of mathematics and computer sciences. Also, the
Atlas Mathematical Conferences Abstracts
provides conference announcements
and conference abstracts for mathematicians worldwide.
Conferences are announced in the
NADigest
and other online newsletters.
Many conference announcements are available directly from scientific societies and associations such as those listed below.
 Graduate Programs
 involving numerical analysis and scientific computing exist throughout
the world.
Information on these graduate programs is available on the
World Wide Web.
For example, see the
Computational and Applied Mathematics (CAM)
program at
The University of Texas at Austin.
 Webpages
 have been established by various research interest groups. An example of some of these are:
Individuals have webpages such as the authors of this textbook:
Many professional societies and associations have webpages such as those
listed below.
 Journals
 are published for the dissemination of recent research developments and associated algorithms and mathematical software. The bibliography to this book contains a listing of many of the primary numerical analysis journals. For some of these journals, tables of contents, and even fulltext articles are available to readers over the Internet. Some of the organizations that publish research journals and disseminate mathematical software are:
For example, the
ACM Transactions on Mathematical Software (TOMS)
publishes refereed articles and computer routines/packages.
The ACM algorithm policy requires the software to be selfcontained with
adequate documentation, to have a test program with sample output, and to
be reasonably portable over a variety of different computers.
The TOMS webpage has a searchable table of contents to the algorithm
papers and links to the software. The software is available from
ACM, Netlib, and many other sources.
You can obtain information from the
ACM Collected Algorithms.
A classification system is used for indexing the algorithms,
and a data base of them is maintained.
The ACM Digital Library is a resource of
bibliographic information, citations, and fulltext articles.
The bibliographic database is free of charge to visitors by registering,
and access to the fulltext database is a payperview subscription
service. Many new and classical mathematical books are available
at SIAM Books.
Alos, mathematical journals such as
SIAM Journal of Numerical Analysis,
SIAM Journal of Scientific Computing,
and many others are available at
SIAM Journals.
Recently, entirely electronic journals have appeared with technical
articles related to the development of numerical analysis
algorithms. Articles from these journals are available over the Internet
for local printing. One such journal is the
Electronic Transactions on Numerical Analysis.
Other journals exist for the exchange of software and related information
in particular scientific disciplines.

Mathematics Archive WWW Server

provides an organized Internet access to a wide variety of mathematical
resources. Primary emphasis is on material that is useful in teaching mathematics and educational software.

Mathematics Information Servers

is an extensive list of mathematics servers on the World Wide Web with
references to many academic departments, electronic journals, sources for
preprints of articles, and information on mathematical software.
 News Groups

are used for a wide range of discussions and questionsandanswers postings in an unmoderated forum. Some USENET news groups for mathematical software are sci.math.numanalysis, sci.math.research,
sci.math.symbolic.
 Newsletters
 are available on the Internet with announcements and general information on mathematical software. Examples of these newsletters are

NADigest is a numerical analysis online newsletter.

MGNetDigest
is a multigrid online newsletter.

ATNet is an approximation theory network and online bulletin.
 Research
 centers and institutes on numerical analysis and scientific computing exist throughout the world such as the
Center for Numerical Analysis (CNA)
in the
Institute for Computational Engineering and Sciences (ICES)
and the
Texas Advanced Computation Center (TACC) at
The University of Texas at Austin.
In the United States of America, large collections of mathematical software
are available from national laboratories
[ Argonne (ANL),
Fermilab,
Lawrence Berkeley (LBL),
Lawrence Livermore (LLNL),
Los Alamos (LANL),
Oak Ridge (ORNL),
Pacific Northwest (PNL),
Sandia,
Stanford Linear Accelerator,
etc.],
national supercomputer centers [
San Diego Supercomputer Center (SDSC),
National Research Scientific Computer Center (NERSC),
etc.] and various government agencies
[ National Institute of Standards and Technology (NIST),
etc.]. Links are available
directly to all of the national laboratories, supercomputer centers, and high
performance research centers.
 Textbooks
 with associated mathematical software are widely available. Many numerical analysis and numerical methods textbooks come with or have associated software. In these books, one may find for each problem area a general discussion of analytical mathematics, the presentation of algorithms, and perhaps the actual implementation of them into computer routines written in one or more computer languages. The algorithms may be listed in the textbook, or the software may
be available for purchase on a diskette or freely available so that the interested reader can download it from the Internet. For example, software supporting this textbook is available by using a Web browser and accessing the files at the following link:
Numerical Analysis: Mathematics of Scientific Computing
by David Kincaid and Ward Cheney. Here one can download sample computer programs that are written in various computer programming languages or that use an advanced mathematical software system. Also available is an uptodate list of errata for this textbook. Also, software is available for
supporting our elementary textbook Numerical Mathematics and Computing, 6th Edition by Ward Cheney and David Kincaid.
Comprehensive Libraries
Comprehensive libraries are large collections of mathematical routines
all written in a uniform style and with high standards of
quality and robustness. Some of these libraries are listed here.

BCSLIB/BCSEXT

is the standard mathematical and statistical subroutine library
developed by Boeing Computer Services.
It contains approximately 250 subroutines for applications in a wide
variety of areas.
BCSEXT is the Boeing Computer Services Extended Library, containing routines using outofcore methods for solving large linear algebraic problems.
Applications include large (sparse or dense) linear systems and
eigenvalue problems as well as fast Fourier transform in two and
three dimensions.

CMLIB
 is the Core Mathematics LIBrary of the
National Institute of Science and Technology (NIST).
It is a collection of approximately 750 highquality publicdomain Fortran subprograms that are easily transportable. The subroutines in this library solve many of the standard problems in mathematics and statistics. It contains mostly externally available software programs such as BLAS, EISPACK, FISHPACK, FCNPACK, FITPACK, LINPACK, and QUADPACK.

CERN Library
 is maintained by the European Laboratory for Particle Physics. This library is primarily for the support of highenergy physics research, but it contains many routines for general mathematical use. With some restrictions, the library is distributed to outside organizations.

ESSL/PESSL
 is an Engineering and Scientific Subroutine Library for use on IBM computers. It is a stateoftheart collection of over 450 mathematical routines for use on a wide range of IBM computers for solving scientific and engineering applications. The library has been tuned for specific IBM computer architectures such as workstations and parallel computers. It can be called from applications written in Fortran, C, or C++. PESSL is the parallel implementation of the IBM Engineering and Scientific Subroutine Library (ESSL). It consists of scalable mathematical routines that support parallel processing on IBM computer systems and clusters of IBM workstations. Parallel ESSL supports the Single Program Multiple Data (SPMD) programming model using either the Message Passing Interface (MPI) signal handling library or the MPI threaded library.

LibSci
 is a library of commonly used mathematical and scientific routines developed for use on Cray supercomputer systems. For example, it includes routines for linear algebra, fast Fourier transforms, filtering, packing unpacking, and vector gather scatter.

IMSL Libraries
 are Ccoded or Fortrancoded numerical and graphical libraries developed by Visual Numerics, Inc. These libraries contain a large collection of subroutines and function subprograms (over 500) that provide access to highquality implementations of numerical methods in mathematics and statistics. They have evolved over more than thirty years. These libraries are available for use on a wide range of computer platforms with subsets of them available for use on PCs. Other mathematical software products are available from Visual Numerics, such as PVWAVE and Stanford Graphics.

NAG Libraries
 are Fortran77/90coded or Ccoded numerical and statistical libraries for scientists, engineers, researchers, and software developers with applications involving mathematics, statistics, and optimization. Developed by the Numerical Algorithms Group (NAG), these software libraries have comprehensive numerical capabilities and were coded with the collaboration of numerical and statistical specialists. The largest version of the library has over 1000 routines. Some of this software has evolved over more than twenty years, producing stateoftheart products with robust performance on over eighty computing platforms from PCs to supercomputers. Also, NAG developed the first fully standard Fortran 90 compiler, and the computer algebra system Axiom is available from NAG.

SLATEC
 is a large collection of Fortran mathematical subprograms distributed by the
Department of Energy (DOE) Energy Science and Technology Software Center.
It is characterized by portability, good numerical technology, good documentation, robustness, and quality assurance. The primary impetus for this library was to provide portable, nonproprietary, mathematical software for supercomputers at a consortium of government research laboratories. The original acronym stood for the national laboratories involved (
Sandia,
Los Alamos,
Air Force Technical Exchange Committee). Subsequently, the library committee admitted three additional national laboratories (
Lawrence Livermore (LLNL),
Oak Ridge,
Sandia Livermore) plus the
National Energy Research Supercomputer Center and the
National Institute of Standards and Technology (NIST).
Also see
SLATEC Fortran 90 version
In addition to those listed above, some other general mathematical software libraries are NSWC, NUMAL, NUMPAC, PORT, Scientific Desk, VECLIB, and
more.
Interactive Systems
In general, a fully interactive mathematical software system contains a powerful set of commands that the user enters at a computer terminal or workstation by using the keyboard or the mouse to click on an icon. An immediate response is displayed on the screen. It may be a computation (numerical or symbolic) or a visual display such as a graph or figure. The programming burden is reduced since there is no compilelinkexecute cycle. The capabilities of interactive systems can be extended by customizing the set of commands or icons. Interactive systems are able to integrate nonnumerical tasks with numerical computations. It seems that graphical and symbolic computing are best done in an interactive computing environment. For this reason, a recent trend is combining numerical computing with symbolic computing and graphical visualization into a totally interactive system.
Computer algebra systems have special capabilities useful in numerical mathematics. One such feature is arbitrary precision or
multipleprecision floatingpoint arithmetic.
In general, programming languages use the computer hardware
for doing computer arithmetic so that the precision is fixed.
The primary purpose of
computer algebra systems is for exact mathematical calculations
with floatingpoint computations a secondary capability.
Nevertheless, one obtains a bonus of being able to carry out
arbitrary precision floatingpoint computations in these systems.
Unless the user specifies otherwise, symbolic systems generally
avoid evaluations that introduce inexact results and
evaluate expressions symbolically (with numbers rendered as rational
fractions having arbitrarily long numerators and denominators or
represented as symbols).
The user can request that floatingpoint evaluations of numbers
be carried out with arbitrarily long precision.
The following are a sampling of interactive mathematical software
systems.

CPLEX
 is a software package used to solve linear programming problems, including integer, mixed integer, and network linear programming problems. CPLEX can be used as an interactive program or as a callable subroutine library. The CPLEX interactive problem solver allows users to enter, modify, and solve problems from computer terminals. This is particularly useful if one is solving a problem once or if one is prototyping a solution method. The callable library provides access to the CPLEX optimization, utility, problem modification, query, and file I/O routine directly from C or Fortran programs.

LabView
 is a graphical programming package for the development of tests, measurements, and control applications.

Maple
 is an interactive symbolic computation system containing symbolic, numerical, graphical, and programming capabilities. Maple performs equation solving, linear algebra, calculus, complex analysis, and more with virtually unlimited precision. Maple was developed as an interactive system for computer algebraic manipulations associated with symbolic computing, but many more expanded capabilities have been added. It is available on a wide range of computers from personal computers and workstations to supercomputers.

Mathematica
 is an interactive software system for numerical, symbolic, and graphical computing as well as visualizations. The user can integrate Mathematica output (computations, graphics, animations, etc.) with ordinary text entirely within the Mathematica system for the preparation of complete electronic documents to be used as technical reports or presentations. A wide variety of Mathematica application libraries are available. MathLink is a communication protocol that allows the exchange of information between Mathematica and other computer packages such as Matlab or Excel. A programming language based on pattern matching can be used for extending the capabilities of the Mathematica system. Mathematica is available on over twenty platforms from PCs to scientific workstations to largescale scientific mainframe computers.

Matlab
 is a computing environment that provides computation, visualization, and applicationspecific tool boxes. Matrix notation is used to produce a MATtrix LABoratory with a builtin set of commands for standard algorithms involving numerical computations. A matrixoriented language may be used for largescale computation and data analysis. Interactive 2D and 3D graphical capabilities are available for analysis, transformation, and visualization of data. Matlab can dynamically link with C or Fortran programs. The package is available on a wide range of computers such as PCs, workstations, and supercomputers. Matlab has more than 20 tool boxes available for specialized applications such as signal and image processing, control system design, frequency domain identification, robust control design, mathematics, statistics, and data analysis, neural networks and fuzzy logic, optimization, and splines. Also, symbolic computing is available with a tool box interface to a version of Maple.

Octave
 is a highlevel language, which is mostly compatible with Matlab, and is primarily intended for numerical computations. It provides a command line interface for solving linear and nonlinear problems numerically and for performing other numerical experiments. Octave has extensive tools for solving common numerical linear algebra problems such as finding roots of nonlinear equations, integrating ordinary functions, manipulating polynomials, and integrating ordinary differential and differentialalgebraic equations. It is extensible and customizable via userdefined functions written in Octave's own language, or using dynamically loaded modules written in different programming languages. GNU Octave is free software, which may be redistributed or modified under the terms of the
GNU General Public License
as published by the
Free Software Foundation

REDUCE
 is an interactive computer algebra system designed for general algebraic computations of interest to mathematicians, scientists, and engineers. While it is used by many as an algebraic calculator for problems that can be done by hand, the main aim of REDUCE is to support calculations that are feasible only by a computer solution.
Some of the symbolic algebraic computer systems other than those listed
here are
Axiom,
Derive,
GANITH,
Magma,
Mathcad,
Milo,
MuPAD,
Pari,
Schur,
and
SymbMath.
Intermediate Libraries and Software Packages
There are many Intermediate Libraries and
software packages available for solving
mathematical problems of various types.
Some examples of just a few of them,
which are primarily publicdomain software,
are the following:

FITPACK
 is a mathematical library that performs curvefitting and surfacefitting using splines under tension. Features include a variety of dimensions for base and target spaces, approximation of data with apparent poles, and the approximation of integrals, derivatives, and lengths of the approximating curves.

NSPCG
 are packages of subroutines for solving large sparse linear systems of equations by iterative methods. Features of the software include automatic parameter estimation and implementaton of accurate termination criteria. These packages can be employed to solve linear systems arising from either finite difference or finite element modeling of elliptic partial differential equations.

LAPACK
 contains routines for solving systems of simultaneous linear equations, leastsquares solutions of linear systems of equations, eigenvalue problems, and singularvalue problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.

MINPACK
 is a library of routines for solving systems of nonlinear equations and nonlinear least squares problems. One can either choose whether to use routines that calculate the Jacobian automaticaly or to supply routines for the Jacobian.

PDE2D
 is a finite element code that solves systems of linear and nonlinear timedependent, steadystate, and eigenvalue partial differential equations in general twodimensional regions and in threedimensional boxes. It can handle problems in such areas as elasticity, diffusion, heat condition, potential energy, and fluid mechanics. It has an interative interface and extensive graphical output capabilities.

ODEPACK
 is a collection of subroutines for the solution of ordinary differential equations, with initial values given. These routines are particularly useful for solving the ODE systems that arise when the method of lines is used to solve timedependent partial differential equations. They were developed at
Lawrence Livermore National Laboratory (LLNL).

PETSc
 is a Portable Extensible Toolkit for Scientific Computing for solving partial differential equations and related problems. It is a suite of data structures and routines for building large scale applications consisting of parallel linear and nonlinear equation solvers and unconstrained minimization modules.
Argonne National Laboratory (ANL)
supports this project.

ScaLAPACK
 is a collaborative effort involving several institutions [
Oak Ridge National Laboratory,
Rice University,
University of California (Berkeley},
University of California (Los Angeles),
University of Illinois (Urbana/Champaign), and
University of Tennessee (Knoxville)] and comprises four components: dense and band matrix software (ScaLAPACK), large sparse eigenvalue software (PARPACK and ARPACK), sparse direct systems software (CAPSS and MFACT), and preconditioners for large sparse iterative solvers (ParPre).