Brief notes for configuring, installing and using PetSc and ScaLAPACK with Fortran 90/95 (assuming you're running linux (preferably Debian) and have Intel compilers (7.1) and the MKL (7.21) already installed in /opt/intel).


MPICH
# tar -xzf mpich-1.2.6.tar.gz
# cd mpich-1.2.6
# setenv CC icc
# setenv CXX icpc
# setenv FC ifc
# setenv F90 ifc
# ./configure --with-arch=LINUX --prefix=/opt/mpich-1.2.6  --with-device=ch_p4 --enable-debug --without-mpe --enable-sharedlib
# make
# sudo mkdir /opt/mpich-1.2.6
# sudo make install
# ssh-keygen -t dsa
# cd $HOME/.ssh
# cp id_dsa.pub authorized_keys
{To build mpich2 configure with --with-device=ch3:sock --enable-fast --enable-debuginfo --without-mpe --prefix=/opt/mpich2}

PetSc*
# tar -xzf petsc-2.3.3.tar.gz
# cd petsc-2.3.3
# setenv PETSC_DIR `pwd`
# ./config/configure.py --with-mpi-dir=/opt/mpich-1.2.6 --with-blas-lapack-dir=/opt/intel/mkl721 --prefix=/opt/petsc-2.3.3 --with-shared=yes
# make all
# make test
# sudo mkdir /opt/petsc-2.3.3
# su; make install; exit
{Now add the PETSC_DIR (/opt/petsc-2.3.3) and PETSC_ARCH (linux-gnu-intel) variables in your .bashrc/.cshrc}

BLACS (needed by ScaLAPACK)
# tar -xzf mpiblacs.tgz
# cd BLACS
# cp BMAKES/Bmake.MPI-LINUX Bmake.inc
Edit and configure Bmake.inc
# make MPI
# cd LIB
Now create symlinks as follows:
libblacsCinit_MPI-LINUX-0.a -> blacsCinit_MPI-LINUX-0.a
libblacsF77init_MPI-LINUX-0.a -> blacsF77init_MPI-LINUX-0.a
libblacs_MPI-LINUX-0.a -> blacs_MPI-LINUX-0.a
# cd ../../; sudo chown -R root\: BLACS
# sudo mv BLACS /opt

ScaLAPACK
# tar -xzf scalapack-1.8.0.tgz
# mv scalapack-1.8.0 SCALAPACK
# cd SCALAPACK
Edit and configure SLmake.inc
# make lib
# cd ..; sudo chown -R root\: SCALAPACK
# sudo mv SCALAPACK /opt

Note: Now its a good idea to set/update your environment variables. However the best approach (i.e., the one I use) is to use the environment-modules package (installation instructions are available here). It allows you to keep/use multiple versions of a program without having to worry about environment variables. For example you can keep two mpich installations, one configured with Intel compilers and the other with GNU compilers and switch between the two using a simple command.


Some examples:

MKL (BLAS/LAPACK)
# ifc svd.f90 -I/opt/intel/mkl721/include -L/opt/intel/mkl721/lib/32 -lmkl_lapack95 -lmkl_lapack -lmkl_ia32 -lguide -lpthread
# ./a.out
{Multithreaded BLAS 3 will parallelize parts of the above code on a shared memory machine if you set the OMP_NUM_THREADS variable to 2 or more; Also please note that Fortran 95 wrapper files (modules and libraries) are installed in /opt/intel/mkl721/include and /opt/intel/mkl/lib/32 respectively}

# ifc matmul.f90 -L/opt/intel/mkl721/lib/32 -lmkl_ia32 -lguide -lpthread
# ./a.out
{Compares the performance of matrix multiplication}

# ifc pardiso.f90 -L/opt/intel/mkl721/lib/32 -lmkl_solver -lmkl_lapack -lmkl_ia32 -lguide -lpthread
# ./a.out
{Tests the functionality of the PARDISO solver}

PetSc
# make -f Makefile_petsc petsc_ex (petsc_ex.f90; Makefile)
# mpirun -np 2 ./petsc_ex

ScaLAPACK
# mpif90 parallel_svd.f90 -L/opt/SCALAPACK -lscalapack -L/opt/BLACS/LIB -lblacs_MPI-LINUX-0 -lblacsF77init_MPI-LINUX-0 -lblacs_MPI-LINUX-0 -L/opt/intel/mkl721/lib/32 -lmkl_lapack -lmkl_ia32 -lguide -lpthread
{Note the recursive referencing of libraries}
# mpirun -np 4 ./a.out


Other useful libraries:

SLEPc
Make sure that PETSC_DIR (/opt/petsc-2.3.3) and PETSC_ARCH (linux-gnu-intel) variables are defined
# tar -xzf slepc-2.3.3
# cd slepc-2.3.3
# setenv SLEPC_DIR `pwd`
# ./config/configure.py --prefix=/opt/slepc-2.3.3
# make all
# make test
# sudo mkdir /opt/slepc-2.3.3
# su; make install; exit
{Now add the SLEPC_DIR (/opt/slepc-2.3.3) variable in your .bashrc/.cshrc}

Example:
# make slepc_ex (slepc_ex.f90; Makefile)
# mpirun -np 2 ./slepc_ex

MUMPS* (a parallel sparse direct solver based on LU factorization; depends on ScaLAPACK and (optionally) Metis)
# tar -xzf MUMPS_4.7.3.tar.gz
# cd MUMPS_4.7.3
# cp Make.inc/Makefile.INTEL.PAR Makefile.inc
Edit and configure Makefile.inc
# make double
{use 'make all' for simple, cmplx and cmplx16 versions of the lib}
# cd ..; sudo chown -R root\: MUMPS_4.7.3
# sudo mv MUMPS_4.7.3 /opt

Example:
# mpif90 -fpp mumps_ex.f90 -I/opt/MUMPS_4.7.3/include -L/opt/MUMPS_4.7.3/lib -ldmumps -lpord $SCALAPACK -L/opt/parmetis-3.1 -lmetis
# mpirun -np 2 ./a.out < mumps_inp

---
[*] Other sparse iterative/direct solvers which you might find useful are hypre, Aztec, SuperLU etc.