========================================================================

              Open Fabrics Enterprise Distribution (OFED)
               MVAPICH2-1.5.1 in OFED 1.5.2 Release Notes

                             September 2010


Overview
--------

These are the release notes for MVAPICH2-1.5.1. MVAPICH2 is an MPI-2
implementation over InfiniBand, iWARP and RoCEE (RDMAoE) from the Ohio
State University (http://mvapich.cse.ohio-state.edu/).


User Guide
----------

For more information on using MVAPICH2-1.5.1, please visit the user
guide at http://mvapich.cse.ohio-state.edu/support/.


Software Dependencies
---------------------

MVAPICH2 depends on the installation of the OFED Distribution stack with
OpenSM running. The MPI module also requires an established network
interface (either InfiniBand, IPoIB, iWARP, RoCEE uDAPL, or Ethernet).
BLCR support is needed if built with fault tolerance support. Similarly,
HWLOC support is needed if built with Portable Hardware Locality feature
for CPU mapping.


ChangeLog
---------

* Features and Enhancements
    - Significantly reduce memory footprint on some systems by changing
      the stack size setting for multi-rail configurations
    - Optimization to the number of RDMA Fast Path connections
    - Performance improvements in Scatterv and Gatherv collectives for
      CH3 interface (Thanks to Dan Kokran and Max Suarez of NASA for
      identifying the issue)
    - Tuning of Broadcast Collective
    - Support for tuning of eager thresholds based on both adapter and
      platform type
    - Environment variables for message sizes can now be expressed in
      short form K=Kilobytes and M=Megabytes (e.g.
      MV2_IBA_EAGER_THRESHOLD=12K)
    - Ability to selectively use some or all HCAs using colon separated
      lists.  e.g. MV2_IBA_HCA=mlx4_0:mlx4_1
    - Improved Bunch/Scatter mapping for process binding with HWLOC and
      SMT support (Thanks to Dr. Bernd Kallies of ZIB for ideas and
      suggestions)
    - Update to Hydra code from MPICH2-1.3b1
    - Auto-detection of various iWARP adapters
    - Specifying MV2_USE_IWARP=1 is no longer needed when using iWARP
    - Changing automatic eager threshold selection and tuning for iWARP
      adapters based on number of nodes in the system instead of the
      number of processes
    - PSM progress loop optimization for QLogic Adapters (Thanks to Dr.
      Avneesh Pant of QLogic for the patch)

* Bug fixes
    - Fix memory leak in registration cache with --enable-g=all
    - Fix memory leak in operations using datatype modules
    - Fix for rdma_cross_connect issue for RDMA CM. The server is
      prevented from initiating a connection.
    - Don't fail during build if RDMA CM is unavailable
    - Various mpirun_rsh bug fixes for CH3, Nemesis and uDAPL interfaces
    - ROMIO panfs build fix
    - Update panfs for not-so-new ADIO file function pointers
    - Shared libraries can be generated with unknown compilers
    - Explicitly link against DL library to prevent build error due to
      DSO link change in Fedora 13 (introduced with gcc-4.4.3-5.fc13)
    - Fix regression that prevents the proper use of our internal HWLOC
      component
    - Remove spurious debug flags when certain options are selected at
      build time
    - Error code added for situation when received eager SMP message is
      larger than receive buffer
    - Fix for Gather and GatherV back-to-back hang problem with LiMIC2
    - Fix for packetized send in Nemesis
    - Fix related to eager threshold in nemesis ib-netmod
    - Fix initialization parameter for Nemesis based on adapter type
    - Fix for uDAPL one sided operations (Thanks to Jakub Fedoruk from
      Intel for reporting this)
    - Fix an issue with out-of-order message handling for iWARP
    - Fixes for memory leak and Shared context Handling in PSM for
      QLogic Adapters (Thanks to Dr. Avneesh Pant of QLogic for the
      patch)


Main Verification Flows
-----------------------

In order to verify the correctness of MVAPICH2-1.4.1, the following
tests and parameters were run.

Test                            Description
====================================================================
Intel                           Intel's MPI functionality test suite
OSU Benchmarks                  OSU's performance tests
IMB                             Intel's MPI Benchmark test
mpich2                          Test suite distributed with MPICH2
NAS                             NAS Parallel Benchmarks (NPB3.2)


Mailing List
------------

There is a public mailing list mvapich-discuss@cse.ohio-state.edu for
mvapich users and developers to
- Ask for help and support from each other and get prompt response
- Contribute patches and enhancements

========================================================================
