DAPL-ND - DAPL NetworkDirect Stand Alone installer v1.1	[3-13-2014]

  DAPL-ND installers for Windows DAPL over NetworkDirect can be downloaded
  from http://www.openfabrics.org/downloads/Windows/DAPL-ND/

    Windows Server 2008 (R2) use DAPL-ND_win7_x64.msi
    Windows Server 2012 (R2) use DAPL-ND_win8_x64.msi	

The installation of DAPL enables Intel MPI to utilize RDMA devices via the NetworkDirect v2 API; see 
%PROGRAMFILES%\DAPL\imb.bat discussion below.
Intel MPI for Window 4.1.3.045 has been valided for DAPL-ND.

uDAT / uDAPL files built using the Mellanox 4.40 release SDK.

Install from account 'administrator' as it creates

  %windir%\system32\
                Dat2.dll
                Dapl-ND.dll
                Dapl2test.exe
                NDList.exe 

  %PROGRAMFILES%\DAPL\

  %SystemDrive%\DAT\dat.conf

Use Control Panel --> Programs and Features to remove DAPL

Within %PROGRAMFILES%\DAPL\ you will find
        Dt-svr.bat      Dapltest server (server can  support multiple clients)
        Dt-cli.bat      Dapltest client
        Imb.bat         Intel MPI Benchmarks driver script.

After DAPL installation, verify ND devices are in sync w.r.t. ordering.
For all hosts to be used in IMPI runs, execute NDlist.exe and verify ND0 or the NDx instance
referenced in 'C:\DAT\dat.conf' is the IPoIB IPv4 address you are expecting.
If not, disable the IPoIB instance which is not ACTIVE (aka associated port has no cable).
If the above fails, then disable IPv6 on the IPoIB interfaces as IPv6 addressing can confuse
DAPL2-ND.dll IPv4 address lookup; a known issue which is being addressed.

Assuming Intel MPI and DAPL are installed.

Running Dapltests – see manual.htm for further details.

        cd  “%ProgramFiles%”\DAPL\

        dt-svr.bat ND0    # ND0 == 1st NDv2 configured device, see NDlist, normally the 1st ACTIVE IPoIB instance.

        At the dapltest client host (can be the same although not very interesting performance wise).

                dt-cli.bat  ND0  IPoIB-IPv4-addr-where-dt-svr-is-running dt-command   # use cmd == ‘X’ valid cmds are displayed.
                     Example: dt-cli ND0 10.10.4.101 X

                Use ‘dt-cli ND0 10.10.4.101 stop’ to shutdown the dapltest server.

                A good place to start is ‘dt-cli ND0 a.b.c.d conn’, then ‘dt-cli ND0 a.b.c.d transmete’

Running Intel MPI benchmark tests: (See start--> Intel SW-->MPI library reference for details

                cd  “%ProgramFiles%”\DAPL
                imb.bat<enter> to display cmd-line format.

                Add MPI hosts to the file hostfile.txt , format is a single IPoIB hostname or IPv4 address per line.

                imb.bat num-nodes imb-test ranks-per-node  MPI-device(rank-access-method:dat.conf-dev-name)

                imb 4 PingPong 2 rdma:ND0        # 4 nodes, PingPong test, 2 ranks-per-node, rdma access:ND0,
                                                 # rdma uses IB for all rank communications.

                imb 4 all 2 rdssm:ND0            # 4 nodes ‘all’ IMB tests (takes about 15 minutes for 4 nodes), 
                                                 # rdssm:ND0, rdssm uses sysV shared memory intranode and IB RDMA internode.