Dalco Supermicro System - Eiger
This page describes Eiger in the following sections
Short Description of the System
The EIGER cluster at CSCS (a DALCO SUPERMICRO system) is a tightly coupled computing cluster system with three types of dual-socket six-cores and twelve-cores (Magnycours) AMD Opteron 2427/6174 processor nodes for a total of 19 cluster nodes available to users. There are 15 visualization nodes with NVIDIA GeForce 285 GTX cards with NVIDIA M2050 and C2070 FERMI cards, 7 nodes with 2GB per core or 24 GB per node and 8 nodes with 4 GB per core or 48 GB per node. There are 4 additional nodes for GPGPU development with access to NVIDIA TESLA S1070 devices and to NVIDIA C2070 FERMI devices. A dedicated Infiniband QDR fabric infrastructure, supports both parallel-MPI traffic and the internal parallel scratch file system I/O data traffic. In addition, a commodity 10 GbE LAN ensures interactive login access, home, project and application file sharing among the cluster nodes.
How to Access Eiger
Only users with an approved HP2C projects can have access to eiger and users who in their production projects asked for 'data analysis systems'. Eiger is indeed the visualization cluster which replaced Horus.
Eiger is accessible from the front end machine ela (ela.cscs.ch) as eiger.cscs.ch.
ssh user_name@eiger.cscs.ch
[same password as ela]
Programming Environment and Supported Software
The software environment on Eiger is controlled using the modules framework which gives an easy and flexible mechanism to access to all of the CSCS provided compilers, tools and applications.
Compilers for Eiger are described in details here. GNU, PGI, Pathscale and Intel compilers are available as well the GPU development environments including CUDA and PGI accelerator compilers. Nodes are running the Novell SUSE Linux Enterprise Server 11 Operating System.
Submission of Batch Jobs and Interactive Jobs
The SLURM batch queuing system is used for the submission of jobs on Eiger. The batch system can also be used to gain access to an interactive batch job, where you are provided with a set of compute nodes at your disposal and an interactive shell prompt from which to directly use these nodes using the mpirun command.
Interactive batch jobs should only be requested for small period of time and should be requested on as small a numbers of processors as is required. Since an interactive batch job is allocated through the standard batch scheduling algorithms, you should only request this type of access if there are already sufficient free resources on the machine so that the interactive session can begin immediately.
Details of batch submission and how to set up an interactive batch job are available here. For a list of the most useful SLURM commands, please have a look at the corresponding FAQ section under the User Forum.
Data Storage
/scratch
Eiger has a scratch space (/scratch/eiger/user_name) of about 66 TB connected via high speed QDR IB interconnect. Note that this storage is not backed up and is cleared on regular intervals so please ensure that you do not target this as a long term storage.
/project
Access to the shared storage (/project) is also available through the high speed interconnect.
For further information regarding transfer of large amount of data, please contact help(at)cscs.ch.
For further information, please have a look at Data Management or contact help(at)cscs.ch.
Detailed Machine Description
The EIGER cluster at CSCS (a DALCO SUPERMICRO system) is a tightly coupled computing cluster system, running Novell SUSE Linux Enterprise Server 11 Operating System release and includes 23 nodes based on the dual-socket six-cores/twelve-cores AMD Opteron 2427/6174 processor architecture running at 2.2 GHz, offering 24 GB of main system memory per node, for a total of 318 cpu cores and 728 GB aggregate memory. 8 out of 19 cluster nodes offers a larger main system memory capacity up to 48 GB.
SLURM V 2.3.0 is the main batch queuing system installed and supported on the cluster in order to let end-users access in a shared or reserved mode any available visualization/computing resource.
This cluster has several classes of nodes, covering special functionality:
- Class 0: Administration Node (1x)
- Class 1: Login Node (1x)
- Class 2: Visualization Nodes (7x)
- Class 3: Fat Visualization Nodes (8x)
- Class 4: Advanced Development Nodes (4x)
- Class 5: Storage Nodes (2x)
- Class 6: Test Nodes (2x)
EIGER Cluster features summary table
======================================
Node name | Kind | GPU | GPU-type | DVI-OUT | GPU-c | GPU-# | GPU-m | GPU-f | GPU-mt |
|---|---|---|---|---|---|---|---|---|---|
eiger160 | login | Matrox | - | yes | - | - | - | - | - |
eiger170 | admin | Matrox | - | yes | - | - | - | - | - |
gpfs01 | gpfs | Matrox | - | yes | - | - | - | - | - |
gpfs02 | gpfs | Matrox | - | yes | - | - | - | - | - |
=== |
|
|
|
|
|
|
|
|
|
eiger180 | test | GTX 480 | fermi | yes | 480 | 2 | 1.5 GB | 1.4 Ghz | GDDR5 |
eiger181 | test | GTX 480 | fermi | yes | 480 | 2 | 1.5 GB | 1.4 Ghz | GDDR5 |
eiger200 | vis | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger201 | vis | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger202 | vis | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger203 | vis | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger204 | vis | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger205 | vis | GTX 480 | fermi | yes | 480 | 2 | 1.5 GB | 1.4 Ghz | GDDR5 |
eiger206 | vis | GTX 480 | fermi | yes | 480 | 2 | 1.5 GB | 1.4 Ghz | GDDR5 |
=== |
|
|
|
|
|
|
|
|
|
eiger207** | visfat | 2xM2050 | fermi | none | 448 | 2 | 2.6 GB | 1.15Ghz | GDDR5 |
eiger208** | visfat | 2xM2050 | fermi | none | 448 | 2 | 2.6 GB | 1.15Ghz | GDDR5 |
eiger209** | visfat | 2xC2070 | fermi | yes | 448 | 2 | 5.4 GB | 1.15Ghz | GDDR5 |
eiger210** | visfat | 2xC2070 | fermi | yes | 448 | 2 | 5.4 GB | 1.15Ghz | GDDR5 |
=== |
|
|
|
|
|
|
|
|
|
eiger220 | visfat | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger221 | visfat | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger222 | visfat | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
eiger223 | visfat | GTX 285 | geforce | yes | 240 | 1 | 2 GB | 1.48Ghz | GDDR3 |
=== |
|
|
|
|
|
|
|
|
|
eiger240* | adn | S1070 | tesla | none | 240 | 2 | 4 GB | 1.30Ghz | GDDR3 |
eiger241* | adn | S1070 | tesla | none | 240 | 2 | 4 GB | 1.30Ghz | GDDR3 |
eiger242** | adn | 2xC2070 | fermi | yes | 448 | 2 | 5.4 GB | 1.15Ghz | GDDR5 |
eiger243** | adn | 2xC2070 | fermi | yes | 448 | 2 | 5.4 GB | 1.15Ghz | GDDR5 |
(*) The S1070 shares 2 x C1060 to one node (multi-GPU node)
(**) NVIDIA "FERMI" multi-GPU nodes, with GPU MEM ECC ENABLED!
As an high speed network interconnect, the cluster EIGER relies on a dedicated Infiniband QDR fabric infrastructure, supporting both parallel-MPI traffic and the internal parallel scratch file system I/O data traffic. In addition, a commodity 10 GbE LAN ensures interactive login access, home, project and application file sharing among the cluster nodes, and a standard 1 Gbe administration network is also
reserved for cluster management purposes.



