Production infrastructure

From i2basque

(Difference between revisions)
Jump to: navigation, search
(FIRST CLUSTER)
Line 6: Line 6:
------
------
-
==== FIRST CLUSTER====
 
-
-----------------------
 
-
An IBM iDataPlex Cluster HPC which consists on the following hardware and software:
 
-
 
-
'''Hardware''' :
 
-
 
-
* A manager frontend x3650 with two Intel Quadcore Xeon Harpertown 2,83GHz processors, 16GB RAM and 146 GB SAS hard disk.
 
-
* A computing cluster with 128 computing elements composed by 2 Intel QuadCore Xeon Harpertown processors, (8 processors identified), 500 GB SATA internal disk and 64 GB RAM each.
 
-
* Two IBM Systems Storage  DS4700  FC with 30 TB of effective storage shared among all elements by means of GPFS.
 
-
* 5 more x3650 management elements  with two Intel Quadcore Xeon Harpertown 2,83GHz processors, 16GB RAM and 146 GB SAS hard disk, used for backup and cluster management
 
-
* An IBM Systems Storage  DS4700  SATA with 60 TB used for backup.
 
-
* All machines are interconnected by an Infiniband network.
 
-
* All machines run 64 bits software, operating systems included.
 
-
Resuming totals: 1024 processors, with 8,2 TB RAM, 30 TB shared storage on disk and 60TB storage for backup.
 
-
The benchmark used to evaluate the speed of the cluster has been Linpack, and for a cluster configuration with 1064 processes using the 128 computing nodes, this benchmark has reported a speed of 11 TeraFlops.
 
-
 
-
'''Software''':
 
-
 
-
* Operating system of the compute elements: Red Hat Enterprise Linux version 5 Update 2 with kernel 2.6.18 in the frontend and compute elements.
 
-
* Development software: GCC v.4.1.2 , Intel Cluster Toolkit Compiler 11.0, Python 2.4.3, Perl, Java, Php
 
-
* HPC software: OpenMPI 1.2.6 and MVAPICH 1.0.1 (mpi over infiniband) for intel, gcc and pgi compilers
 
-
* HTC Job management software: Slurm 2.3.4
 
-
* Other software: xCAT2, GPFS, Tivoli Storage Manager, R-2.9.2, NCL NCARG 5.1.0, GrADS 2.0.a7,
 
-
* Other libraries: HDF5 1.8.3, Jasper 1.701.0, CDO 1.3.1, Genoverlay, NetCDF 3.6.2, NCView 1.93g, Udunits 1.12.4
 
-
 
-
* Any software needed in the investigation can be requested, and such a request will be considered.
 
-
 
-
User accounts are based in NIS, and all accounts and their directories are shared among the machines.
 
-
-------------------
 
==== SECOND CLUSTER====
==== SECOND CLUSTER====

Revision as of 14:18, 19 September 2013

We have available three clusters for HPC, their production infraestructure are explained below.

To access any of our computing resources must complete and submit the following form to grid@i2basque.es.




SECOND CLUSTER


The hardware of the cluster 'gt3' consists on:

  • A manager frontend with an Intel QuadCore Xeon processor, 4GB RAM and 146 GB SAS hard disk.
  • A computing cluster with 80 computing elements composed by 2 Intel QuadCore Xeon processors, (8 and 16 processors identified), 73GB SAS disk and 16 GB RAM each.
  • A PowerVault NAS with 24 TB of effective storage shared among all elements by means of NFS.
  • All machines are interconnected by a Gigabit network.
  • All machines run 64 bits software, operating systems included.

Resuming totals: 1072 processors, 1.27 TB RAM, 24 TB storage on disk and 6.21 Tflops

The benchmark used to evaluate the speed of the cluster has been Linpack.


In this cluster the following software is installed:

  • Operating system of the compute elements: CentOS 5.3 with kernel 2.6.18 in the frontend and compute elements.
  • Development software: GCC 4.1.2 , Intel C C++ Compiler 11.1, Intel Fortran Compiler 11.1, Intel MKL 11.1, Python 2.5.2 and 2.4.3, Php, Perl, Java
  • HPC software: LAM, OpenMPI and MPICH-MPICH2 MPI libraries
  • HTC Job management software: Sun Grid Engine (SGE)
  • Other software: MCNPX 27, OpenFoam 1.6, Meep 1.1.1, MPB 1.4.2, HDF5 1.8.5, Systemimager, NIS, Ganglia, C3 Tools, IPTables
  • Other libraries: libctl 3.1, harminv 1.3.1, FFTW 2.1.5, h5utils 1.12.1
  • Any software needed in the investigation can be requested, and such a request will be considered.

The cluster offers a single system image and it is accessed only through the host gt3.i2basque.es

User accounts are based in NIS, and all accounts and their directories are shared among the machines.

Jobs are submitted using the SGE job management software, jobs can be batch, parallel or multiple.

The whole system has been installed using Systemimager, so the task of adding/deleting a compute element from the system is quite easy. Moreover, a backup of an freshly installed image of the system is stored.

The C3 Tools are used to ease managing tasks such as the execution of a command on a certain number of hosts, or copy a file to several hosts.

All general utilities and shared software is installed in a shared directory stored in the NAS.

IPTables is used as a firewall to secure this cluster.

Second cluster real time Monitoring


Personal tools