HPs OpenSSI Linux Cluster Projecт

HP’s OpenSSI Linux Cluster Project www

openssi.org Bruce J. Walker HP Fellow Office of Strategy and Technology

6/11/03hp ssi project

hp-ssi.ppt

page 1

Overview of HP OpenSSI Cluster

Single HA root filesystem Consistent OS kernel on each node Join cluster early in boot Strong Membership Single view of filesystems, devices, processes, ipc objects Single management domain Load balancing of connections and processes Dynamic service provisioning

hp-ssi.ppt

page 2

6/11/03

Goals

Become the Industry Standard in Linux Clustering Scale up and Scale down Modular technology that applies to all forms of clustering - single technology base for HPC and Commercial

hp-ssi.ppt

page 3

6/11/03

Clustering Goals

To various degrees, all of: High Availability Scalability Manageability Usability

hp-ssi.ppt

page 4

6/11/03

HP OpenSSI Linux Clusters

Ideal/Perfect Cluster in all dimensions

SMP

SMP

Typical HA Cluster

HP SSI Linux Clusters

HUGE

Really BIG

log scale

hp-ssi.ppt

page 5

6/11/03

HP OpenSSI Linux Clusters Approach

2 or more “service” nodes + optional “compute” nodes Computing can be done on service nodes Service nodes provide availability and 2 forms of load balancing Compute nodes for larger scale; Vproc for clusterwide process space, process launch and process movement Lustre for scale up filesystem story (including root) CFS or GFS for scale down Diskless node option No daemons need on compute nodes for job launch or stdio Integration with other open source components: Lustre, LVS, PBS, Maui, Ganglia, SuperMon, SLURM, …

hp-ssi.ppt

page 6

6/11/03

Availability

No Single (or even multiple) Point(s) of Failure Automatic Failover/restart of services in the event of hardware or software failure Filesystem failover integrated and automatic Application Availability is simpler in an SSI Cluster environment; statefull restart easily done; Application transparent checkpoint/restart Online software upgrade (ongoing) Architected to avoid scheduled downtime Node eviction via transparent process migration

hp-ssi.ppt

page 7

6/11/03

HP OpenSSI Linux Clusters - Manageability

Single Installation; single root filesystem Joining the cluster is automatic as part of booting and doesn’t have to managed Trivial online addition of new nodes Use standard single node tools (SSI Admin) Visibility of all resources of all nodes from any node Applications, utilities, programmers, users and administrators often needn’t be aware of the SSI Cluster Simpler HA (high availability) management

hp-ssi.ppt

page 8

6/11/03

HP OpenSSI Linux Cluster Ease of Use

Can run anything anywhere with no setup; Can see everything from any node; service failover/restart is trivial; automatic or manual load balancing; powerful environment for application service provisioning, monitoring and re-arranging as needed

hp-ssi.ppt

page 9

6/11/03

Scalability

Two aspects – overhead and load balancing

hp-ssi.ppt

page 10

6/11/03

Scalability - Overhead

No daemons needed per node (single HA clusterwide init) Node monitoring adjustable and could co-ordinate noise Minimal cross-node co-ordination Efficient kernel-to-kernel rexec() for process launch Parallel multicast boot

hp-ssi.ppt

page 11

6/11/03

Scalability – Load Balancing

Load balancing of connections and processes SSI makes distributing function very easy SSI allows any process on any processor All resources transparently visible from all nodes: filesystems, IPC, processes, devices*, networking* OS version in local memory on each node Migrated processes use local resources and not home-node resources

hp-ssi.ppt

page 12

6/11/03

Component Contributions to HP OpenSSI Cluster Project

Lustre

HP OpenSSI Cluster Project

HP contributed

Open source and integrated

To be integrated

Appl. Avail.

CLMS

GFS

Mosix

Beowulf

Vproc

DLM

LVS

OCFS

IPC

DRBD

CFS

EVMS

Load Leveling

hp-ssi.ppt

page 13

6/11/03

SSI Cluster Architecture/ Components

Kernel Interface

8. DLM

12. DRBD

14. Init; booting; run levels

18. Timesync

13. Packaging and Install

16. Appl Availability; HA daemons

17. Application Service Provisioning

19. MPI, etc.

3. Filesystem

6. IPC

1. Membership

5. Process Loadleveling

CFS

GFS

Lustre

4. Process Mgmt VPROC

7. Networking/ LVS

9. Devices/ shared storage devfs

10. Kernel data replication service

2. Internode Communication/ HA interconnect

15. Sysadmin;

11. CLVM/ EVMS (TBD)

Physical filesystems

hp-ssi.ppt

page 14

6/11/03

Vproc Features

Clusterwide unique pids (decentralized) Peer-to-peer – no master node process and process group tracking under arbitrary failure and recovery no polling reliable signal delivery under arbitrary failure process always executes system calls locally no “do-do” at “home node”; never more than 1 task struct per process for HA and performance, processes can completely move therefore can service a node without application interruption process always only has 1 process id transparent process migration clusterwide /proc, clusterwide job control single init Unmodified “ps” shows all processes on all nodes transparent clusterwide debugging (ptrace or /proc) integrated with load leveling (manual and automatic) exec time and migration based automatic load leveling fastnode command and option on rexec, rfork, migrate architecture to allow competing remote process implementations

hp-ssi.ppt

page 15

6/11/03

HP SSI Linux Clusters - Status

Version 1.0 just released – Binary, Source and CVS options Functionally complete RH9 and RHel3 Debian release also available IA-32 and Itanium Platforms Runs HPTC apps as well as Oracle RAC Available at OpenSSI.org

hp-ssi.ppt

page 16

6/11/03

Ongoing Enhancements

2.6-based version Improved scalability (two node classes) Integrate power/console management SMP Clusters (multiple nodes within a large SMP) Vproc without single root (flexibility) Node eviction

hp-ssi.ppt

page 17

6/11/03

HP OpenSSI Linux Clusters - Conclusions

HP has recognized that Linux clusters are important part of the future. HP has recognized that combining scalability with availability and manageability/ease-of-use is key to clustering HP is leveraging its merger with Compaq (Tandem/Digital) to bring the very best of clustering to a Linux base

hp-ssi.ppt

page 18

6/11/03