DOC

Windows Compute Cluster Server 2003 - Reviewers Guide

By Alma Kelly,2014-08-10 06:53
11 views 0
Windows Compute Cluster Server 2003 - Reviewers Guide

Reviewers Guide

    Published: May 2006

    For the latest information, please see http://www.microsoft.com/hpc

Abstract

     ??For customers solving complex computational problems, Microsoft Windows Compute Cluster Server 2003 accelerates time-

    to-insight by providing a high-performance computing (HPC) platform that is easy to deploy, operate, and integrate using an organization’s existing infrastructure and tools. In the past, setting up and configuring a cluster was a technically complex task that often required dedicated support staff. Windows Compute Cluster Server 2003 has made the task easy by offering prescriptive setup procedures that simplify network configuration, the ability to load nodes remotely using Remote Installation

    Services (RIS), automated node configuration, and tools and technologies that help organizations configure cluster security settings. The integrated Job Scheduler, which can be accessed through a command-line interface (CLI) or through several application programming interfaces (APIs), can be used to submit and manage cluster workloads. Active Directory? integration provides end-to-end identity management and security features, while the Microsoft Management Console (MMC) supports extensible snap-ins and integration with Microsoft Operations Manager (MOM). Organizations can run leading applications from key independent software vendors (ISVs) on their clusters to help meet their needs in a timely and cost-effective manner while maintaining high performance. They can run parallel jobs using the Message Passing Interface (MPI) support provided in Windows Compute Cluster Server 2003, a full implementation of the MPI chameleon (MPICH) standard. Microsoft Visual Studio? 2005 provides developers an integrated development environment that includes parallel compiling and debugging capabilities. Windows Compute Cluster Server 2003 provides seamless integration with the Microsoft Windows Server? 2003

    operating systems, resulting in security, storage, and productivity gains for organizations.

Contents

    Overview of Windows Compute Cluster Server 2003 .............................................................. 1 Primer on Windows Compute Cluster Server 2003 ................................................................. 2 What is Windows Compute Cluster Server 2003? ............................................................... 2

    About MPI, MPICH, and MS-MPI ..................................................................................... 4 System Requirements......................................................................................................... 5

    Hardware Requirements................................................................................................... 5

    Software Requirements .................................................................................................... 6

    Network Requirements ..................................................................................................... 7 Getting Started ....................................................................................................................... 8 Setting up a Cluster ............................................................................................................ 8 Create a head node ............................................................................................................ 8 Install the Compute Cluster Pack ........................................................................................ 8 Configure the Cluster ........................................................................................................ 11 Technical Overview .............................................................................................................. 13 Architecture ...................................................................................................................... 13 Network Topology ............................................................................................................. 14

    Scenario One: Two NICs on the Head Node; One NIC on the Compute Nodes .............. 14

    Scenario Two: Two NICs on Each Node ......................................................................... 16

    Scenario Three: Three NICs on the Head Node; Two NICs on the Compute Nodes ....... 17

    Scenario Four: Three NICs on Each Node ..................................................................... 18

    Scenario Five: One NIC Per Node ................................................................................. 20 Features ........................................................................................................................... 20

    Compute Cluster Administrator ...................................................................................... 20

    Compute Cluster Job Scheduler ..................................................................................... 24 Task Execution ................................................................................................................. 26

    CLI ................................................................................................................................. 28

    The Compute Cluster Pack Application Programming Interface (CCPAPI) ..................... 29

    MS-MPI Features ........................................................................................................... 30 Cluster Security ................................................................................................................ 31

    Cluster Administrators .................................................................................................... 31

    Cluster Users ................................................................................................................. 31 Security Considerations for Jobs and Tasks ...................................................................... 31 Summary .............................................................................................................................. 33 Related Links ........................................................................................................................ 34

    1

Overview of Windows Compute Cluster Server 2003

    ??Microsoft Windows Compute Cluster Server 2003 provides an integrated application

    platform for developing, deploying, running, and managing high-performance computing (HPC) applications. Using this platform, individuals and organizations can perform multi-node workload computing using commodity hardware in an environment that will shorten their time to insight.

    HPC is increasingly being achieved with clusters of industry standard servers that can range from a few nodes (individual computers) to hundreds of nodes. Wiring, provisioning, configuring, monitoring, and managing these nodes and providing appropriate, secured user access is a complex endeavor that often requires costly support and administrative resources. Because users typically spend more time on cluster administration and management tasks than on running jobs, organizations experience a loss in productivity, as well. The goals of Windows Compute Cluster Server 2003 are to simplify management and reduce the total cost of ownership (TCO) of compute clusters, making them accessible to a broader audience. Based on these goals, Windows Compute Cluster Server 2003 has been designed to be intuitive to administer and manage. Its installation and system configuration processes are fully prescribed and largely automated. In addition, users will probably be familiar with the standard Windows features that it includes for deploying and managing clusters remotely. For example, because Windows Compute Cluster Server 2003 is fully integrated with the Microsoft Windows Server System? solution stack, users can also benefit from the advanced ?management technologies available in the Active Directory directory service and in Microsoft

    Operations Manager (MOM). Users familiar with the Windows Server? platform can become

    productive faster.

    Users whose work demands HPC solutions also require applications that execute complex computations and elaborate data output. Microsoft has worked with independent software vendors (ISVs) to port applications to Windows Compute Cluster Server 2003 that serve several markets, such as manufacturing, life sciences, geological sciences, and financial services. To help deliver on the promise of usability, a full-function Job Scheduler is provided which enables comprehensive job management through a Job Manager user interface (UI) or through a command line interface (CLI).

    Windows Compute Cluster Server 2003 supports the execution of parallel applications based on the Message Passing Interface (MPI) standard. Users can take advantage of the ?enhancements in Microsoft Visual Studio 2005 aimed at parallel computing, including

    support for the OpenMP standard and a parallel debugging capability that supports MPI. When a user submits a job to the cluster, the job is recorded in the head node database along with its properties, entered into the execution queue, and then run when the resources it requires become available. Because the cluster is in the user’s Active Directory domain, jobs

    execute using that user’s permissions. As a result, the complexity of using and synchronizing different credentials is eliminated, and the user does not have to use different methods of sharing data or compensate for permission differences among different operating systems. This means that Windows Compute Cluster Server 2003 offers transparent execution, access to data, and integrated security technologies.

    1 Windows Compute Cluster Server 2003 Reviewers Guide 1

Primer on Windows Compute Cluster Server 2003

    HPC has evolved considerably during the last 15 years. The “supercomputer” solutions of the

    early 1990s provided enormous parallel computing power, but they cost tens of millions of U.S. dollars and required a great deal of expertise to deploy, manage, and maintain. Consequently, customers for these solutions were generally limited to governments and large research institutions.

    The most advanced supercomputing solutions available today are still too expensive for many organizations, costing about one hundred million dollars. Almost any organization can harness the power of HPC by clustering inexpensive and readily available commodity hardware, thanks to the increased computing power based on 64-bit processor architecture and the associated increase in memory address space. The cost to create a small cluster for HPC is on the order of five to 10 thousand U.S. dollarsabout 10,000 times less than it cost 15 years

    ago to produce the same computing power.

    This decrease in the cost of clustered HPC solutions gives organizations more options. For example, they can submit large scale or high precision jobs to supercomputer systems while executing smaller, routine tasks using a local cluster.

    Unfortunately, most solutions are still difficult and costly to use, manage, and integrate into an overall computing infrastructure. In addition, they often do not support standard applications. With the release of Windows Compute Cluster Server 2003, Microsoft aims to remove the barriers that prevent individual engineers and scientists from independently leveraging the computing power available today. In addition, large enterprises with HPC needs can also benefit, because their resources are freed up to deploy local clusters that solve more problems more quickly.