Dual Boot: ? Windows Compute Cluster Server 2003 ? and SUSELinux Enterprise Server 10
Setup and Configuration Guide Published: July 2007, Updated: December 2007 For the latest information, see http://www.microsoft.com/hpc
Contents
Introduction ........................................................................................................................... 4 Prerequisites ......................................................................................................................... 6 Mixed Compute Cluster Configuration ............................................................................... 6 Compute Nodes ............................................................................................................. 6 Windows Compute Cluster Head Node ........................................................................... 6 Infrastructure Server ....................................................................................................... 7 Software Requirements and Versions Used....................................................................... 7 Windows Compute Cluster Server .................................................................................. 7 SUSE Linux Enterprise Server........................................................................................ 7 Hardware Used ................................................................................................................. 7 2 embedded NC370i Multifunction Gigabit Server Adapters ............................................... 8 Prepare for Windows Compute Cluster Server 2003 and SUSE Linux Enterprise Server 10
SP1 Dual Boot .................................................................................................................. 8 Prepare for Deployment of Operating Systems ...................................................................... 9 Prepare for SUSE Linux Enterprise Server Deployment ..................................................... 9 Configure the Deployment Server ................................................................................... 9 Create the AutoYaST Control File................................................................................. 20 Integrate the Linux HPC Software into the Repository ................................................... 23 Integrate the Dual-Boot Script in to the Repository ........................................................ 24 End of script ........................................................................................................................ 24 Create the Post-Installation Script................................................................................. 25 Prepare for Windows Compute Cluster Server 2003 Deployment .................................... 27 Prerequisites ................................................................................................................ 27 Install and Configure RIS .............................................................................................. 28 Add an Image to RIS .................................................................................................... 28 Inject Drivers ................................................................................................................ 37 Include Scripts into RIS ................................................................................................ 37 Modify the Answer File ................................................................................................. 38 Deploy the Operating Systems to the Compute Nodes......................................................... 39 Deploy SUSE Linux Enterprise Server ............................................................................. 39 Deploy Windows Compute Cluster Server 2003 .............................................................. 40 Prerequisites ................................................................................................................ 40 Deploy Windows .......................................................................................................... 41 Approve Compute Nodes ............................................................................................. 47 Switch Operating Systems on the Compute Cluster ............................................................. 49 Boot from Windows Compute Cluster Server 2003 to SUSE Linux Enterprise Server ....... 51 Boot from Linux to Windows Compute Cluster Server ...................................................... 51 Verify the Cluster Setup ....................................................................................................... 53 Windows Compute Cluster Server 2003 .......................................................................... 53 Run a Simple Command .............................................................................................. 53 Run an MPI Command on All Nodes ............................................................................ 55 Log Entries ................................................................................................................... 59 SUSE Linux Enterprise Server ........................................................................................ 59
Summary ............................................................................................................................ 60
Appendices ......................................................................................................................... 61
Appendix 1 - Background Information .............................................................................. 61
This section provides background material on dual-boot scenarios and Windows Compute Cluster Server 2003. ..................................................................................... 61
Dual-Boot Scenarios .................................................................................................... 61
Introduction to Windows Compute Cluster Server 2003 ................................................ 61
Appendix 2 - Deployment Methods for WCCS and Linux ................................................. 63
Windows Compute Cluster Server 2003 Deployment Methods...................................... 63
SUSE Linux Enterprise Server Deployment .................................................................. 65
Appendix 3 - Example of WCCS Deployment with ADS ................................................... 66
Requirements ............................................................................................................... 66
Appendix 4 - Table of Installation Options ........................................................................ 70
Note: If you already have SUSE Linux Enterprise Server installed on your HPC cluster, you can install Windows as a second OS with either RIS or ADS; modifications will be required to provide partition space for the Windows OS. If you already have Windows Computer Cluster Server installed on your HPC cluster, you will need to create space for the Linux partitions. You can then use standard SUSE Linux Enterprise Server installation methods.
....................................................................................................................................... 71
Appendix 5 - Related Links .............................................................................................. 72
Links for Windows ........................................................................................................ 72
Microsoft Knowledge Base Articles: .............................................................................. 73
Links for Linux .............................................................................................................. 74
AutoYaST:.................................................................................................................... 74
Appendix 6 - Checklist for WCCS Deployment with RIS ................................................... 75
Appendix 7 - Summary of Scripts .................................................................................... 76
Appendix 8 - Sample hpcautoyast.xml Script ............................................................. 77
Appendix 9 - Sample RIS Answer File ............................................................................. 84
Introduction
This document describes the installation and configuration of a high-performance computing ? ?(HPC) cluster for a dual boot of MicrosoftWindows Compute Cluster Server 2003 (WCCS)
and SUSE? Linux Enterprise Server 10 SP1. You can use this document as a guide for setting up your own dual boot cluster, although you may need to modify the procedures described here according to your own particular hardware and software configurations.
Because HPC clusters represent a significant investment of resources, maximizing the benefits of your existing investment can result in huge savings. By using the same cluster for two operating systems, you gain the flexibility of an additional cluster without having to invest in further hardware. There are a number of scenarios in which the ability to switch easily between SUSE Linux Enterprise Server and Windows Compute Cluster 2003 on your HPC cluster is an advantage, including proof-of-concept scenarios, application migration scenarios, and test scenarios.
For a dual-boot installation using Windows Compute Cluster Server 2003 and SUSE Linux Enterprise Server 10 SP1, both must be installed on each of the compute nodes of your cluster. There are several options available for OS installation. In the setup discussed here, Remote Installation Services (RIS) were used for the WCCS installation. RIS, now called the Windows Deployment Services (WDS) legacy mode, is a standard component of WCCS. It was installed on the head node of the compute cluster. A dedicated network installation server installed with SUSE Linux Enterprise Server was used to deploy SUSE Linux Enterprise Server to compute nodes using AutoYaST, the Novell? auto-installation mechanism.
To operate in a dual-boot environment using Windows Compute Cluster Server 2003 and SUSE Linux Enterprise Server 10 SP1, both must be installed on different partitions on the hard disks of all of the compute nodes. For Linux, two hard disk partitions are required. For Windows Compute Cluster Server 2003, one hard disk partition is required.
SUSE Linux Enterprise Server was the first OS deployed to the compute nodes in this setup. It was installed on Partition 1 and Partition 2, leaving the remaining free space, Partition 3, for the Windows Compute Cluster Server 2003 OS. WCCS was then deployed by using RIS. This configuration was used because it is not possible to create custom partitions with RIS.
Typically, a dual-boot configuration provides a boot menu that lets users choose which OS they want to boot. A boot menu will not work in a compute cluster dual-boot environment, however, because there are no users at the compute nodes. To switch between operating systems, the disk partition of the desired OS is flagged as active and the nodes are rebooted. This process can be automated by executing a task that runs on all of the compute nodes. If the cluster is running Windows Compute Cluster Server 2003, either the compute cluster administrative GUI or a script can be used to flag the SUSE Linux Enterprise Server partitions as active. If the cluster is running SUSE Linux Enterprise Server, various methods can be used to run a script on all of the compute nodes to flag the Windows Compute Cluster Server 2003 partition as active. In this setup, the Secure Shell protocol (SSH) was used to run the
script.
4Dua Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 4
There are other configurations and deployment methods available for a dual-boot HPC cluster, but these were not verified for this document. Manual installation is not described here, but it can be easily derived from the provided scripts and installation guidelines. The installation methods and configurations that you choose will depend on the number of compute nodes that you use, and on the order in which you choose to install the operating systems. Appendix 2 provides information about installation options including Windows Deployment Services (WDS); Appendix 3 gives an example of Windows Compute Cluster Server 2003 deployment with the Automated Deployment Services (ADS).
In this document, it is assumed that all compute nodes and the required infrastructure servers of the compute cluster do not operate with different OS versions concurrently. This is technically feasible, but it requires a sophisticated infrastructure setup that is beyond the scope of this document. This document does not describe how to dual boot the cluster‘s head
node or the infrastructure servers. For these servers, it is possible to use a boot menu to choose the desired OS at startup. Infrastructure servers may require additional partitions in order to run the two operating systems.
The focus of this document is the deployment of the Windows Compute Cluster Server 2003 OS and SUSE Linux Enterprise Server 10 SP1 to the compute nodes of a cluster and the operation of the cluster in a dual-boot environment. Configuration of Linux-based and Windows-based HPC clusters is not described. For information on the installation of an HPC cluster, see the related links in Appendix 5.
This document is intended for Linux administrators and Windows IT professionals who are familiar with HPC and who have some familiarity with Windows Compute Cluster Server 2003. For more information about the products and methods described, see the links in Appendix 5.
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 5
Prerequisites
Before you begin the dual-boot configuration and deployment, be sure to read through all of the procedures and verify that your configuration meets all requirements. You should also become familiar with the product documentation for the operating systems and for the deployment methods (see Appendix 5 for related links). Be sure to test the deployment procedures on one node before deploying the operating systems to the entire cluster. You should also verify the operation of your cluster by running a small cluster-aware application.
Note: This document describes one particular installation, in which all infrastructure services, ?such as DNS, Active Directory, and DHCP, were installed on the compute cluster head node. Your configuration may be different, and these services may be installed on a dedicated infrastructure server. You may therefore need to modify the procedures described.
Mixed Compute Cluster Configuration
A compute cluster uses several types of servers, some required and some optional. The servers that you need will depend on your cluster configuration.
In this installation, a dedicated deployment server was used for the SUSE Linux Enterprise Server deployment. RIS, as available on the WCCS head node by default, was used to deploy the Windows Compute Cluster Server 2003 OS. An infrastructure server was not used; all services were placed on the head node.
Note: If you are not building a cluster for test or evaluation purposes, you may have services such as DNS, Active Directory, and DHCP, on dedicated servers, such as the infrastructure server described below.
Compute Nodes
， The compute nodes‘ BIOS must be configured so that the Preboot Execution
Environment (PXE) is the first boot option.
， If the compute nodes have two or more Network Interface Cards (NICs), PXE must
boot only on the interfaces connected to the compute cluster‘s private network.
， Manufacturers‘ network drivers may be required for the RIS installation.
Windows Compute Cluster Head Node
One server is required to act as the compute cluster head node. This server can be configured to act as a compute node, as well.
， The head node can provide the required infrastructure services, such as DNS, Active
Directory, and DHCP; this is an option in a test or evaluation environment, or in a
small cluster running on an isolated network.
Note: Windows Internet Connection Sharing (ICS) can assign IP addresses to the
nodes, but you can use a DHCP server on the head node or on the infrastructure
server instead. ICS has an address range limit of 192.168.0, and limits the number of
nodes in a cluster to 250.
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 6
， For RIS, the head node must have two NTFS partitions on the hard disk and must be
configured with at least two NICs.
， Configuration of the infrastructure of the head node is not described in this document.
Refer to the Windows WCCS documentation for information on infrastructure
configuration; for the link, see Appendix 5.
Infrastructure Server
This server is optional in a test or evaluation environment.
， The infrastructure server can provide DNS, DHCP, file services and Active Directory
services if these services are not provided by the head node due to configuration and
performance concerns.
， The infrastructure server can be configured with two NICs, one for the private network
and one for the public network.
Software Requirements and Versions Used
Windows Compute Cluster Server
This installation used Windows Compute Cluster Server 2003 with Service Pack 1 (SP1), which consists of Windows Server 2003 Compute Cluster Edition (CCE) with Service Pack 2 (SP2) and the Microsoft Compute Cluster Pack (CCP) with Service Pack 1 (SP1). The head node used Windows Server 2003 Standard x64 Edition with Service Pack 2(SP2).
The CCP is a combination of interfaces, utilities, and management infrastructure; it can be purchased separately and deployed to an existing server running Windows Server. Other supported operating systems for both head node and compute nodes include Windows Server 2003, Standard x64 Edition; Windows Server 2003, Enterprise x64 Edition; Windows Server 2003 R2 Standard x64 Edition; and Windows Server 2003 R2 Enterprise x64 Edition.
SUSE Linux Enterprise Server
For Linux installation and deployment, SUSE Linux Enterprise Server 10 SP1 was used and verified on x64 hardware with serial attached storage (SAS) drives and included the MPICH (MPI) RPMs. The GRand Unified Bootloader (GRUB) was also used.
Hardware Used
This installation used the Microsoft Enterprise Engineering Center‘s 64-node Skagit cluster for
the verification of this document. There are many possible hardware configurations, and you may need to modify the procedures based on the equipment that you use.
The Skagit cluster uses:
， 4 HP BladeSystem c7000 enclosures, each with:
o At least one GbE2c Ethernet Blade Switch
o At least one 4X DDR IB Switch Module for HP c-Class BladeSystem (IFB)
， 64 HP ProLiant BL465c G1 Server Blade nodes, each with:
o 2 dual-core AMD Opteron Processors 2216 HE at 2.40 GHz (1 MB L2)
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 7
o 8 GB PC2-5300 DDR2 Registered DIMMs
o 1 HP Smart Array E200i
o 2 73.4GB SAS Drives initialized with RAID 2 embedded NC370i Multifunction Gigabit Server Adapters
Prepare for Windows Compute Cluster Server 2003 and SUSE Linux
Enterprise Server 10 SP1 Dual Boot
Before you begin the deployment and configuration of the compute cluster, verify the following:
Check when Prerequisites completed
The PXE boot option is disabled on all of the network adapters on the head node.
If using a Windows infrastructure server, the PXE boot option on all NICs on the infrastructure server must be disabled.
The PXE boot option must be disabled on the Linux deployment server.
All drivers that are not integrated into the Windows setup files by default must be available.
All of the compute cluster hardware must be set up and working.
A naming schema must be provided.
An IP address schema must be provided.
The DHCP service must be installed on the compute cluster head node or on the infrastructure server.
The DHCP scopes for the cluster‘s private and MPI networks must be created.
The DHCP scope options (at a minimum, DNS servers and DNS domain name) must be created.
The DHCP server must be authorized in Active Directory.
The MAC addresses of the network adapters connected to the private network on all
nodes must be available. These are required for configuring the Linux deployment
server and the Windows DHCP service.
The DHCP service must be configured to use static IP addresses for node identification.
DNS must be installed and configured on the head node or on the infrastructure server.
Active Directory must be installed on the head node or on the infrastructure server.
The DHCP service must be configured so that it serves the nodes on both the private network and the MPI network.
All required hotfixes must be available.
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 8
Prepare for Deployment of Operating Systems
Begin dual-boot configuration by preparing for the OS deployment to the compute nodes. Be sure to read through this entire document before beginning installation. It is also important to become familiar with the product documentation; for related links, see Appendix 5.
Prepare for SUSE Linux Enterprise Server Deployment
In this setup, SUSE Linux Enterprise Server 10 SP1 was deployed as the first OS from a dedicated deployment server by using AutoYaST, the Novell automated installation mechanism.
Note: Because AutoYaST uses unicast-based and file-based installation, it might not scale successfully for a very large number of nodes (the maximum simultaneous installations is usually 40-50 nodes).
To prepare for SUSE Linux Enterprise Server deployment, configure the deployment server, create the AutoYaST control file, and integrate the Linux HPC software into the repository.
Configure the Deployment Server
The deployment server must be connected to the private network of the cluster, and must be assigned a static IP address that is in the private network range. The following section describes the procedures that are used to configure the deployment mechanism on SUSE Linux Enterprise Server 10 SP1. It is possible to use other hardware and software configurations. You should modify the procedures based on your particular configuration.
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 9
1. Install SUSE Linux Enterprise Server 10 SP1 on the deployment server. a. Disable the the firewall after the installation. To do this, launch YaST
(Computer->YaST) and open Security and Users->Firewall. In the Service
Start section choose Manually, then choose Stop Firewall Now in the
Switch On and Off section. Click Next and then Accept to save settings and
shut off the firewall. Once you've got the configuration set and have
successfully performed a PXE boot installation, you can re-enable the firewall
with the allowed service ports, if desired (they are: NFS 2049, TFTP 69,
DHCP Server 547 and others as needed).
Dual Boot: Windows Compute Cluster Server 2003 and SUSELinux Enterprise Server 10 10