DOC

HP-UX_RAC_Test_Plan

By Holly Matthews,2014-06-22 18:32
9 views 0
HP-UX_RAC_Test_Plan

    HP Services – MSDD Enterprise Solutions Integration

    Infrastructure Test Plan

    for

    Oracle Real Application Clusters

    on HP-UX with MC Service Guard

     System, Network, Database, Backup & Recovery

    Test Information and Results

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    Authors

    Venkatesh Iyer - HPS MSDD ESI Americas

    Courtney Llamas - HPS MSDD ESI HIA

    Additional Contributors

    Bill Clements - HPS ??

    Serge Dingelhoff HPS – MSDD ESI HIA

    Bethany Lapaglia HPS – MSDD ESI HIA

    Marc Zablonski HPS – MSDD ESI HIA

    HP Services MSDD2 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    Table of Contents

     1INTRODUCTION............................................................................................................................... ................................................................................................ 4

     2HIGH AVAILABILITY TESTS............................................................................................................................... ......................................................................... 5

     2.1 SYSTEM TESTS............................................................................................................................... .................................................................................................. 5

     2.1.1Reboot Tests............................................................................................................................... ............................................................................................. 5

     2.1.2System Boot-up of Mirror Disk............................................................................................................................... ................................................................ 6

     2.1.3 PV Link Tests – Tux410............................................................................................................................... .......................................................................... 7

     2.1.4PV Link Tests – Tux411............................................................................................................................... ........................................................................... 82.2 MC/S ERVICE GUARD SPECIFIC TESTS............................................................................................................................... ................................................................. 10

     2.2.1MC/SG Cluster Reformation After Bootup............................................................................................................................... ............................................ 10

     2.2.2Heartbeat LAN Failover Test............................................................................................................................... ................................................................ 11

     2.2.3Test Application LAN Failover............................................................................................................................... .............................................................. 12

     2.2.4System Fail Test .................................................................................................................................................................................................................... 13

     2.3 ORACLE RAC SPECIFIC TESTS............................................................................................................................... .......................................................................... 15

     2.3.1Oracle Controlled Startup and Shutdown............................................................................................................................... ............................................. 15

     2.3.2Simultaneous Startup and Shutdown............................................................................................................................... ..................................................... 17

     2.3.3Oracle Process Failure and Instance Recovery............................................................................................................................... .................................... 18

     2.3.4Additional Oracle Services............................................................................................................................... .................................................................... 22

     2.3.5Transparent Application Failover (TAF)............................................................................................................................... .............................................. 22

     2.3.6Oracle DataGuard ............................................................................................................................... ................................................................................ 23

     3DATABASE BACKUP & RECOVERY............................................................................................................................... .......................................................... 273.1 D ATABASE BACKUP & RECOVERY USING DATA PROTECTOR AND RMAN............................................................................................................................... ............... 27

     3.1.1Full Offline Backup and Restore............................................................................................................................... ........................................................... 27

     3.1.2Full Online Backup and Restore............................................................................................................................... ............................................................ 27

     3.1.3Restarting Online Backup............................................................................................................................... ...................................................................... 29

     3.1.4Full Online Backup Including Archive Logs............................................................................................................................... ......................................... 30HP Services MSDD3 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard 1Introduction

    This document outlines the MC/ServiceGuard and Oracle Real Application Cluster (RAC) implementation test procedures. It is designed to document the system, network, database, backup and recovery tests completed during the testing process. This will demonstrate that the MC/ServiceGuard and Oracle RAC has been installed and configured as designed in a highly available environment. HP Services MSDD4 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    2High Availability Tests

    2.1 System Tests

    2.1.1Reboot Tests

    1System Reboot - Reboot all Systems – Confirm they all boot up normally with any errors before any testing. Ensure that all processes

    that are required have been started up. Shut application packages down before test. Ensure that only the cluster is running.

    AReboot Node “tux411” – Node “tux411” reboots After the system reboot,

    .Perform reboot after normally. No errors during all components should

    1removing node from the system startupfunction normally without

    MC/ServiceGuard cluster.any disruption. The With TCP Keep Alive set to

    Run cmrunnode after reboot Database Instance and 6 minutes, all connections

    cmhaltnodefor system to rejoin the Listener on “tux410” failed over to surviving

    shutdown –r 0cluster.should still be available node after 6 minute lapse.

    without interruption.

    With TCP Keep Alive at

    Database Instance and default of 2 hours, all

    Listener should be connections on failed node

    restarted automatically hung indefinitely.

    after reboot on “tux411”

    B.Reboot Node “tux410” – Node “tux410” reboots After the system reboot, With TCP Keep Alive set to 1Perform reboot after normally. No errors during all components should 6 minutes, all connections

    removing node from the system startup.function normally without failed over to surviving

    MC/ServiceGuard cluster.any disruption. The node after 6 minute lapse.

    Make sure that Node Run cmrunnode after reboot Database Instance and

    “tux411” is up and has for system to rejoin the Listener on “tux411” With TCP Keep Alive at

    joined the cluster.cluster.should still be available default of 2 hours, all

    without interruption.connections on failed node

    cmhaltnodehung indefinitely.HP Services MSDD5 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    shutdown –r 0Database Instance and

    Listener should be

    restarted automatically

    after reboot on “tux410”

    2.1.2System Boot-up of Mirror Disk

    2Boot-up Off Mirror - Test of Mirrored Boot Disk. Manually boot of mirrored disk by interrupting boot process and entering mirror disk path.

    AReboot Node “tux411” – Node “tux411” reboots After the system reboot, .Perform reboot after normally. No errors during all components should 1removing node from the system startupfunction normally without MC/ServiceGuard cluster.any disruption. The

    Database Instance and cmhaltnodeRun cmrunnode after reboot Listener on “tux410” shutdown –r 0for system to rejoin the should still be available

    cluster.without interruption.Halt reboot and enter

    alternate path for bootDatabase Instance and

    Listener should be

    restarted automatically

    after reboot on “tux411”.B.Reboot Node “tux410” – Node “tux410” reboots After the system reboot, 1Perform reboot after normally. No errors during all components should removing node from the system startup.function normally without MC/ServiceGuard cluster.any disruption. The Make sure that Node Run cmrunnode after reboot Database Instance and “tux411” is up and has for system to rejoin the Listener on “tux411” joined the cluster.cluster.should still be available

    without interruption.

    HP Services MSDD6 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    cmhaltnode

    shutdown –r 0Database Instance and

    Listener should be

    Halt reboot and enter restarted automatically

    alternate path for bootafter reboot on “tux410”

    2.1.3 PV Link Tests – Tux410

    2.Test PVLinks – Test LVM PVLinks on each system by disabling the connection from the disk / SAN to the server. Simply remove the

    3fibre channel cable/s from the server (no sparks fly!). Vgdisplay –v vgname will display the primary and alternate disks. Observe

    messages in syslog. Run ioscan and xpinfo before and after removal of each link.APackage Running on Node Disk LDEV’S are distributed There should be no

    .“tux410” and “tux411”across the both paths. interruption with the

    1current or existing user’s No errors reported in alert Fail Fibre link on Node Alternate Link (0/0/12/0) sessions. All activities logs, no effect on the “tux410”. Disconnect the will takeover all LDEV’s.should be continuously sessions or database.primary link on path active. No data loss

    0/0/10/0.expected and full access to

    the application and data

    After test plug primary should be preserved.Primary link is restoredlink back in. Wait for 2

    minutes before next test.No interruption to the

    system or SAN.

    Package Running on Node Disk LDEV’S are distributed There should be no B.

    “tux410” and “tux411”across the both paths. interruption with the

    current or existing user’s No errors reported in alert Fail Fibre link on Node Alternate Link (0/0/10/0) sessions. All activities logs, no effect on the

    HP Services MSDD7 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    “tux410”. Disconnect the will takeover all LDEV’s.should be continuously sessions or database.primary link on path active. No data loss

    0/0/12/0.expected and full access to

    the application and data

    After test plug primary should be preserved.Primary link is restoredlink back in. Wait for 2

    minutes before next test.

    No interruption to the

    system or SAN.

    2.1.4PV Link Tests – Tux411

    3.Test PVLinks – Test LVM PVLinks on each system by disabling the connection from the disk / SAN to the server. Simply remove the 3fibre channel cable/s from the server (no sparks fly!). Vgdisplay –v vgname will display the primary and alternate disks. Observe messages in syslog. Run ioscan and xpinfo before and after removal of each link.CPackage Running on Node Disk LDEV’S are distributed There should be no

    .“tux410” and “tux411”across the both paths. interruption with the

    1current or existing user’s No errors reported in alert Fail Fibre link on Node Alternate Link (0/0/12/0) sessions. All activities logs, no effect on the “tux410”. Disconnect the will takeover all LDEV’s.should be continuously sessions or database.primary link on path active. No data loss

    0/0/10/0.expected and full access to

    the application and data

    After test plug primary should be preserved.Primary link is restoredlink back in. Wait for 2

    minutes before next test.No interruption to the

    system or SAN.

    HP Services MSDD8 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    DPackage Running on Node Disk LDEV’S are distributed There should be no

    “tux410” and “tux411”across the both paths. interruption with the .

    current or existing user’s No errors reported in alert Fail Fibre link on Node Alternate Link (0/0/10/0) sessions. All activities logs, no effect on the “tux410”. Disconnect the will takeover all LDEV’s.should be continuously sessions or database.primary link on path active. No data loss

    0/0/12/0.expected and full access to

    the application and data

    After test plug primary should be preserved.Primary link is restoredlink back in. Wait for 2

    minutes before next test.

    No interruption to the

    system or SAN.

    HP Services MSDD9 of 30HP ConfidentialA.01.00 6/22/2013

     Infrastructure Test Plan for Real Application Clusters on HP-UX with MC Service Guard

    2.2 MC/Service Guard Specific Tests

    2.2.1MC/SG Cluster Reformation After Bootup

    Test MC/SG Cluster Reformation After Bootup – Node will join cluster. This assumes that one node is up and the cluster is up.4.

    1

    AReboot Node “tux411” – Cluster will reform and node Package continues to run on

    .Package “lh1dbci” “tux411” will join the “tux410”. Database Instance

    Arunning on “tux410” cluster.and Listener continue to run With TCP Keep Alive

    on “tux410”, upon reboot of set to 6 minutes, all

    “tux411”, Database Instance connections failed over

    and Listener will restart. to surviving node after 6

    Remastering of resources will minute lapse.

    occur in the Database.

    With TCP Keep Alive at

    default of 2 hours, all

    connections on failed

    node hung indefinitely.Reboot Node “tux410” – Cluster will reform and node Package continues to run on B.

    Package “lh1dbci” “tux410” will join the “tux411”. Database Instance

    running on “tux410”. cluster.and Listener continue to run With TCP Keep Alive

    on “tux411”, upon reboot of set to 6 minutes, all

    “tux410”, Database Instance connections failed over Halt package “lh1dbci” and Listener will restart. to surviving node after 6 and move it to “tux411”Remastering of resources will minute lapse.cmhaltpkg lh1dbcioccur in the Database.

    cmrunpkg lh1dbci –n With TCP Keep Alive at tux411default of 2 hours, all

    connections on failed After reboot startup cluster node hung indefinitely.Package will restart on services on Node “tux410”“tux410” and cluster will be cmrunnode –n tux410in “normal” state.

    HP Services MSDD10 of 30HP ConfidentialA.01.00 6/22/2013

Report this document

For any questions or suggestions please email
cust-service@docsford.com