DOC

NA_UserGuide v33

By Robert Fox,2014-07-10 23:43
8 views 0
NA_UserGuide v33

    NMON_Analyser User Guide Version 3.1 Last update 10-07-11 23:40

    NMON_Analyser User Guide for

    V3.3

    Preface

    NMON_Analyser is designed to complement NMON (Nigel’s Monitor) in analysing and

    reporting performance problems; it produces graphs for virtually all sections of output created

    using the “spreadsheet output” mode of NMON as well as doing some additional analyses for ESS, EMC and FAStT subsystems. It will also work with files produced by topasout and with

    other tools that produce data in “NMON” format. It is written in VBA for Excel and will work with Excel 2003 or later.

NMON was originally written by Nigel Griffiths (nag@uk.ibm.com) and is now (since AIX 5.3

    TL09 and AIX 6.1 TL02) part of topas. NMON_Analyser is written/maintained by Stephen

    Atkins (steve_atkins@uk.ibm.com) . Both Nigel and Stephen are members of the Systems and Technology Group in IBM.

Support for NMON_Analyser is provided on a best efforts basis. Please direct questions to the

    User Forum (see below) rather than contact the author direct.

Links:

     NMON_Analyser home page

     http://www-941.haw.ibm.com/collaboration/wiki/display/Wikiptype/nmonanalyser

     User Forum

     http://www.ibm.com/developerworks/forums/dw_forum.jsp?forum=749&cat=56

    New in V3.3

    1. Support for new features in NMON12 for POWER6 and AIX V6.1

    2. Improved support for topasout and Linux

    3. Dynamic invokation of Windows SORT command for large files

    4. Automatic support for locales that use comma as a decimal separator

    5. New facility to allow merging of NMON files for trend analysis

    6. Automated creation of user-defined PivotChart

    7. Removal of SVCTimes option (replaced by new sheet from NMON12)

    8. Removal of DISKSORT option in preparation for Analyser V4

    9. Changed REPROC option to work in all multi-file situations

    10. New summary workbook for multi-file mode

Stephen Atkins, MBCS CITP

    Consulting IT Specialist

    IBM Systems and Technology Group

    IBM UK

    Stephen Atkins/UK/IBM or steve_atkins@uk.ibm.com

    Page 1 of 29

NMON_Analyser User Guide Version 3.0 Last update 10-07-11 23:40

    Table of Contents

    Preface ................................................................................................................................................................... 1 New in V3.3 ....................................................................................................................................................... 1

     Table of Contents ................................................................................................................................................... 2

    Collecting data using NMON.................................................................................................................................. 4 Collecting data using topas (xmwlm) ...................................................................................................................... 5 Using NMON_Analyser ......................................................................................................................................... 5 Analyser options ................................................................................................................................................. 6 National Language Settings ................................................................................................................................ 7

     Batch Processing Options ................................................................................................................................... 7

    Formatting Options ............................................................................................................................................. 7 Pivot Chart ......................................................................................................................................................... 8 Printing Options ................................................................................................................................................. 8 Web Publishing Options ..................................................................................................................................... 8 Interpreting the output sections ............................................................................................................................... 8 Notes on the “Weighted Average” as used in the Analyser .................................................................................. 8 SYS_SUMM ...................................................................................................................................................... 9 AAA ................................................................................................................................................................ 10 BBBB .............................................................................................................................................................. 11 BBBC .............................................................................................................................................................. 11 BBBD .............................................................................................................................................................. 11 BBBE............................................................................................................................................................... 11 BBBG .............................................................................................................................................................. 11 BBBL............................................................................................................................................................... 11 BBBN .............................................................................................................................................................. 11 BBBP ............................................................................................................................................................... 11 BBBR .............................................................................................................................................................. 12 BBBV .............................................................................................................................................................. 12 CPUnnn ........................................................................................................................................................... 12 CPU_ALL ........................................................................................................................................................ 12 CPU_SUMM .................................................................................................................................................... 12 General notes for DISK, ESS, EMC, FASt and DG sheets ................................................................................. 12 EMC/PowerPath subsystems ......................................................................................................................... 13 FAStT subsystems ........................................................................................................................................ 13 DGBUSY ......................................................................................................................................................... 14 DGREAD ......................................................................................................................................................... 14 DGSIZE ........................................................................................................................................................... 14 DGWRITE ....................................................................................................................................................... 14 DGXFER ......................................................................................................................................................... 14 DISKBSIZE ..................................................................................................................................................... 14 DISKBUSY...................................................................................................................................................... 14 DISKREAD ..................................................................................................................................................... 14 DISKSERV ...................................................................................................................................................... 14 DISKWAIT ...................................................................................................................................................... 14 DISKWRITE .................................................................................................................................................... 14 DISKXFER ...................................................................................................................................................... 15 DISK_SUMM .................................................................................................................................................. 15 DONATE ......................................................................................................................................................... 15 EMCBSIZE/FAStBSIZE .................................................................................................................................. 15 EMCBUSY/FAStBUSY ................................................................................................................................... 15 EMCREAD/FAStREAD................................................................................................................................... 15 EMCWRITE/FAStWRITE ............................................................................................................................... 15 EMCXFER/FAStXFER .................................................................................................................................... 15 Page 2 of 29

NMON_Analyser User Guide Version 3.1 Last update 10-07-11 23:40

    EMCSERV/FAStSERV .................................................................................................................................... 15 ERROR ............................................................................................................................................................ 16 ESSBSIZE ....................................................................................................................................................... 16 ESSBUSY ........................................................................................................................................................ 16 ESSREAD ........................................................................................................................................................ 16 ESSWRITE ...................................................................................................................................................... 16 ESSXFER ........................................................................................................................................................ 16 FCREAD.......................................................................................................................................................... 16 FCWRITE ........................................................................................................................................................ 16 FCXFERIN ...................................................................................................................................................... 16 FCXFEROUT .................................................................................................................................................. 16 FILE ................................................................................................................................................................ 16 FRCA............................................................................................................................................................... 17 IOADAPT ........................................................................................................................................................ 17 IP ..................................................................................................................................................................... 17 JFSFILE ........................................................................................................................................................... 17 JFSINODE ....................................................................................................................................................... 18 LAN ................................................................................................................................................................. 18 LARGEPAGE .................................................................................................................................................. 18 LPAR ............................................................................................................................................................... 18 MEM ............................................................................................................................................................... 19 MEMUSE ........................................................................................................................................................ 19 MEMNEW ....................................................................................................................................................... 20 MEMPAGES4K/64K/16MB/16GB .................................................................................................................. 20 MEMREAL...................................................................................................................................................... 20 MEMVIRT....................................................................................................................................................... 21 NET ................................................................................................................................................................. 21 NETPACKET .................................................................................................................................................. 21

     NETSIZE ......................................................................................................................................................... 21

    NFS sheets ....................................................................................................................................................... 21 PAGE............................................................................................................................................................... 21 POOLS ............................................................................................................................................................ 22 PROC............................................................................................................................................................... 22 PROCAIO ........................................................................................................................................................ 23 RAWCPUTOTAL, RAWLPAR ....................................................................................................................... 23 TCPUDP .......................................................................................................................................................... 23 TOP ................................................................................................................................................................. 23 Interpreting the %Processor by PID chart ...................................................................................................... 25 UARG .............................................................................................................................................................. 25 VM .................................................................................................................................................................. 26 WPAR sheets ................................................................................................................................................... 26 WLM sheets ..................................................................................................................................................... 26 ZZZZ ............................................................................................................................................................... 26 Error Handling ..................................................................................................................................................... 26 Common problems ........................................................................................................................................... 26 Known bugs/problems (V3.3, topas_nmon) ....................................................................................................... 27 How to report a problem ....................................................................................................................................... 27 Excel/VBA Resources/Links ................................................................................................................................ 27 Appendix: Notes on Batch Operation .................................................................................................................... 29 Sample .bat files ............................................................................................................................................... 29 getcsv.bat ..................................................................................................................................................... 29 analyser.bat................................................................................................................................................... 29 putxls.bat ...................................................................................................................................................... 29 Control.txt .................................................................................................................................................... 29 NMON Analyser Batch.xls ........................................................................................................................... 29

Page 3 of 29

NMON_Analyser User Guide Version 3.0 Last update 10-07-11 23:40

    Collecting data using NMON

    Please make sure you have the latest versions of both NMON and NMON_Analyser before starting a new engagement. If you want automatic notification of a new release of the Analyser

    send a note to steve_atkins@uk.ibm.com and I’ll add you to my distribution list. Nigel maintains a similar list for NMON.

You will need to have root privileges in order to get a complete set of data on the BBBP sheet.

    In order to collect data for the DISKBUSY sheets you need to make sure that iostat data collection is enabled:

     chdev -l sys0 -a iostat=true

    For spreadsheet output mode (comma separated values) use the following flags when invoking nmon:

    -f spreadsheet output format [note: default -s300 -c288]

     Output file is _YYYYMMDD_HHMM.nmon

    -F same as -f but user supplied filename

    -c number of snapshots

    -d requests disk service and wait times (DISKSERV and DISKWAIT)

    -i Ignore processes using less than this amount of CPU when generating

    TOP section useful for reducing data volumes

    -g file containing disk group definitions

    -l number of hdisks per sheet - defaults to 150, maximum 250. See notes

    -m

NMON changes to this directory before saving the file

    -r goes into spreadsheet file [default hostname]

    -s interval between snap shots

    -x capacity planning (15 mins for 1 day = -fdt -s900 -c96)

     -t include top processes in the output

    -T as t plus saves command line arguments in UARG section

    -A include data for async I/O (PROCAIO) sections

    -D prevents DISK sections being produced (useful when Disk Groups are

    being used because there are too many hdisks to process)

    -E stops ESS sections being produced (necessary when Disk Groups are

    being used because there are too many vpaths to process)

    -J prevents JFS sections being produced (prevents Excel errors when you

    have more than 255 filesystems)

    -L includes LARGEPAGE section

    -N include NFS sections

    -S include WLM sections with subclasses

    -W include WLM sections without subclasses

    -Y include SUMMARY section (very efficient alternative to t if PID level

    data is not required)

     example: nmon_aix51 -F asterix.nmon -r Test1 -s6 -c12

Notes:

    1. The f (or F) flag must appear first.

    Page 4 of 29

NMON_Analyser User Guide Version 3.1 Last update 10-07-11 23:40

    2. The value of the -l flag controls the number of hdisks per sheet on the DISK sheets and per

    line on the BBBD sheet. There are two factors to consider when choosing this value.

    Excel has a limit of 256 columns per sheet; however, both NMON and NMON_Analyser use

    some columns, so the upper limit is really 250. The second factor is that Excel VBA has

    an upper limit of 2048 bytes for input line length. This particularly affects users of EMC

    systems that use long hdisk names (e.g. hdiskpower123). The default of 150 is safe for

    such systems. Other users may set the value to 250 in order to reduce the number of output

    sheets.

    3. Consider the value of the -s flag very carefully. The shorter the interval between

    snapshots, the more variable the values for each resource will be. If you use an interval of 1

    second, don’t be surprised to see many of your disks hitting 100% busy for short periods.

    For normal monitoring, 10-minute intervals (-c 600) provide a good balance.

    4. The graphs produced by NMON_Analyser look best when the number of snapshots

    (specified by the -c flag) is 300 or less.

    5. The TOP section (produced by specifying the -t flag) can generate large amounts of output

    and the size of the output can grow exponentially if a large value is specified for the -c flag.

    If you want the TOP section then specify no more than 250 snapshots - ideally less. Collecting data using topas (xmwlm)

    You need AIX V5.3 TL5 Service pack 4 with APAR IY87993 or later. Note that only the output using the a flag can be analysed. In particular, cross-partition statistics cannot be analysed; if you wish to get a report for the entire machine, collect data from each LPAR separately and then use NMON_Consolidator to merge the data. Commands like the following are required to collect the data.

    topasout -a /etc/perf/daily/xmwlm.yymmdd

    :

    cp /etc/perf/daily/xmwlm.yymmdd_01 hostname.ddmmyy.topasout.csv

Using NMON_Analyser

    ; FTP the input file to your PC ideally using the ASCI or TEXT options to make sure that

    lines are terminated with the CRLF characters required by Windows applications. ; Open the NMON_Analyser spreadsheet and specify the options you want on the “Analyser”

    and “Settings” sheets (see below). Save the spreadsheet if you want to make these options

    your personal defaults.

    ; Click on the "Analyse nmon data" button and find/select the .nmon file(s) to be processed.

    You may select several files in the same directory. If you wish to process several files in

    different directories you may wish to consider using the “FILELIST” option described below.

    ; You may see the message SORT command failed for “filename” if the file has >65K

    lines and the filename (or directory name) contains blanks or special characters. Either

    rename the file/directory or just pre-sort the file before using the Analyser.

    Page 5 of 29

NMON_Analyser User Guide Version 3.0 Last update 10-07-11 23:40

    Analyser options

GRAPHS The first option is either ALL or LIST. If the value is LIST then only those sheets

    which appear in the LIST on the Settings sheet will have graphs drawn for them.

    This option is particularly useful if the graphs are to be printed/published or to

    reduce the amount of memory/fonts/disk space required when analysing files from

    large systems.

     The second is either CHARTS, PICTURES, PRINT or WEB. The meaning of these

    are as follows:

    ; CHARTS produce Excel charts in-place on the selected sheets

    ; PICTURES graphs will be produced on a separate “Charts” sheets as

    pictures. Selecting this option can reduce the size of the

    output file by up to 90%.

    ; PRINT implies PICTURES. Pictures will be printed to the

    designated printer (see Printing Options” below)

    ; WEB implies PICTURES. Automated web publishing (see

    “Web Publishing” below)

    INTERVALS specifies the first and last time interval to be processed. Intervals outside this

    range will be discarded after parsing. Note that these are numbers between 1-

    99999 and are not time values. Setting a value of 2 for the first interval is

    useful in discarding the very large numbers that often appear at the start of an

    NMON collection run with AIX. If you have used a splitter program on the

    input file, or if you are analysing data from a LINUX system, then you should

    leave this as 1.

    MERGE specifying YES here results in NMON_Analyser merging all of the input files to

    form a single file. The input files must be unsorted. By default the Analyser

    will delete the TOP and UARG sections during the merge process; specify TOP to

    prevent this from happening but be aware that processing time will be increased

    and that if the TOP section exceeds 65K lines then data will be lost. Specify KEEP

    to stop the merged file from being deleted at the end of the run. Specify ONLY if

    you simply want to create a merged file for future analysis.

    PIVOT specifying YES here results in NMON_Analyser creating a Pivot Chart from the

    specified sheet after all other processing (including printing/publishing the other

    charts) has completed. See additional parameters on the “Settings” sheet.

    ESS specifying NO here results in NMON_Analyser bypassing the additional analysis

    performed for ESS subsystems. This will result in faster analysis and can allow

    larger files to be analysed successfully when “out of memory” errors occur.

    FILELIST the name of a control file containing a list of nmon output files to be processed by

    the Analyser. Leave this field blank for normal operation. The name must be

    fully qualified (e.g. c:\nmon\testcases\filenames.txt). The names

    specified in the text file must contain full path information. Wildcard characters

    may be included in the filenames so long as they conform to Windows standards.

    For example:

     c:\nmon\testcases\*.nmon

    Page 6 of 29

NMON_Analyser User Guide Version 3.1 Last update 10-07-11 23:40

    NB: if you save the spreadsheet with a value in this field, the Analyser will

    automatically begin execution the next time you open it. This is defined as

    “batch mode” (see Appendix). You can stop the execution by pressing

    Ctrl+Break or by deleting/renaming the Control File.

    National Language Settings

    The following fields can be found on the “Settings” sheet.

    SORTINP Change this to NO if you use AIX (or the nmon2csv script) to pre-sort the file.

    This option is primarily useful in countries where the Windows collating sequence

    sorts the string "AA" after "Z" (e.g. Denmark and Norway).

    Batch Processing Options

    The following fields can be found on the “Settings” sheet.

    REPROC Change this to NO if you want to bypass processing of input files which may have

    been processed in a previous run. This is useful if you make use of wildcards in

    the batch control file. Note that the REPROC option only takes effect when you

    have specified more than one input file.

    OUTDIR the name of an existing directory in which output files will be saved by default.

    This is primarily intended for batch operation (see FILELIST above) but also works

    to set the default directory for interactive sessions. If the directory does not exist

    (or OUTDIR is blank) then output files are saved back to the same directory as the

    corresponding input files.

     Example: C:\NMON\Analyser\Output\

    Formatting Options

    The following fields can be found on the “Settings” sheet.

    BBBFont Enter the name of a fixed pitch font to be used for formatting the BBBC and

    BBBP sheets. Courier provides acceptable results.

    GWIDTH Change the values in this row to make the generated graphs bigger or smaller.

    The default value of 0 means that the Analyser will dynamically size the graphs

    according to your screen size, font settings or page size. Be careful not to set a

    value larger than your page width when printing.

    GHEIGHT Change the values in this row to make the generated graphs bigger or smaller. If

    you specify a value here you must also specify a value for GWIDTH. LIST A comma-separated list a sheets for which the Analyser is to draw graphs. Only

    used if the GRAPHS option is set to LIST.

    NOLIST The default is KEEP. If you change this to DELETE then all sheets which do not

    appear in LIST will be deleted after analysis. This can dramatically reduce the

    size of files that are to be kept for long periods.

    REORDER specifying YES here results in NMON_Analyser reordering the sheets to improve

    navigation to more relevant information.

    TOPDISKS the maximum number of hdisks/vpaths to include on disk graphs. A value of 0

    produces graphs containing all the hdisks on a sheet (up to 250). Graphs Page 7 of 29

NMON_Analyser User Guide Version 3.0 Last update 10-07-11 23:40

    containing more than 50 hdisks will be automatically scaled to fit and may

    therefore exceed the size of the screen.

    xToD Format to be used for timestamps on Time of Day graphs. Anything acceptable

    to Excel as a Number Format Code may be entered. Default is hh:mm.

    Note that the date is also available within the timestamp and you may therefore

    use something like dd-mmm-yy hh:mm if, for example, you have merged multiple

    NMON files together. If you use something other than the default string you

    may need to increase the value of GHEIGHT experiment with different values if

    you don’t see what you expect.

    Pivot Chart

    These parameters are used to construct a pivot chart. The required parameters are: Sheetname, PageField, RowField, ColumnField, DataField and xlFunction (can be COUNT, SUM,

    MIN, AVG, MAX). This is primarily useful for the TOP and SUMMARY sheets but might prove useful for other, possibly user-supplied, data sheets.

    Printing Options

    The following fields can be found on the “Settings” sheet. Note - these only take effect if you

    select PRINT for the OUTPUT option on the Analyser sheet.

    LSCAPE Change to YES if you want the Analyser to set the page orientation to Landscape.

    By default the Analyser will fit one chart per page when printing landscape. COPIES Set to the number of copies to be printed.

    PRINTER The name of the printer. Specifying a value of PREVIEW will cause the Analyser

    to invoke the Excel print preview function - useful for testing. You may also

    specify DEFAULT to print to the system default printer or the name of network

    printer.

    The Analyser adds page headers and footers.

    Web Publishing Options

    The following fields can be found on the “Settings” sheet.

    PNG Change to NO if your browser can’t handle the PNG graphics format. Graphics

    will be generated as GIF files.

    SUBDIR If this is YES then all supporting files, such as background textures and graphics,

    are organized in a separate folder. If this is NO then supporting files are saved in

    the same folder as the Web page.

    WEBDIR the name of an existing directory in which HTML files will be saved by default.

    If the directory does not exist (or OUTDIR is blank) then output files are saved

    back to the same directory as the corresponding input file.

     Example: C:\NMON\Analyser\HTML\

    Interpreting the output sections

    Notes on the “Weighted Average” as used in the Analyser

    Several graphs produced by the Analyser show average, weighted average and maximum values. Although everyone understands averages and maximums, the concept of a weighted average is a little more difficult to grasp.

    Page 8 of 29

NMON_Analyser User Guide Version 3.1 Last update 10-07-11 23:40

    One of the problems we are faced with in analysing sample data is that resources on the target system may be idle for long periods during the collection. For example, the NMON data collection may be started some time before the system reaches peak utilisation and may not be stopped until the workload being monitored has long since finished. Although this does not affect the maximums it can severely affect the accuracy of the averages.

    The idea of a weighted average is to apply a weighting factor to each snapshot to indicate how relevant that snapshot is to the average. In NMON_Analyser, we use the value of the measurement itself as the weighting factor. In effect, this produces a figure that shows how busy a resource is when it is active. For example: a database log disk is only active during the middle part of a benchmark. We record the following figures for %tm_act (DISKBUSY):

    Snapshot %tm_act

    1 0

    2 0

    3 0

    4 0

    5 0

    6 6.1

    7 6.3

    8 6.5

    9 9.1

    10 5.9

    11 0

    12 0

    13 0

    14 0

    15 0

    The average for this set of data is 2.3 and the weighted average is 7.0. The weighted average gives a better picture of how busy the disk is while logging is taking place. NMON_Analyser uses the weighted average as a sort key when sorting the contents of disk sheets.

    Note that, occasionally, NMON generates very large numbers for the first interval and this heavily skews the weighted average values. If you see this problem occurring then change the value for the FIRST parameter to 2 in order to exclude the first interval completely. This is only a problem for AIX.

    SYS_SUMM

    This section is entirely generated by the Analyser and contains a useful summary of data taken from other sheets. Note the the avg/max values for User%, Sys%, Wait% and Idle% are independent and will not add up to 100%. The CPU% column shows avg/max values for the sum of usr%+sys% during each interval. For micro-partitions, the values are shown as a percentage of the Virtual Processors they do not relate to the CPU_ALL values.

    Page 9 of 29

NMON_Analyser User Guide Version 3.0 Last update 10-07-11 23:40

    For non-partitioned or dedicated CPU partitions the graph shows the total CPU Utilisation (%usr + %sys) together with the Disk I/O rate (taken from the DISKXFER sheet) by time of day. For micro-partitions the graph shows the number of physical CPUs being used instead of CPU%.

The value “Max:Avg” is simply the maximum value divided by the average. If monitored over

    a long period of time the value for CPU% can be useful in spotting a system reaching saturation level (the ratio will steadily decrease). If you have historical data then the value can also be useful in determining how much latent demand is present in a system running at saturation level.

    AAA

    The AAA section is generated by NMON at the start of the data collection and contains information about the system and NMON itself contents vary by release, following is for 12e

    AIX this is the release / maintenance level of AIX being used on the target system as

    reported by the lslpp command.

    build the particular build of nmon used to collect this data

    command the command line used to invoke NMON.

    cpus the number of CPUs in the system and the number active at the start of data

    collection.

    date date at the start of the collection.

    disks_per_line the setting of the -l flag on the NMON command line or the default value.

    hardware the processor technology used in the target system.

    host the hostname of the target system.

    Interval the time (in seconds) between snapshots.

    kernel information from the kernel - useful in identifying the type of kernel (32-bit or 64-

    bit) and whether this is an LPAR.

    LPARNumberName the LPAR number followed by the name

    MachineType machine type and model of the system

    progname the name of the NMON executable.

    runname taken from the NMON command line if specified using the -r parameter, else

    defaults to hostname.

    SerialNumber the machine serial number

    snapshots the number of snapshots - this is used by the Analyser. The Analyser will

    modify this value to match the number of snapshots actually found in the input

    file.

    subversion detailed information about the nmon version used to collect this data including

    the date and time it was created.

    time time as shown by the system clock at the start of the collection. timestampsize the number of characters used for timestamps in each record. The default is 5. TL the Technology Level of the AIX release

    user the name of the user executing the NMON command.

    version the version of NMON used to collect this data.

    VIOS the release/maintenance level if this is a VIOS LPAR.

    analyser the version of NMON_Analyser used to generate the output file together with the

    elapsed time (in seconds) for processing this file.

    Page 10 of 29

Report this document

For any questions or suggestions please email
cust-service@docsford.com