Software Performance Monitoring

download Software Performance Monitoring

of 15

  • date post

  • Category


  • view

  • download


Embed Size (px)


Software Performance Monitoring. Daniele Francesco Kruse. July 2010. Summary. Performance Monitoring Performance Counters Core and Nehalem PMUs – Overview Nehalem : Overview of the architecture μops flow in Nehalem pipeline Perfmon2 Cycle Accounting Analysis - PowerPoint PPT Presentation

Transcript of Software Performance Monitoring

  • Software Performance MonitoringDaniele Francesco KruseJuly 2010

  • SummaryPerformance MonitoringPerformance CountersCore and Nehalem PMUs OverviewNehalem : Overview of the architectureops flow in Nehalem pipelinePerfmon2Cycle Accounting AnalysisThe 4-way Performance MonitoringPfmCodeAnalyser : a new tool for fast monitoringDavid Levinthal : the expertConclusions2

  • Performance MonitoringDEF : The action of collecting information relatedto how an application or system performsHOW : Obtain micro-architectural level informationfrom hardware performance countersWHY : To identify bottlenecks, and possibly remove themin order to improve application performance3

  • Performance CountersAll recent processor architectures include a processorspecific PMU

    The Performance Monitoring Unit contains several performance counters

    Performance counters are able to count micro-architectural events from many hardware sources (cpu pipeline, caches, bus, etc)4

  • Core and Nehalem PMUs - Overview5Intel Core Microarchitecture PMU


    2 programmable countersIntel Nehalem Microarchitecture PMU


    4 programmable core-counters 1 fixed uncore-counter (UNCORE_CLOCK_CYCLES) 8 programmable uncore-counters

  • Nehalem : Overview of the architecture6Core 0Core 1Core 2Core 3Level 3 Cacheshared, writeback (lazy write) and inclusive (contains L1 & L2 of each core)Integrated Memory ControllerLink to local memory (DDR3)Quick Path InterconnectLink to I/O hub (& to other processors, if present)L1D CacheL1I CacheL2 CacheTLBswritebackwritebackunified, writeback, not-inclusiveDTLB0 & ITLB (1st Level), STLB (unified 2nd Level)

  • ops flow in Nehalem pipeline7We are mainly interested in UOPS_EXECUTED (dispatched) and UOPS_RETIRED (the useful ones).

    Mispredicted UOPS_ISSUED may be eliminated before being executed.Instruction Fetch& Branch Prediction UnitDecoderRetirement& WritebackRe-OrderBufferResourceAllocatorExecutionUnitsReservationStationUOPS_ISSUEDUOPS_RETIREDUOPS_EXECUTED

  • Perfmon2A generic API to access the PMU (libpfm)Developed by Stphane EranianPortable across all new processor micro-architecturesSupports system-wide and per-thread monitoringSupports counting and samplingCPU HardwareLinux KernelGeneric PerfmonArchitectural PerfmonPMUUser spacePfmonOther libpfm-based Appslibpfm8

  • Cycle Accounting AnalysisTotal Cycles (Application total execution time)Issuing opsNot Issuing opsStalled(no work)Not retiring ops(useless work)Retiring ops(useful work)Store-FwdL2 missL2 hitLCPL1 TLB miss9

  • The 4-way Performance Monitoring101. OverallAnalysis2. Symbol LevelAnalysis3. Module LevelAnalysis4. Modular SymbolLevel AnalysisOverall (pfmon)ModularSamplingCounting

  • PfmCodeAnalyser : a new tool for fast monitoring11Unreasonable (and useless) to run a complete analysis for every change in code

    Often interested in only small part of code and in one single event

    Solution: a fast, precise and light singleton class called PfmCodeAnalyser

    How to use it:



    //code to monitor


  • PfmCodeAnalyser : a new tool for fast monitoring12PfmCodeAnalyser::Instance("INSTRUCTIONS_RETIRED", 0, 0, "UNHALTED_CORE_CYCLES", 0, 0, "ARITH:CYCLES_DIV_BUSY", 0, 0, "UOPS_RETIRED:ANY", 0, 0).start();

    Event: INSTRUCTIONS_RETIRED Total count:105000018525Number of counts:10 Average count:10500001852.5

    Event: UNHALTED_CORE_CYCLES Total count:56009070544Number of counts:10 Average count:5600907054.4

    Event: ARITH:CYCLES_DIV_BUSY Total count:28000202972Number of counts:10 Average count:2800020297.2

    Event: UOPS_RETIRED:ANY Total count:138003585913Number of counts:10 Average count:13800358591.3

  • David Levinthal : the expert13Intel senior engineer specialized in performance monitoring of applications

    He is going to be here from the 13th to the 30th of July

    He will give a detailed lecture about performance monitoring on the 21st or 22nd of July in 513-1-024

    Anyone whos interested in partecipating may contact us for details

  • Conclusions14We gave a very brief introduction to Nehalem multicore architecture and to Performance Monitoring using hardware performance counters

    A consistent set of events has been monitored across the 4 different analysis approaches in CMSSW, Gaudi and Geant4 (Cycle Accounting Analysis)

    A monitoring tool has been developed for quick performance monitoring: PfmCodeAnalyser

    The report and background of the work done for CMSSW is available at

  • Questions ?