How to: Take hang and crash dumps using Procdump

Today we are going to see how we can user Procdump to generate dump in differente scenarios. With the help of Procdump we have the ability to generate dumps based on performance counter values.

 Advantages over adplus:

1.       No need to install Debugging tools for Windows
2.       Light (approx. 300kb) instead of large Debugging tools package (>10MB)
3.       Does not require cscript (for adplus.vbs) or .NET framework (adplus.exe)
4.       When attached in crash mode – procdump CAN be detached WITHOUT terminating process – but is not              recommended in production Servers
5.       No excessive logging is generated
6.       No need to set Perfmon Alerts/Data Collector Sets to trigger batch files.
7.       Dumps generated based on performance counter threshold have greatest chance of getting dump from time of issue  instead of manually triggered dumps.

Lets take a look on different scenarios . 

Scenarios:
1.       Version store 623:
 
i.       Start Perfwiz or ExPerfwiz (based on Exchange version)
ii.      Attach Procdump
d:\>procdump -ma store.exe -p "\Database(Information store)\Version buckets allocated" xValue -s 1 -n 3 -accepteula d:\dumps\store_623.dmp
Where xValue is number as calculated using event 623 as per
http://msexchangeteam.com/archive/2006/04/19/425722.aspx
iii.      Stop Perfwiz/ExPerfwiz when dumps have been generated
iv.      Get blg and .dmp files and also event logs from time of issue

Switches:
-p: Performance counter to monitor
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
-ma: Write a dump file with all process memory
Explanation: The arguments configure Procdump to generate a full (-ma) dump of the Store.exe process when the “Version Buckets” exceed specific calculated value for 1 second (-s 1), to generate up to 3 dumps (-n 3), to save the dumps in the d:\dumps directory with names that begin with “store_623” and then exit
 
 
2.       RPC Latency/Store unresponsive:
 
i.       Start Perfwiz or ExPerfwiz (based on Exchange version)
ii.      Attach Procdump
d:\>procdump -ma store -p "\MSExchangeIS\RPC Requests" xValue -s 30 -n 3 -accepteula d:\dumps\store_rpc.dmp
Where xValue is:
30     – For Exchange 2003
70     – For Exchange 2007 or Exchange 2010
Important: These values are standard recommendations – they would need to be modified on a case by case basis depending on actual values recorded on customer’s Server
iii.     Stop Perfwiz/ExPerfwiz when dumps have been generated
iv.      Get blg and .dmp files and also event logs from time of issue
Switches:
-p: Performance counter to monitor
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
-ma: Write a dump file with all process memory
Explanation: The arguments configure Procdump to generate a full (-ma) dump of the Store.exe process when Store’s “RPC Requests” exceed 30 or 70 (depending on Exchange version) for 30 second2 (-s 30), to generate up to 3 dumps (-n 3), to save the dumps in the d:\dumps directory with names that begin with “store_rpc” and then exit. 
                                                    
 
3.       High CPU issues:
 
i.      Start Perfwiz or ExPerfwiz (based on Exchange version)
ii.      Important – be sure to include “Thread” counter in Log
For Perfwiz – Select All Exchange and all OS counters
For Experfwiz – use with “-threads” switch
iii.      Attach Procdump
d:\>procdump -c 75 -s 10 -n 3 -u store.exe -accepteula d:\dumps\store_cpu_10sec.dmp
Important: These values are standard recommendations – they would need to be modified on a case by case basis depending on actual values recorded on customer’s Server
iv.     Stop Perfwiz/ExPerfwiz when dumps have been generated
v.      Get blg and .dmp files and also event logs from time of issue
Switches:
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
-ma: Write a dump file with all process memory
-c: CPU usage
-u: to monitor CPU usage relative to a single core
Explanation: The arguments configure Procdump to generate a dump of the Store.exe process when Store’s CPU usage exceeds 75% (-c 75) relative to a single core (-u) for 10 seconds (-s 10), to generate up to 3 dumps (-n 3), to save the dumps in the d:\Dumps directory with names that begin with “store_cpu_10sec” and then exit
 
4.       Virtual Memory Fragmentation 9582:
 
i.      Create a batch file – dump.bat
ii.     Copy following code in batch file
==============================================
d:\procdumpv2>procdump -ma store.exe -p "\MSExchangeIS\Exchmem: Number of Additional Heaps" 5 -s 1 -n 3 -accepteula d:\dumps\store_9582_1run.dmp
Echo “Dumps for first run generated”
d:\procdumpv2>procdump -ma store.exe -p "\MSExchangeIS\Exchmem: Number of Additional Heaps" 25 -s 1  -n 3 -accepteula d:\dumps\store_9582_2run.dmp
Echo “Dumps for first run generated”
d:\procdumpv2>procdump -ma store.exe -p "\MSExchangeIS\Exchmem: Number of Additional Heaps" 40 -s 1  -n 3 -accepteula d:\dumps\store_9582_3run.dmp
Echo “ALL Dumps generated”
=============================================
iii.     Start Perfwiz
iv.      Run dump.bat file
v.       Stop Perfwiz when all dumps have been generated
vi.      Get blg and .dmp files and also event logs from time of issue
Important: These values are standard recommendations – they would need to be modified on a case by case basis depending on actual values recorded on customer’s Server
Switches:
-p: Performance counter to monitor
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
-ma: Write a dump file with all process memory
Explanation: The arguments configure Procdump to generate a full (-ma) dump of the Store.exe process when Store’s “Exchmem: Number of Additional Heaps” counter exceeds specified value for 1 second (-s 1), to generate up to 3 dumps (-n 3), to save the dumps in the d:\dumps directory with names that begin with “store_9582_*run” and then exit.
 
 5.     General purpose hang dump – full       

Single dump:
d:\>procdump -ma store.exe -accepteula d:\dumps\store_full_hang.dmp
Concurrent dumps:
d:\>procdump -ma store.exe -s 30 -n 3 -accepteula d:\dumps\store_full_hang.dmp
Get .dmp file and also event logs from time of issue
Switches:
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
-ma: Write a dump file with all process memory
Explanation: The arguments configure Procdump to generate full (-ma) dump of the Store.exe process – either a single dump or upto 3 dumps (-n 3) 30 seconds apart (-s 30), to save the dumps in the d:\dumps directory with names that begin with “store_full_hang” and then exit. 
  
 

6.       General purpose hang dump – mini

Single dump:
d:\>procdump store.exe -accepteula d:\dumps\store_mini_hang.dmp
Concurrent dumps         
d:\>procdump store.exe -accepteula -s 30 -n 3 d:\dumps\store_mini_hang.dmp
Get .dmp file and also event logs from time of issue
Switches:
-s: Consecutive seconds threshold must be hit before dump is written
-n: Number of dumps to write before exiting.
Explanation: The arguments configure Procdump to generate mini dump of the Store.exe process – either single dump or upto 3 dumps (-n 3) 30 seconds apart (-s 30), to save the dumps in the d:\dumps directory with names that begin with “store_mini_hang” and then exit.

7.       Standard crash dump configuration

 d:\>procdump -t -e -ma store.exe -accepteula d:\dumps\store_crash.dmp
Get .dmp file and also event logs from time of issue
Switches:
-t: Write a dump when the process terminates
-e: Write a dump when the process encounters an unhandled exception
Explanation: The arguments configure Procdump to attach to Store.exe process, monitor for Exceptions or Termination of process, if exception or termination is detected, generate a full dump of Store.exe, to save the dumps in the d:\dumps directory with names that begin with “store_crash” and then exit
 
 Please email at mail@messagingserversupport.com for any queries or feedback.
 
Cheers,
Team MSS   


Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>