Use CrashDiag Site Extension to Capture Dump for Intermittent Exception issues on Microsoft Azure Web App

While running Customer’s .NET/PHP/ Node processes in Azure Web App environment, it may intermittently crashes due to code or performance issues. It’s important to capture the crash dump when such crash/exception happen automatically for further investigation.

This article is to introduce the new CrashDiag Site Extension, which can easily help us to capture the necessary data when intermittent unhandled exception happens. Now it can work for these scenarios:

1.     Unhandled Exception happening

2.     Specific first chance exception happening based on filtering parameter

3.     Monitoring new launched processes continuously

4.     Configure total dump file numbers and debugger process instances

5.     Native App, Managed App, Web Job with 32bit/64bit

6.     Safe Stop at any time without terminating target process

To use it, follow below steps:

1. Install

2. Configure (1^st chance or 2^nd chance exception handling):

3. Start

4. Stop

Install

1. Open Azure Portal, navigate to App Services -> your Web App -> Tools -> Extensions.

2. Click Add, and choose Crash Diagnoser to install it.

Configure

1. After installation, select it from the Installed Web App Extension blade, click Browse:

2. Now we can configure it based on our troubleshooting scenarios (1^st chance or 2^nd chance exception handling):

2.1 Application Crashed (unexpected terminated due to 2nd chance unhandled exception)

2.1.1 Click the 2^nd Chance Unhandled Exception tab, choose your target process name. The default options include w3wp, node, php-cgi. In this sample, it is w3wp

If you want to monitor other process, such as mywebjob.exe, just type the process name mywebjob manually in the field.

2.2 Application didn’t crash, but has specific exceptions generated. We need to filter 1st chance exception and perform analysis.

2.2.1 Click the 1^st Chance Exception tab, set the exception code and process name.

Besides the default options of exception code, you can type other specific code which occurred on the application. For example, if the application reported FileNotFound exception, you can type FileNotFound or NotFound in the Exception Code field. In this sample, it is 0xC0000005

To monitor all 1^st chance exceptions, type *

But this may generate other kind of dump files other than you expected.

3. Click the Advanced Settings tab to make sure the settings are properly for you. The current default setting are suitable for common scenarios, you can tune them as you wish.

Consider one dump can be hundreds MB size, you may want to set the Maximum dump file number less than 10.

Start

1. After configuration, click the Start button to start monitoring. If your application is running, around 10 seconds, you will see the CrashDiag status changed from “Stopped on Instance name” to “Monitoring n Processes on Instance name”. In this sample, it is “Monitoring 1 Process on RD000D3A8013E6”

2. If the application crashed, you will see the *.dmp files are generated in the output path. In this sample, it is in d:\home\data. Then You can open Kudu portal and further investigate the dump file.

Stop

1. After troubleshooting, click SafeStop to stop the Crash Diagnoser. This Stop action will not terminate the target process and can be executed at any time.

If the Instance is running a Crash Diag, and you click Start again, the portal will reminder you to safe stop it first:

Question & Answer

1. After install the Site Extension, will the Web App be restarted?

Yes. It will be restarted as applicationhost.xdt is used in the package. Without restarting, the route to CrashDiag cannot be found.

If your web app is required to be running on this instance during business hours, please do this Installation on maintainer time. It will take less than 1 minute.

2. Can it analyze dump automatically?

Not Yet. Its current purpose is to capture intermittent exception data as dump files for root cause analysis.

3. If I set target process as W3WP, will it monitor the Kudu site W3WP process as well?

No. It will not capture processes which is running for Kudu, DaaSConsole, and DaaSRunner

4. If the site extension is updated, will it be auto-updated?

Yes. It will be auto-updated. A popup message will remind you for this:

Your web app will not be restarted during the auto-update.

5. If I don’t know which kind of exception I’m facing, how to configure it?

Please check if there is Process Terminated or Exit logs in the D:\home\LogFiles\eventlog.xml when the issue happened. If yes, then can use 2^nd chance exception configure as above.

Normally 1^st chance exception will not cause application crash. If you need to capture dump for it, please understand the performance impact on the target process can be a little higher than the 2^nd chance exception capture. This is because the debugger needs to filer all 1^st exceptions happen in the target process.

6. Will it monitor all Web Instances?

Through this UI, it will only monitor the current Instance in the current Kudu site (https://appname.scm.azurewebsites.net/crashdiag) instead of all instances.

The SafeStop option will stop all monitoring tasks on all instances.

Thanks!

-Freist Li from APGC DSI Team