My recent experience with Microsoft Exchange Writer

Today I would like to discuss my recent experience with Microsoft exchange VSS writer. Recently I encountered issue with VSS backup for exchange 2010 server and during troubleshooting I learnt something new.

Here are my environment details:
1 Domain controller DC/GC/DNS
1 Exchange 2010 SP1 MBX + Hub transport server
1 Exchange 2010 SP1 CAS server
We only have the Microsoft Software Shadow Copy Provider.

Issue description – Net backup fails to take exchange 2010 mailbox database backup and Microsoft exchange writer goes into either waiting for completion or failed state.

When I run Vssadmin list writers I get below information.
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {5d2bbc2e-43e7-4328-8836-7a82cfb5e0a9}
State: [8] Failed
Last error: Retryable error

Now to get the Microsoft exchange writer into stable state we have to either restart the following services or reboot the server.
Microsoft Exchange Information Store service
Microsoft Exchange Replication service
Microsoft Software Shadow Copy Provider service
Volume Shadow Copy service
(In most of the scenario when we try to restart Microsoft exchange information store service, it goes into hung state so it’s always suggested to reboot the server) 

I found another way to get the Microsoft exchange writer into stable state without rebooting server or restarting any services. Here are the steps

1- Download and install VSS SDK 7.2 from following link
http://www.microsoft.com/downloads/details.aspx?FamilyID=0b4f56e4-0ccc-4626-826a-ed2c4c95c871&displaylang=en
2- Now create a test mailbox database and mount it. Create a test mailbox in the test database and generate few transaction log files by sending few mails to test mailbox.
3- Create components.txt file at following location
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\betest\obj\amd64

The Component.txt will look like this:
"{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}":"Microsoft Exchange Server\Microsoft Information Store\Server_Name\ 060fbad2-045a-425d-89b1-b56f20d93d88";
76fe1ac4-15f7-4bcd-987e-8e1acb462fb7 – This is Microsoft exchange writer ID.
060fbad2-045a-425d-89b1-b56f20d93d88 - This should be the database guid for which you want to take backup. In our case it would be test database guid

4- Now open command prompt and navigate to following location.
C:\Program Files (x86)\Microsoft\VSSSDK72\TestApps\betest\obj\amd64
5- Now run the following command to take test database backup
Betest.exe /B /E /T 1 /S output.xml /C components.txt /D "C:\Backup" > output.txt

Here are switches information:
/B                                Perform a backup
/E                                Exclude the BETEST test writer
/T                                 Backup type (1=FULL, 2=LOG, 3=COPY, 4=INCREMENTAL,5=DIFFERENTIAL)
/S         <filename>       Name of the backup document to save (this is used for restoring)
/C         <filename>       Name of the file containing list of components to backup
/D          <path>            Saves backup files to this location (Specify the location where you have enough         free space on disk)

6-    Once the command is completed, open up output.txt file and scroll down to the bottom. Let’s look for section “status After Backup Complete”  and check the status for Microsoft Exchange Writer. It should be stable. 
Example:
Status after Backup Complete (9 writers)

Status for writer System Writer: STABLE(0×00000000)
Status for writer SqlServerWriter: STABLE(0×00000000)
Status for writer MSDEWriter: STABLE(0×00000000)
Status for writer COM+ REGDB Writer: STABLE(0×00000000)
Status for writer Microsoft Exchange Writer: STABLE(0×00000000)
Status for writer Event Log Writer: STABLE(0×00000000)
Status for writer Registry Writer: STABLE(0×00000000)
Status for writer IIS Metabase Writer: STABLE(0×00000000)
Status for writer WMI Writer: STABLE(0×00000000)
Backup document saved successfully in output.xml

Similarly we can take backup for any other mailbox database provided backupinprogress is not set to true on that database otherwise you will get the below error in the application log.

Log Name:      Application
Source:        MSExchangeIS
Date:          3/24/2012 12:19:45 PM
Event ID:      9840
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCH2010.XYZ.Local
Description:
An attempt to prepare the database 'Mailbox Database' for backup failed because the database is already in the process of being backed up. The error code is -2403. (Note that if a backup was recently aborted, then it may take several minutes for the system to detect the aborted backup and initiate backup cleanup procedures, so this message may be generated if an attempt was made to backup a database before a previous backup attempt had fully terminated.)

For more information on VSS backup please read following articles.

Troubleshooting Exchange 2007 VSS Backups
http://msexchangeteam.com/archive/2008/08/25/449684.aspx
Backup Sequence Diagrams
http://msdn.microsoft.com/en-us/library/aa579076.aspx

If you have any queries or feedback please post your comments in the comment section or write an email at mail@messagingserversupport.com.

Cheers,
Team MSS   

Exchange Server 2010