7 Steps to Maintaining Exchange Server Without User Impact
by Will N. chat_bubble_outline 0 comment(s)
Ever hated patching your servers after hours? If you’re like me always looking for ways doing stuff easier and faster, there is a better way.
I’m sharing my notes with you from the field on how to maintain Exchange servers (updates, reboots, etc.) during regular business hours virtually without user impact.
I’m assuming for this article that your environment consists of at least two Exchange 2013/2016/2019 servers both running the CAS and Mailbox role, setup in a Database Availability Group (DAG) and with a pair of load balancers. For the example below my Exchange servers are named Ex01 and Ex02, and we are placing Ex02 into maintenance mode.
A quick overview of the requirements:
- You cannot take all Exchange servers down for maintenance at the same time; no server will be left to serve clients (well, duh).
- Your (remaining) Exchange servers must be capable of handling the day-time load if an Exchange server is unavailable.
- The majority of members in the Database Availability Group (DAG) must be up at any point in time for the DAG to be operational. That means if you have 2 Exchange servers and 1 DAG Witness, you can only take up to one of these down without interrupting DAG operation.
- Moving databases to another server is visible to users. I’m recommending to do that one database at a time. If you move all databases at once, the Disconnect visible to users may extend from the usual less than 30 seconds to several minutes. You may also move the databases the day/night before.
I realize you can automate most of the steps below with PowerShell and Automation tools, but I felt it would be appropriate to provide you with the individual steps allowing you to adapt these to your environment.
Placing an Exchange Server into Maintenance Mode
Step 1 – Confirm Replication / Witness Server Health
Issue the following commands in the Exchange Management Shell, and confirm that all tests have passed.
Test-ReplicationHealth -Server Ex01
Test-ReplicationHealth -Server Ex02
If any errors are displayed, resolve them before proceeding.
Step 2 – Confirm the Database Copy Status
Run these two commands.
Get-MailboxDatabaseCopyStatus -Server Ex01
Get-MailboxDatabaseCopyStatus -Server Ex02
Confirm that the Status of each database is either “Mounted” or “Healthy”. Confirm that the CopyQueueLength and ReplayQueueLength is a small number, like less than 10. Confirm the ContentIndexState is “Healthy” for all databases.
To avoid loss of email, resolve any issues identified above before proceeding.
Step 3 – Move the Databases
As mentioned before, moving databases between Exchange servers is visible to users. I’m recommending to move one database at a time. If you move all databases at once (from Ex02 to Ex01 in our example), due to the workload on Exchange the Disconnect visible to the users may extend from the usual less than 30 seconds to greater than 10 minutes. You may move the databases the day/night before if you desire. Please proceed with caution.
Log in to the Exchange Admin Center (like https://mail.yourcompanyname.com/ecp). Under “servers”, on the “databases” tab, select the first database that is mounted on Ex02. On the right-hand side, click “Activate” to activate it on Ex01.
Allow a minute for the workload to settle. Repeat until all databases are off from Ex02.
Step 4 – Disable the Server on the Load Balancers
Log in to your Load Balancers. Disable Ex02 on the Load Balancers, so no more user traffic is sent to Ex02. Your specific directions depend on your make and model.
Step 5 – Drain the Mail Queues
Issue the following command.
Set-ServerComponentState Ex02 -Component HubTransport -State Draining -Requester Maintenance
Confirm that the mail queues are empty. If they aren’t, allow 10 minutes to clear.
Get-Queue –Server Ex02 | ? {$_.Identity -notmatch “Poison” -AND $_.Identity -notmatch “Shadow”}
Step 6 – Disable Database Activation
Since you’ve already manually moved all databases off from Ex02, the commands below mark the server as disabled for database activation.
Set-MailboxServer Ex02 -DatabaseCopyActivationDisabledAndMoveNow $True
Set-MailboxServer Ex02 -DatabaseCopyAutoActivationPolicy Blocked
Step 7 – Server Wide Disable
The final step is to mark all Exchange services as inactive.
Set-ServerComponentState Ex02 -Component ServerWideOffline -State Inactive -Requester Maintenance
Allow 10 minutes for the command to take effect.
You’re now free to patch and reboot Ex02, or whatever else needs to get done on this server.
When done with the maintenance, it’s important to follow the steps in the next section to take the server out of maintenance and place it back into production.
Taking an Exchange Server Out of Maintenance Mode
These steps are pretty much the reverse of the before. I’m assuming we’re taking Ex02 out of maintenance mode.
Step 1 – Server Wide Enable
Enable the Exchange components with the following.
Set-ServerComponentState Ex02 -Component ServerWideOffline -State Active -Requester Maintenance
Step 2 – Enable Database Activation
Issue these commands.
Set-MailboxServer Ex02 -DatabaseCopyActivationDisabledAndMoveNow $False
Set-MailboxServer Ex02 -DatabaseCopyAutoActivationPolicy Unrestricted
Step 3 – Enable the Mail Queues
Run the following command.
Set-ServerComponentState Ex02 -Component HubTransport -State Active -Requester Maintenance
Step 4 – Enable the Server on the Load Balancers
Log in to your load balancers. Enable Ex02, resuming user traffic to Ex02.
Step 5 – Confirm Replication / Witness Server Health
Issue the following command in the Exchange Management Shell, and confirm that all tests passed.
Test-ReplicationHealth -Server Ex01
Test-ReplicationHealth -Server Ex02
If any errors are displayed, resolve them before proceeding.
Step 6 – Confirm the Database Copy Status
Run these two commands.
Get-MailboxDatabaseCopyStatus -Server Ex01
Get-MailboxDatabaseCopyStatus -Server Ex02
Confirm that the Status of each database is either “Mounted” or “Healthy”. Confirm that the CopyQueueLength and ReplayQueueLength is a small number, like less than 10. Confirm the ContentIndexState is “Healthy” for all databases.
To avoid loss of email, resolve any issues identified above before proceeding.
Step 7 – Move the Databases
As mentioned before, I’m recommending to move one database at a time. If you move all databases to the another server at once, due to the workload on Exchange the disconnect users will see may extend from the usual less than 30 seconds to greater than 10 minutes. Please proceed with caution.
Log in to the Exchange Admin Center. Under “servers”, on the “databases” tab,
- You may either move all the databases to Ex02 (if you’ll be doing maintenance on Ex01)
- Or you may balance the mounted databases between the servers if that’s how it’s intended in your organization (50% on Ex01, and the other 50% on Ex02)
Once again, move the databases one at a time, and give it a minute in between to allow Exchange to settle the workload.
P.S.: I’ve used these steps in production environments patching and rebooting one Exchange server at a time during regular business hours. The total impact to end users included a brief disconnect from Exchange of about 30 seconds for users where I had to move their database.
Liked this article? Thoughts or ideas? Please let me know below.
***
Will Neumann a technical architect specializing on enterprise messaging. He and his team help their clients build and maintain enterprise-level features and services. To learn more, contact Will at +1-780-665-4948 (Mountain Time).
