Rebuilding a Cisco Unified Communications Manager Publisher from the Subscriber Database

Today we are going to go over how to rebuild a CUCM Publisher because of failure and restore the data from the Subscriber server. There are a number of reasons why you would need to do this. I have had to perform this a number of times so it is not really out of the ordinary. For example you may have an issue with disk degradation on the UCS. Or maybe there has been some other corruption that has taken place and the server won't come up or the services won't start. 

List of Step to Complete This Rebuild

1. Gather Cluster Information
2. Stop Replication on the Subscriber
3. Install New CUCM Publisher
4. Update the Process Node on the Publisher
5. Prepare the Server for the Restore from the Subscriber
6. Reboot the Publisher Server
7. Validate That the Two Servers are now Authenticated
8. Add a Backup Device and Run a Manual Backup
9. Restore the Backup Using the Subscriber Database
10. Reboot Both Servers and Validate Database Replication

Gather Cluster Information

So the first step is to gather some information, To do this we are going to login to the CLI on the Subscriber server. The Publisher is shutdown at this point. We are going to run two commands

show network cluster

show version active

These commands are going to give us the server hostnames, ip addresses and any software cop files that have been installed such as device packs. Those will need to be installed along with the Publisher as the restore will not have that information.

Stop Replication on the Subscriber

Next we need to stop the database replication on the subscriber. We are going to be installing a new Publisher and we don't want the subscriber to try to pull information from it until the restore is done. To do this we run the following command

utils dbreplication stop

Install New CUCM Publisher

Now it is time to install the new Publisher. To do this we need to have the OVA template and the bootable ISO to install the CUCM server. If you have updated the server since it was first installed and you don't have the bootable ISO you can download the image from Cisco.com and I have a video below on how to make that file a bootable file for the installation.

Install the new server and make sure that you have the same hostname, ip address, DNS, gateway, etc. All must be the same as it was or the data from the Subscriber will not match.

Here is a video about the installation of Cisco Unified Communications Manager 12.5 to help you along on this step

Update the Process Node on the Publisher and Validate It

Once the new Publisher is installed we now need to add the subscriber to it so that it can see it. To do this goto System --> Server and click on Add New and put in the hostname of the subscriber server and a description.

Prepare the Server for the Restore from the Subscriber

Now we need to run a command that will prepare the server for the restore from the subscriber. That command is

utils disaster_recovery prepare restore pub_from_sub

There is no feedback from the cli for this command. It will just accept it and we move on.

To validate that the process node has been updated run this command

run sql select name,description,nodeid from processnode

Reboot the Publisher Server

It is very important at this point to reboot the server. If we don't the process node will not be authenticated and the restore will not be allowed to happen. So run this command in the CLI

utils system restart

Validate That the Two Servers are now Authenticated

To make sure that all is good up to this point we need to make sure that the two servers are authenticated. To do that run the following command

show network cluster

Add a Backup Device and Run a Manual Backup

Now we need to add a backup device and run a manual backup. This is important as the Restore Wizard needs to have a backup to restore from even those we are going to use the database from the subscriber. If you had a backup device prior to this you can put that in. If you didn't (Why didn't you?!) you can use a free SFTP software and set it up for this purpose. 

Goto Backup --> Backup Device and put in the information needed.

Then Goto Backup --> Manual Backup and perform a manual backup.

Here is what the backup will look like when complete.

Restore the Backup Using the Subscriber Database

Now goto Retore --> Restore Wizard and choose the backup device that was just used for the backup. It will find the backup that we just did. Choose the features you are going to restore which will be all of them. The next page is the important one. You need to select the Publisher node under "Select the Servers to be restored for each feature" and leave the subscriber unchecked.

Under the next section "Select the server from which database data need to be restored" hit the dropdown box and select the Subscriber server.

Now the restore will begin. The restore will take quite a bit more time than the backup. When complete it will look like this

Reboot Both Servers and Validate Database Replication

So now it is time to reboot the servers so that the database replication can begin. Remember that we turned off the replication on the subscriber. Rebooting it will allow it to start up again and pull data from the Publisher. So run the following command on both servers.

utils system restart

Once both server are rebooted and completely up (check this by logging into the GUI) you can login to the CLI and run this command to check the database replication

utils dbreplication runtimestate

You are looking for both servers to show (2) Setup Completed. If it doesn't show that or it shows one server Synching then wait a little longer and run the command again.

Once that is done you have now completed the rebuild and restore of the Publisher. Congratulations! 

I hope that this tutorial has been helpful and informational. If you run into any issues please comment below and I can try to help out.