CDH Upgrade – Part 1, Upgrading Cloudera manager

What I actually wanted to show in this post was a major version upgrade (from CDH4 to CDH5, for example). Unfortunately, CDH4 is no longer available for download (at least not directly from Cloudera). So instead I will do a minor upgrade, from my CDH 5.6.0 that is running on my lab cluster to the newly released 5.7.0

The upgrade is done in two steps, first we have to upgrade Cloudera manager itself along with its agents and then the whole CDH platform.

Lets start.

First, let’s take a look at the current version. In Cloudera manager go to support menu and click “about”. This will bring up the about screen that also shows the current version:


For this upgrade I followed the instructions from here:

First, we should stop some services like Hive and Oozie and remove Kafka if we are using it.

In my cluster I do not have Kafka and since this is for educational purpose only, I will stop the entire cluster.

Now, on the node which runs cloudera manager server, stop the server, agent and database:

[[email protected] ~]# service cloudera-scm-agent stop

Stopping cloudera-scm-agent:                               [  OK  ]

[[email protected] ~]# service cloudera-scm-server stop

Stopping cloudera-scm-server:                              [  OK  ]

[[email protected] ~]# service cloudera-scm-server-db stop

waiting for server to shut down............................................................... failed

pg_ctl: server does not shut down

Hmmm… something is wrong, embedded postgresql database will not stop.

We will have to force stop it. First of all, locate user cloudera-scm in /etc/passwd since this is the user who runs postgresql.

cloudera-scm:x:496:493:Cloudera Manager:/var/lib/cloudera-scm-server: /sbin/nologin

Change the last section to /bin/bash or any other shell of your choice to enable this user to login.

First we find the postgresql data directory:

cat /etc/init.d/cloudera-scm-server-db | grep DATA_DIR=


Then change to cloudera-scm user and do this:

pg_ctl -D /var/lib/cloudera-scm-server-db/data -m immediate stop

waiting for server to shut down.... done

server stopped

The database stopped. Probably there were some clients connected. To make sure the db is ok, we will start it and shut it down again using the service (exit back to root user):

[[email protected] ~]# service cloudera-scm-server-db start

DB initialization done.

waiting for server to start.... done

server started

[[email protected] ~]# service cloudera-scm-server-db stop

waiting for server to shut down.... done

server stopped

it worked. Lets get back to our upgrade.

Note: This procedure only works if you’re using the embedded postgresql database. If you are using external database you should manually stop it now and start it later.

Make sure you have the cloudera yum repository setup in /etc/yum.repos.d:

[cloudera-manager] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64 name=Cloudera Manager baseurl= gpgkey = gpgcheck = 1"

Now upgrade the cloudera manager server:

Yum clean all

yum update cloudera-manager-server cloudera-manager-daemons cloudera-manager-server-db-2 cloudera-manager-agent

Let yum download and install everything.

View full size image

Now start the database and the cloudera manager server:

[[email protected] yum.repos.d]# service cloudera-scm-server-db start

DB initialization done.

waiting for server to start.... done

server started

[[email protected] yum.repos.d]# service cloudera-scm-server start

Starting cloudera-scm-server:                              [  OK  ]

The Cloudera manager service was upgraded succesfuly. Now we will upgrade the cloudera manager agents on all nodes.

Login to cloudera manager and you will immediately see the cloudera manager upgrade wizard (it may take a while until the scm-server is up and receiving connections, since it has to process some extra data from the upgrade):

View full size image

Just follow the wizard which is basically the same one you used when you first installed CDH (if you used cloudera manager to install CHD).

View full size image

View full size image

At the end of the upgrade the nodes are inspected and the inspection raised a warning that not all component versions are identical across all hosts:

View full size image

Underneath was a very long and complicated list of components. So I decided to ignore it for now, because my cluster was functioning just fine until now.

Then it says that some configuration parameters need to be changed, that’s OK, just let it change them:

View full size image

And finally, you will discover that client component is stale for some components. Just click on the icon and then on “deploy client configuration”, let it change the configuration and restart.



View full size image

Now we can check gain and see that the Cloudera manager version has changed.


In this post we finished upgrading Cloudera manager along with its agents. In the next post we will go on and upgrade CDH itself.

CDH Upgrade – Part 2, upgrading CDH itself


This entry was posted in Cloudera, Hadoop and tagged , , . Bookmark the permalink.

Leave a Reply