In this post I will describe the process of enabling Level 1 TLS encryption on all Cloudera manager communications. This is the basic level that encrypts agent-server communication along with server-browser communication. There are higher levels of encryption but we will not discuss them this time. I used self signed certificate which is not recommended for production clusters but will do for the demonstration.
My motivation to enable TLS encryption came from another post, replicating data across clusters using Cloudera manager. Such replication is possible without TLS encryption, but it keeps showing an annoying warning message if it’s disabled. After I started dealing with TLS encryption I thought it deserves a post of its own (actually several posts if I want to cover also level 2 and 3).
Creating the self signed certificate
First of all, set JAVA_HOME. The easiest way is to use the java which is shipped with Cloudera, but any Oracle Java home will do (do not use Open JDK as java home).
export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera export PATH=$JAVA_HOME/bin:$PATH
Now create directories for the keystore:
mkdir -p /opt/cloudera/security/x509/ /opt/cloudera/security/jks cd /opt/cloudera/security/jks
Now run the following command to create a keystore. Make sure to change CN and alias to be your full server name:
keytool -genkeypair -keystore cms.keystore -keyalg RSA -alias cloudera1.lan -dname \ "CN=cloudera1.lan" -storepass cloudera -keypass cloudera -validity 365
Java comes with a default keystore called cacerts. We do not want to change it so we copy it to an alternate keystore named jssecacerts where we will add our certificate. Java looks for certificates in jssecacerts first and if it is not present it goes on and checks cacerts. So copy cacerts to jssecacerts:
cp $JAVA_HOME/jre/lib/security/cacerts $JAVA_HOME/jre/lib/security/jssecacerts
Export the certificate from the keystore to a file:
keytool -export -alias cloudera1.lan -keystore cms.keystore -rfc -file /opt/cloudera/security/selfsigned.cer -storepass cloudera
Copy the self signed certificate to x509 directory:
cp /opt/cloudera/security/selfsigned.cer /opt/cloudera/security/x509/cmhost.pem
Importing the certificate into the truststore
First, copy /opt/cloudera/security/selfsigned.cer to /tmp on every node of the cluster (including the one where scm server runs).
Then, for every node export the JAVA_HOME as shown at the beginning of this post and then Use the following statement to import our self signed certificate into the default truststore:
keytool -import -alias cloudera1.lan -file /tmp/selfsigned.cer -keystore $JAVA_HOME/jre/lib/security/jssecacerts -storepass changeit
You will be asked if you trust this certificate, please enter “yes”:
Owner: CN=cloudera1.lan Issuer: CN=cloudera1.lan Serial number: 301a7193 Valid from: Tue Dec 20 10:32:06 IST 2016 until: Mon Mar 20 10:32:06 IST 2017 Certificate fingerprints: MD5: 7E:2D:E2:FC:C3:BE:45:E6:19:92:5B:A2:00:EB:85:F3 SHA1: 3F:18:2F:A2:E2:7B:30:C1:E1:2E:83:CE:7A:2F:26:83:D8:96:E7:60 SHA256: 3C:80:98:E5:85:35:C4:08:9D:6F:00:9C:27:44:93:F6:0B:E1:41:44:5B:13:60:FA:AF:5B:EC:65:9F:84:A5:48 Signature algorithm name: SHA256withRSA Version: 3 Extensions: #1: ObjectId: 184.108.40.206 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 04 CD 6B E0 EC 11 00 D3 48 7C 51 7C 2B A2 F2 4C ..k.....H.Q.+..L 0010: 53 01 3E C5 S.>. ] ] Trust this certificate? [no]: yes Certificate was added to keystore
Make sure to import the certificate on every node.
After that you can delete the certificate file from /tmp.
Configuring Cloudera manager to use TLS
Open Cloudera manager and select Administration -> settings -> security category.
Change the settings as shown in the screenshot. The password is “cloudera” and make sure you check the “Use TLS for admin console”:
Save the changes.
Click on “Cloudera management service” and choose the configuration tab.
In “scope” choose clousera management service (service wide) and in category choose security. Change the values as in the screenshot. The password here is “changeit”:
Save your changes.
Now restart cloudera scm server:
service cloudera-scm-server restart
After the server is completely up, when you try to go to the regular admin URL at port 7180, you will be redirected to https on port 7183. You will also see a warning saying that the certificate for this site cannot be verified (that’s because we used self signed certificate). Follow the instructions here to get rid of this security message.
Once again go to Cloudera manager and select Administration -> settings -> security category. Check the option “Use TLS encryption for agents”:
Then restart scm server once again.
Now for every node in the cluster, we have to enable TLS usage. Login to every node and edit /etc/cloudera-scm-agent/config.ini
Change use_tls to 1 (the default is 0) and then restart the agent.
service cloudera-scm-agent restart
After all that is complete, login to Cloudera manager and go to Cloudera managment service. You will see a stale configuration icon:
Click it and let it restart the service.
After the restart, if everything is Ok you should have a functional TLS encrypted cluster.
Three important things to note:
- This procedure was tested on CDH 5.9 clster on Centos 6.8 OS. It should work also for a little older versions but the parameter names may be slightly different.
- While creating, exporting and importing the certificates, make sure CN and alias are consistent across all actions.
- On production cluster you should not use self signed certificates and also choose stronger passwords.
That’s all for this time.