This blog is an addendum to my earlier post on enabling SSO for Splunkweb administration console. That post deals with only one splunkweb instance. What if customer wanted to enable SSO for farm of search heads running behind a load balancer. This deployment is popular with the new search head clustering feature introduced in Splunk 6.2 version.

In order to enable the splunk web SSO for the search head clustering deployment, you need to have access to a load balancer (hardware or software with the ability to route the requests based on Cookie or IP persistence ) that plays the vital role to achieve the SSO seamlessly among the backend search heads. This procedure has been tested with both hardware and software load balancers.

  • Big IP with IP persistence – Hardware Load Balancer
  • Apache httpd reverse proxy with sticky cookies. – Software Load Balancer

The configuration is simple

  • Enable the SSO using the usual procedure as described here
  • Create the LDAP authorization strategy in all the participating search heads in the search head clustering deployment. (You can simply copy the authentication.conf to all the search heads, make sure the bind user password is in plain text before copying, once you restart the splunkd these plain text passwords will be obfuscated )
  • Set the load balancer configurations appropriately, especially the stickiness for the requests. If the requests ends up in the wrong server then you would see unexpected results.

Setting up Big IP LB for SHC

It is relatively easy to setup the sticky load balancing for the HTTP traffic. You can find documents related to this here

How to setup apache reverse proxy for SHC load balancing

In this section I am going to describe the procedure about the configuration of apache reverse proxy for load balancing along with cookie based persistence.

Pre-Requisites

  • Apache httpd server

    Server version: Apache/2.4.10 (Unix)Server built: Jan 1 2015 13:25:30Server’s Module Magic Number: 20120211:36Server loaded: APR 1.4.6, APR-UTIL 1.4.1Compiled using: APR 1.4.6, APR-UTIL 1.4.1

  • Required apache modules
    • headers_module
    • proxy_module
    • proxy_balancer_module
    • rewrite_module
    • authnz_ldap_module
  • An LDAP server for authentication

What needs to be configured at Search Heads?

In the search heads that participate in the search head clustering, please enable the usual SSO configuration properties.

web.conf

[settings]
 root_endpoint=/shc
 trustedIP = 127.0.0.1,10.140.104.34
 remoteUser = X_Remote_User
 SSOMode = permissive
 embed_uri = http://proxy.sv.splunk.com:8080/gpass
 allowSsoWithoutChangingServerConf = 1
 remoteUserMatchExact = 1

The above configuration has to be included in all the participating search heads in the search head clustering.

There are couple of new properties added in the Splunk enterprise 6.2

  • remoteUserMatchExact – When matching the remoteUser header, consider dashes and underscores distinct (so “Remote-User” and “Remote_User” will be considered different
    headers
  • allowSsoWithoutChangingServerConf – Usually when configuring SSO, a trustedIP needs to be set both here in web.conf and also in server.conf. If this is set to “1” then we will enable web-based SSO without a trustedIP in server.conf
  • authentication.conf –  Create the LDAP authorization strategy in all the participating search heads in the search head clustering deployment. (You can simply copy the authentication.conf to all the search heads, make sure the bind user password is in plain text before copying, once you restart the splunkd these plain text passwords will be obfuscated ). The LDAP server that is used in this configuration file must be the same as in the httpd.conf used for reverse proxy authentication.

Restart the whole cluster by issuing splunk rolling-restart shcluster-members

That is it, you are ready to setup the load balancing using the apache http server.

Configuring the Apache Web server as Load balancer

Once you configure the apache http server with required modules, you can go ahead configure the web server to act like a load balancer. Here are the relevant portion that needs to be configured for apache http server.

httpd.conf changes

Add the following portion of configuration code that goes in the httpd.conf by-passes the authentication so the embedded reports can be accessed anonymously.

gpass

 

The above configuration snippet enables the embedded reports so when embedded reports are created, those reports will not go through the authentication process, these will be authenticated anonymously.

The next set of configuration stanza enables the following

  • Introduces the LDAP authentication for the URL context /shc
  • Set the X_REMOTE_USER with the user name of the authenticated identity
  • Inserts the ROUTEID cookie that will be used for the sticky load balancing, it is critical to make the LB work correctly.

location

Include the following line as part of httpd.conf

Include conf/extra/httpd-LB.conf

This file has the definition of the backend search heads that participate in the search head clustering. The content of the conf/extra/httpd-LB.conf file looks like this:

lb

Once you restart the proxy with these settings, it will be ready for testing SHC via the load balancer.

How to test

Invoke the proxy url http://proxy.sv.splunk.com:8080/shc, it will redirect to one of the back end server. You can inspect the requests sticking to a particular back end SH, based on the value for ROUTEID. As you can see the screen shot below taken from one of the session via the reverse proxy. The cookie value is computed as SHC.ROUTE as defined in the httpd.conf. If you have noticed the user name in the splunk web nav bar is missing, then all the likely it is due to non sticky load balancing.

routeid

Embedded Reports Can be created from teh splunkweb, when you create it will automatically populate the anonymous url with context /gpass as you can see below

embed

Advertisements