Protecting internal applications with a SAML-aware reverse-proxy (a tutorial)

Problem

My employer wholly embraces the coffee-shop model for employee access, which can induce a bit of stress if your job is to protect company resources.  Historically, we have had to support some applications that:

  1. Don’t support SAML (or whatever flavor of federation you prefer)
  2. Probably wouldn’t be exposed outside of the firewall/VPN at most companies because they were never designed to be Internet-facing

We are an enterprise, but only had a small handful of these ‘naughty’ systems. It wasn’t super cost-effective to jump into a 1500+ employee seat contract with Duo (now Cisco), Cloudflare Access, or ScaleFT Zero Trust Web Access1 just to solve this particular problem across a small number of hosts. Yet, employees were frustrated that most day-to-day operations did not require jumping on a corporate VPN until you had to reach one of these magical systems.

Solution

I designed a SAML-aware reverse-proxy using a combination of Apache 2.4, mod_auth_mellon, and a sprinkling of ModSecurity to add some rate limiting capabilities.  The following examples assume Ubuntu 16.04, but you can use whatever OS you’d like, assuming you know how to get the requisite packages.

Install dependencies and enable Apache modules

sudo apt-get install apache2, libapache2-mod-auth-mellon, libapache2-modsecurity
sudo a2enmod proxy_http proxy ssl rewrite auth_mellon security2

Configure ModSecurity

Our ModSecurity install will do one thing and one thing only: rate limit (by IP) access attempts by non-authenticated users.

Create or overwrite /etc/modsecurity/modsecurity.conf and put the following content:

# A minimal ModSecurity configuration for rate limiting
# on a large number of HTTP 401 Unauthorized responses.
SecRuleEngine On
SecRequestBodyAccess On
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyInMemoryLimit 131072
SecRequestBodyLimitAction ProcessPartial
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /tmp/
SecDataDir /tmp/
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /var/log/apache2/modsec_audit.log
SecArgumentSeparator &
SecCookieFormat 0
SecUnicodeMapFile unicode.mapping 20127
SecStatusEngine On

# ====================================
# Rate limiting rules below
# ====================================

# RULE: Rate-Limit on HTTP 401 response codes
# Set IP address value to a variable
SecAction "phase:1,initcol:ip=%{REMOTE_ADDR},id:'1006'"
# On HTTP status 401, increment a counter (block_script), and expire that value out of cache after 300s
SecRule RESPONSE_STATUS "@streq 401" "phase:3,pass,setvar:ip.block_script=+1,expirevar:ip.block_script=300,id:'1007'"
# On counter variable (block_script) being greater than or equal to '20', deny with HTTP 429 Too Many Requests
SecRule ip:block_script "@ge 20" "phase:3,deny,severity:ERROR,status:429,id:'1008'"

Feel free to add your own ModSecurity rules if you’d like to do things like detecting/blocking remote shell attempts, SQL injection, etc, but that’s not something I intend to cover here.

Modify the site (vhost) configuration

In case it’s non-obvious, in the following commands feel free to change out ‘myservicename’ with an appropriate identifier for service you are protecting with this gateway setup.

Head over to /etc/apache2/sites-enabled and open the vhost config file you intend to add protection to (or modify the default one, if this is a new install).

<IfModule mod_ssl.c>
 <VirtualHost _default_:443>
  ServerAdmin webmaster@localhost
  [...]
  # MSIE 7 and newer should be able to use keepalive
  BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

  ProxyRequests Off
  ProxyPass /secret/ !

  # If fronting a locally-installed app, just forward to
  # the correct listening port. Alternatively,
  # you can address a system on another domain and port.
  ProxyPass / https://127.0.0.1:8000/ retry=10
  ProxyPassReverse / https://127.0.0.1:8000/

  ErrorDocument 401 "\
<html>\
<title>Access Restricted</title>\
<body>\
<h1>Access is restricted to organizational users.</h1>\
<p>\
<a href=\"/secret/endpoint/login?ReturnTo=/\"><strong>Click here to login via single sign-on, or wait for 2 seconds to be redirected automatically.<strong></a><br /><br /><br /><br /><a href=\"/#noredirect\">Temporarily disable redirection.</a>if(window.location.hash == \"\") { window.setTimeout(function(){ window.location.href = \"/secret/endpoint/login?ReturnTo=\" + encodeURIComponent(window.location.pathname + window.location.search); }, 2000); }\
</p>\
</body>\
</html>"

  <Location />
   # Documentation on what these flags do can be found in the docs:
   # https://github.com/Uninett/mod_auth_mellon/blob/master/README.md
   MellonEnable "info"
   AuthType "Mellon"
   MellonVariable "cookie"
   MellonSamlResponseDump On
   MellonSPPrivateKeyFile /etc/apache2/mellon/urn_myservicename.key
   MellonSPCertFile /etc/apache2/mellon/urn_myservicename.cert
   MellonSPMetadataFile /etc/apache2/mellon/urn_myservicename.xml
   MellonIdpMetadataFile /etc/apache2/mellon/idp.xml
   MellonEndpointPath /secret/endpoint
   MellonSecureCookie on
   # session cookie duration; 43200(secs) = 12 hours
   MellonSessionLength 43200
   MellonUser "NAME_ID"
   MellonDefaultLoginPath /
   MellonSamlResponseDump On

   # This 'requirement' is actually going to be
   # optional. We also give some trusted IPs below,
   # and tell Apache we can fulfill either requirement.
   Require valid-user
   Order allow,deny

   # This is where you can whitelist IPs or
   # even entire network ranges, perfect for
   # systems that still need to accept
   # some API traffic from known networks.
   Allow from 10.20.30.0/24
   Allow from 10.10.110.66

   # Allow one of the above to be good enough.
   # You could change this to 'all' if you need
   # to satisfy SSO required AND valid network
   # required.
   Satisfy any
  </Location>

  <Location /secret/endpoint/>
   AuthType "Mellon"
   MellonEnable "off"
   Order Deny,Allow
   Allow from all
   Satisfy Any
  </Location>

 </VirtualHost>
</IfModule>

Create SAML SP metadata files

We’ll download and use a shell script from the mod_auth_mellon authors to create the necessary SP metadata files:

sudo mkdir -p /etc/apache2/mellon/
cd /etc/apache2/mellon/
wget https://raw.githubusercontent.com/Uninett/mod_auth_mellon/master/mellon_create_metadata.sh
bash mellon_create_metadata.sh urn:myservicename https://<YOURDOMAIN>/secret/endpoint

Now your directory structure should resemble the following:

msulliv@c0995a102f21:/etc/apache2/mellon/# ls
mellon_create_metadata.sh urn_myservicename.cert urn_myservicename.key urn_myservicename.xml

mellon_create_metadata.sh is no longer needed and can be deleted, if you so choose.

Create the SAML 2.0 application profile on your IdP

Go to your identity provider and provision the new application. For this example, I’m using Okta (who I highly recommend):

screencapture-workiva-admin-oktapreview-admin-apps-saml-wizard-edit-webfilings_samlgateway_1-2018-08-07-14_21_25.png

Place SAML IdP metadata

Finally, grab the IdP metadata and put it on your clipboard:

Screen Shot 2018-08-07 at 2.24.29 PM.png

Drop its contents into a new file at /etc/apache2/mellon/idp.xml:

msulliv@c0995a102f21:/etc/apache2/mellon# cat idp.xml
<?xml version="1.0" encoding="UTF-8"?>
<md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="http://www.okta.com/exkd2n9ujpQFaUq8f0h7">
<md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
<md:KeyDescriptor use="signing">
<ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:X509Data>
<ds:X509Certificate>MIIDBzCCAe+gAwIBAgIJAJAD/4DMpp7vMA0GCSqGSIb3DQEB
[...]

Restart Apache and Test

sudo systemctl reload apache2

Now head to your application and check out the results:

Screen Shot 2018-08-07 at 2.49.31 PM

Redirected to an auth challenge – perfect!

Extending it further

Quickly adding SAML support to PHP/Python/Rails/Node/etc apps on the same host

In your organization’s homegrown applications where an existing Apache 2 server is acting as a front-end, this same principle can be used to quickly add SAML support. In your vhost config in the Mellon options, add:

<Location />
 [...]
 RequestHeader set Mellon-NameID %{MELLON_NAME_ID}e

In your application, simply check for a value in this header and use it if present. For instance, in Python’s Flask framework:

@login_manager.request_loader
def load_user_from_request(request):

    nameid = request.headers.get('Mellon-NameID')
    if nameid:
        user = User.query.filter_by(username=nameid).first()
        if user:
            return user
        else:
            # Provision user's account for first use 
            user = User(nameid)
            return user

    # return None if method did not login the user
    return None

Back-end on another host

Some applications, like Splunk, can receive login user information via request header (note: Splunk now supports SAML natively, but it still makes for a good example app).  We can direct mod_auth_mellon to send this header along with the information about an authenticated user. Mellon populates the field ‘MELLON_NAME_ID’ with the IdP username ([email protected]) after successful authentication.

In your vhost config in the Mellon options, add:

<Location />
 [...]
 # Pass Splunk a request header declaring the user who has logged in
 # via SAML. The regex test at the end of this line ensures that
 # MELLON_NAME_ID is not an empty string before attempting to set
 # the SplunkWebUser header to the value of MELLON_NAME_ID.
 # Splunk unfortunately freaks out if the SplunkWebUser header is
 # declared but it has no value.
 RequestHeader set SplunkWebUser %{MELLON_NAME_ID}e "expr=-n %{env:MELLON_NAME_ID}"

Be careful to make sure your back-end application is only accessible via this reverse-proxy though, otherwise someone with local network access could simply send the back-end server requests directly with this header to bypass authentication entirely2. In Splunk’s case, that’s what the values under ‘trustedIP’ in $SPLUNK_HOME/etc/system/local/web.conf are for.

Footnotes

1. ScaleFT’s overall offering appears to be very enticing, and I see their recent acquisition by Okta as a great development. Because it addresses several other pain points, we are actively working to deploy ScaleFT at my organization, which will likely replace the home-grown solution described in this post.

2. Do your part to prevent data breaches by seeking assistance from someone with relevant security experience if you are unsure whether or not your back-end application on another host is properly protected from such an attack.

13 thoughts on “Protecting internal applications with a SAML-aware reverse-proxy (a tutorial)

  1. Great Post, how can you add encryption to the headers set by Mellon to avoid passing cleartext values to the appliction tier. Does Mellon provide a library for encrypting such headers?

    Like

  2. nearly got this implemented thanks, having some trouble, however. I’m getting a bad request after authentication and return to …/secret/endpoint/postResponse. my setup is redhat so have not been able to copy exactly what you’ve done (httpd instead of apache2 and /etc/ is different), the proxyreverse is working because if I use “Satisfy any” like you have in the example virtualhost conf it works, however I really need “Satisfy all” and so, am unfortunately, stuck..

    Like

  3. hi matt, thanks for the reply, debug log is on last line is “[Thu Aug 27 19:21:12.948184 2020] [auth_mellon:warn] [pid 36833] [client 10.187.94.236:53120] User has disabled cookies, or has lost the cookie before returning from the SAML2 login server.”. there’s very little other examples I can find. there’s a hint here: https://jdennis.fedorapeople.org/doc/mellon-user-guide/mellon_user_guide.html#mellon_diagnostics that there’s a disagreement in security check and another where the cookie must be set from the sameorigin

    Like

  4. I’m struggling with the whitelist portion of this configuration. This is a static website being hosted by Apache using mod_auth_mellon to perform a SAML redirect to an Okta IDP.

    When I remove the following lines from the Location configuration, I successfully get redirected to Okta for auth:

    Order allow,deny
    Allow from
    Satisfy any

    When I add those lines back in, if I browse from a host in , I am able to browse the site without authentication as I would hope. However, if I browse from an external host, I do NOT get a SAML redirect and instead just receive a 401.

    Any ideas what I’m doing wrong?

    Like

  5. I’ve got SAML working well with ADFS as the IdP and Apache as the SP, however, I’m trying to extend it to multiple Apache servers in a cluster serving the same sites. However, when I do this, one web server doesn’t seem to be aware of the session created in the other web server, so I get redirected back through ADFS multiple times before it finally just fails.

    Do you have any advice on how to cluster the web servers?
    Things I’ve done while following other guides:
    * Both Apache servers are using the same certificate for Mellon
    * Both Apache servers are using the same entity ID

    I’m getting an “Invalid Audience in Conditions” in the Apache error log.

    Like

  6. Great post. I was curious what what would need to change to support multiple hosts in the reverse proxy?

    For example if I wanted my SP to live at https://auth.domain.com would it be possible to setup multiple reverse proxies on the same host to other backends?

    From my understanding, how this is setup, it will only work for one reverse proxy host/backend.

    Also I noticed the mellon GitHub is no longer supported, but there is a fork that is.

    Like

  7. This proved to be very helpful for me, but I’d like to point out two gotchas that tripped me up:

    – ‘urn_myservicenname’ is consistently spelled with two ‘n’s throughout aside from one reference in the text. I mistakenly removed an ‘n’ when setting up my Okta application and that was problematic until I tracked down the problem. (I’ll use another identifier as I roll this out to prod servers, but I was cutting-and-pasting while testing it out)

    – MellonVariable is defined twice, for me the initial value of “cookie” (which I think is the default) was the correct one.

    Aside from that, it worked like a charm. Thanks very much for taking the time to document this.

    Like

Leave a reply to Matthew Sullivan Cancel reply