rbac – Matt's Life Bytes

What is Rancher 2.0?

For those who may not be familiar with it, Rancher 2.0 is an open-source management platform for Kubernetes. Here’s the relevant bits of their elevator pitch:

Rancher centrally manages multiple Kubernetes clusters. Rancher can provision and manage cloud Kubernetes services like GKE, EKS, and AKS or import existing clusters. Rancher implements centralized authentication (GitHub, AD/LDAP, SAML, etc.) across RKE or cloud Kubernetes services.

Rancher offers an intuitive UI, enabling users to run containers without learning all Kubernetes concepts up-front.

It’s important to know that Rancher enhances, but does not modify, the core concepts of Kubernetes. You can think of Rancher as a sort of abstraction layer, through which you can interface with all of your Kubernetes clusters. If you simply removed Rancher, everything in your clusters would continue merrily along as if nothing had changed (though management of those clusters would become a significant burden).

This post is intended to step you through the basics of Rancher 2.0 use, and assumes that you’ve never seen it before, or are just getting started on it. If you like what you see, feel free to install Rancher onto your own development environment and take it for a spin. In my experience, deploying Rancher in a non-production configuration to an existing cluster takes less than three minutes.

Authentication and Authorization

Like all organizations should, my employer uses a directory service to store user information and centralize access control. We tie that provider, Okta, into Rancher, and use it to authenticate. See my previous blog post for details.

After clicking ‘Log In with adfs’, we are greeted by the Rancher UI.

We are currently in Rancher’s ‘global’ context, meaning that while we have authenticated to Rancher, we haven’t selected any given cluster to operate against.

Rancher has a robust role-based access control (RBAC) architecture, which can be used to segment Kubernetes clusters and workloads from each other, and also from human and service accounts. This means that while you might be able to only see one Kubernetes cluster in Rancher, it might be managing another ten away from your view.

At my employer, this means that we can have several clusters, each with very different requirements for security and confidentiality, all under the same Rancher install and configured with the same Identity Provider, Okta.

Interacting With a Cluster

From the Rancher UI’s landing page, we’ll select our ‘inf-dev’ cluster to explore.

This landing page shows us our worker node utilization and the health of critical internal services, like our container scheduler. The most important thing on this screen, however, is the ‘Launch kubectl’ button on the top-right.

Using kubectl across multiple different Kubernetes environments is a nightmare. It usually requires several config files and changes to an environment variable. When you are running on Amazon’s EKS solution like we are, it gets even more complicated because you also need to be running a properly-configured setup to interact with AWS’ IAM layer.

Gross.

With Rancher, things are much easier. Clicking the ‘Launch kubectl’ button drops you into a Ubuntu 18.04 container, fully-configured with a ready-to-use kubectl installation.

If you’re an advanced user / administrator and still need to use kubectl locally, just click the ‘Kubeconfig File’ button to download a ready-to-use configuration file to your system.

Projects and Namespaces

If we click the ‘Projects/Namespaces’ tab, we get a logical breakdown of the services running on our Kubernetes cluster.

Namespaces are a logical separation of resources and virtual networks within a cluster. From the Kubernetes project:

Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called namespaces.

Rancher also provides the concept of projects, which is a way to group namespaces together and apply RBAC policies to them consistently. For now, our organization does not plan to make use of the projects feature, so the default project, named ‘Default’, is simply a 1:1 mapping to the default namespace.

If we click the ‘Project: default’ link, we can jump into our production cluster’s configured services and see what’s running from inside the ‘Workloads’ view.

Managing Services

Let’s select a service from the Workloads page. For this walk-through, I’ll choose the ‘nats-nats’ service.

We are provided the list of pods running our service, as well as other accordion boxes which can help us understand our service’s configuration, such as its environment variables. Rancher supports granting view-only access to groups as needed, so even in a production environment, we can help non-admins understand how a service is configured. Best of all, because Kubernetes uses a special data type for secrets, Rancher will not disclose the values of secrets to users with view-only permissions, even if those secrets are injected into the container through environment variables.

One of my favorite features of Rancher is the ability to tail logs from stdout/stderr in real-time. While you should absolutely not rely on this for day-to-day log analysis, it can be extremely handy for debugging / playing around with your service in a development cluster in real-time.

If you have admin-level access, Rancher also provides the ability to jump into a shell on any of the running containers, instantly.

Just like the real-time log tailing, the ability to execute a shell within a running container can be extremely handy for debugging or manipulating your service in a development cluster in real-time.

Managing Ingress

We’ve got our running workloads, but we need to manipulate our Ingress so that traffic can reach our containers. Click the ‘Load Balancing’ tab to view all defined ingress points in our Kubernetes namespaces.

From here, we can quickly see and understand what hostnames are configured, and how traffic will be segmented based on path prefixes in request URLs (“path-based routing”).

With Rancher, adding or modifying these Ingress configurations is quite simple, and is less confusing (and less error-prone) than using the YAML-based configurations required by ‘stock’ Kubernetes.

Take It For A Spin!

Rancher 2.0 helps fill a few remaining gaps in the Kubernetes ecosystem, and makes cluster management a breeze. With both Rancher and Kubernetes development moving forward at breakneck speeds, I can’t wait to see what the future holds for both of these platforms.

That’s all for now, so go install Rancher and get started!

Background

At the time of this post’s writing, Rancher (an open-source kubernetes cluster manager) v2.0.7 has just landed, and it includes SAML 2.0 support for Ping Identity and Active Directory Federation Services (AD FS). This development comes at the perfect time, as my organization is evaluating whether or not to use Rancher for our production workloads, and we are firm believers in federated identity management through our IdP provider, Okta.

But wait! Why just Ping Identity and AD FS? Isn’t that kind of unusual, given that SAML 2.0 is a standard? Is there something specific to these two implementations?

The short answer is, thankfully, no. After reviewing the relevant portions of the codebase, I can safely say it’s just vanilla SAML. I assume the Rancher team just started with Ping Identity and AD FS because they were the two top requested providers, which I’m sure they took the time to sit down and test against, write up specific integration instructions, screenshots, and so on. But I want to use Okta anyway, dang it! So, let’s go do that.

Configure Rancher

Log into Rancher with an existing local administrator account (the default is, unsurprisingly, ‘admin’). From the ‘global’ context, head over to Security -> Authentication, and select the Microsoft AD FS (yes, even though you aren’t actually going to be using AD FS for your IdP).

Now we tell Rancher which fields to look for in the assertion, and how to map them to user attributes. Okta allows us to specify what field names and values we send to Rancher as part of the setup process for our new SAML 2.0 app, but other IdPs may have pre-defined field names which you must adhere to. Please consult your IdP’s documentation if you have trouble.

I was confused by the ‘Rancher API Host’ field name. After digging around the Rancher source for a bit, I realized it’s literally just the external DNS name for the Rancher service; the same address as you type into your address bar to access your Rancher install.

Rancher’s SAML library includes support for receiving encrypted assertion responses, and appears to require that you furnish it with an RSA keypair for this activity. As a brief aside, I will actually not be enabling the encryption on the IdP side because I think that’s overkill in this use-case (and, frankly, I couldn’t get Okta to play nice with it either). Let’s generate the necessary certificate and key:

openssl req -x509 -newkey rsa:2048 -keyout rancher_sp.key -out rancher_sp.cert -days 3650 -nodes -subj "/CN=rancher.example.com"

Grab the contents of the rancher_sp.key and rancher_sp.cert files and place them into the appropriate configuration blocks (or upload the files from your computer, either way):

Leave that all open in a browser tab; we’ll come back to it shortly. For now, though, we need to go over to Okta.

Configure Okta (or some other IdP)

The rest of these instructions will be Okta-specific, but the concepts are not. Reach out to your IdP provider if you need assistance.

Create a new SAML 2.0 application:

Give it a name and proceed to the SAML settings page.

Single sign on URL:
https://rancher.example.com/v1-saml/adfs/saml/acs

Audience URI (SP Entity ID) (aka Audience Restriction):
https://rancher.example.com/v1-saml/adfs/saml/metadata

You should be able to leave the rest of the general options alone. Create two custom attribute statements. These are how we’ll tell Rancher what username and display name to use.

First attribute statement:
Name: userName
Name Format: Unspecified
Value: user.username

Second attribute statement:
Name: displayName
Name Format: Unspecified
Value: user.firstName + " " + user.lastName

If you haven’t guessed it by now, the user.* declarations are an expression syntax that Okta provides. If you need to use some other values for username/display name, feel free customize the fields Okta uses to fill in these values:
https://developer.okta.com/reference/okta_expression_language/#okta-user-profile

Create a group attribute statement, which will send all the groups you are a member of to Rancher, which will in turn be used to map groups to Rancher roles:

Name: groups
Name Format: Unspecified
Filter: Regex
Value: .*
^ (that's period-asterisk, the regex expression for "match all")

Perhaps you don’t want to send all your group information to your Rancher install; maybe you have a lot of groups not used for authorization for some reason? If that’s the case, you can create your own regular expression to try and ensure you get a tighter match. Do not attempt to restrict access to a given set of groups by using this filter though, as we’ll do that in Rancher directly in a much more user-friendly way.

Double-check that your options look like these options and proceed.

Save your new connector. Once saving is complete, you’ll need to click the ‘Sign-On’ tab and select ‘View Setup Instructions’:

Grab the IdP metadata and put it on your clipboard:

We’ll need it during the next step.

Now before you leave Okta, you need to complete one final task. Make sure you go into the newly-created Rancher SAML 2.0 app and assign it to yourself and anyone else you want to bestow crushing responsibility for production systems onto. If you forget this step, the final steps required later in Rancher’s configuration will fail.

Back to Rancher for the Final Steps

Head back over to that Rancher tab we left open and paste the IdP metadata into the ‘Metadata XML’ box:

Alright, in theory, that’s it. Click the ‘Authenticate with AD FS’ button and say a little prayer. Quick note: if nothing seems to happen, it’s likely because your browser blocked the pop-up. Make sure you disable the pop-up blocker for your Rancher domain and whitelist it in any other extensions you might utilize.

Proceed to sign in to your Okta account if prompted, though it’s likely you are already signed in from the previous steps. If you did everything correctly, you’ll be dropped back to the Rancher authentication page, only this time with some information about your SAML settings. Additionally, hovering over your user icon on the top-right should yield your name and your Okta username. Nifty!

Technically you are done! That said, I would recommend making one more tweak by changing the Site Access settings block to ‘Restrict access to only Authorized Users and Organizations’. This action will disable login from any other non-SAML source, including existing local users, unless the user is listed under the ‘Authorized Users and Organizations’ section, or you’ve explicitly added one of the groups (which are brought over from Okta) that a SAML user is part of. Quick note: Rancher will only know about groups you are a part of (the ones it received from your SAML assertion), which is unfortunately somewhat limiting.

Using Groups for RBAC

By default, your SAML users will receive no access to anything at all. When they log in, they’ll see no clusters. Let’s change that!

Select a cluster -> Members -> Add Member.

Now your users can see the cluster, but none of the Projects or pods inside. Time to repeat this process by authorizing a group to a particular project:

Conclusion

Rancher is a powerful tool for managing Kubernetes clusters, and the recently-landed SAML 2.0 support (with group awareness!) is a major step forward in terms of making the solution enterprise-ready. I’ve enjoyed working with the software and can’t wait to see where the project goes.

P.S. – if anyone from Rancher is reading this, you have my permission to re-use and re-distribute any screenshots or text in this blog post in any of your internal or customer-facing documentation/blog posts/wiki pages, should you find it useful.

Tag: rbac

An introduction to Rancher 2.0