How to Enable Remote Authentication Fallback on F5 BIG-IP
Remote authentication… We all know the drill, it is way better than using local authentication, because then you have to administrate X amount of local databases on your network devices separately. Having a single source of truth for authentication is the dream! You add a user once and he/she will be able to log in to all your network devices.
But sometimes things break. Sometimes servers or networks can be down. If your remote authentication service is not available, logging into devices can be affected. Let’s see how to mitigate this issue on your F5 BIG-IP load balancers.
Authentication types
First of all, we need to talk about authentication types. There are 2 main categories:
- Local
- Remote
Local is straightforward, you set up your users in the BIG-IP configuration, under the System / Users / User list. This will be important later on, I will get there in a minute.
Remote is the other big category, and you can set up remote authentication in a couple of ways. You find the settings under System / Users / Authentication.
We have the following options for remote authentication:
- Radius
- TACACS+
- LDAP
- Active Directory
Related: How to Configure & Renew SSL Certificate on F5 BIG-IP
The test environment
In my test, I’m using RADIUS as a remote authentication solution. I have configured a Ubuntu server with FreeRadius. I configured one user (george) with a very simple password as “password-remote” in the RADIUS server.
The RADIUS server is in the same network as the BIG-IP management IP, but in a production environment, of course, this can be different.
The BIG-IP has been configured with Remote Authentication as you can see in the following picture.
Notice the Fallback to Local setting, it hasn’t been turned on. We will come back to this later.
So in general, whenever someone tries to log in to the BIG-IP web GUI or SSH, the device will ask the RADIUS server, if the user can be allowed to log in or not. If the RADIUS server sends an Access-Accept message to the BIG-IP, then the user is allowed to log in and work with the BIG-IP.
You can see this specific traffic in a packet capture as well.
Authentication scenarios
There are a couple of possible scenarios regarding remote authentication. Let’s see these:
- Authentication server is up and reachable
- Authentication server is down or unreachable, no local fallback
- Authentication server is down or unreachable, with local fallback
Authentication server is up and reachable
This is straightforward, if we have the user credentials which are matching with the authentication server setup, then our login will be allowed.
Authentication server is down or unreachable, no local fallback
We have an issue here, since if we have remote authentication enabled, but without a local fallback, then our BIG-IP will always try and try to reach the remote server. This means we won’t be able to log in with our user credentials. Even if we have a local user in the BIG-IP configuration, BIG-IP won’t ever use that.
One exception here, and this is our only way in similar situations. The “admin” and the “root” users are always checked against the local configuration. So even if we have a remote authentication setup, and the remote server is unreachable, BIG-IP will let us in if we are using the “admin” or the “root” user. The root is only working via SSH, not via the web UI. These are always present, built-in user accounts, and they are always matched against the local credentials (even if the remote authentication server is reachable).
Authentication server is down or unreachable, with local fallback
This is the best solution since we can log in to the BIG-IP even if the remote authentication server is down. However, if the remote authentication server is up and reachable, then BIG-IP will always use that for login attempts, even if we try the correct local username/password, BIG-IP won’t let us in (unless we set up the same username/password both in the local and the remote user database).
As previously discussed, admin/root users can log in with their locally set credentials, even if the remote authentication server is reachable. In this case, BIG-IP won’t even turn to the authentication server.
Related: How to Write F5 iRules with Examples
Enable the Fallback to Local
So now it should be clear why it is important to set up the Fallback to Local option in the remote authentication configuration. Let’s go back to the System / Users / Authentication page, and change the current config.
A really important step is that we need to add a local user as well, so we can fallback to that in case of a remote authentication server failure. Let’s go to System / Users, and press the Create button. Let’s add a new local user for “george”, now with a local password “password-local”.
Now we can test and evaluate the different scenarios. Let’s check the authentication logs as well during the different scenarios. To see that, we need to check the /var/log/secure file’s content.
Remote authentication server is reachable
- Testing with the remote user credentials will allow us to login
[admin@f5-test:Active:Standalone] ~ # tail -f /var/log/secure Aug 27 03:32:20 f5-test.local notice httpd[12030]: pam_radius_auth: pam_radius_auth: user george successfully authenticated Aug 27 03:32:20 f5-test.local info httpd(pam_audit)[12030]: user=george(george) partition=[All] level=Administrator tty=(unknown) host=18.92.203.246 attempts=1 start="Sun Aug 27 03:32:20 2023" end="Sun Aug 27 03:32:20 2023".
- Testing with the local user credentials won’t allow us to login
[admin@f5-test:Active:Standalone] ~ # tail -f /var/log/secure
Aug 27 03:34:50 f5-test.local notice httpd[9582]: pam_radius_auth: pam_radius_auth: user george not authenticated
Aug 27 03:34:50 f5-test.local notice unix_chkpwd[12822]: password check failed for user (george)
Aug 27 03:34:50 f5-test.local notice httpd[9582]: pam_unix(httpd:auth): authentication failure; logname= uid=48 euid=48 tty= ruser= rhost=18.92.203.246 user=george
Aug 27 03:34:52 f5-test.local err httpd[9582]: [auth_pam:error] [pid 9582] [client 18.92.203.246:56794] AUTHCACHE PAM: user 'george' (fallback: false) - not authenticated: Authentication failure, referer: https://34.147.0.203:8443/tmui/login.jsp
Remote authentication server is unreachable
- Testing with the remote user credentials won’t allow us to log in, since the authentication server is not reachable, and the local credentials are different
[admin@f5-test:Active:Standalone] ~ # tail -f /var/log/secure
Aug 27 03:37:42 f5-test.local err httpd[12030]: pam_radius_auth: RADIUS server 10.10.10.13 failed to respond
Aug 27 03:37:45 f5-test.local err httpd[12030]: pam_radius_auth: RADIUS server 10.10.10.13 failed to respond
Aug 27 03:37:45 f5-test.local err httpd[12030]: pam_radius_auth: All RADIUS servers failed to respond.
Aug 27 03:37:45 f5-test.local notice httpd[12030]: pam_radius_auth: Auth server unavailable, trying fallback
Aug 27 03:37:45 f5-test.local notice httpd[12030]: pam_radius_auth: pam_radius_auth: user george not authenticated
Aug 27 03:37:45 f5-test.local notice unix_chkpwd[12889]: password check failed for user (george)
Aug 27 03:37:45 f5-test.local notice httpd[12030]: pam_unix(httpd:auth): authentication failure; logname= uid=48 euid=48 tty= ruser= rhost=18.92.203.246 user=george
- Testing with local user credentials will allow us to login, since fallback is enabled
[admin@f5-test:Active:Standalone] ~ # tail -f /var/log/secure
Aug 27 03:40:09 f5-test.local err httpd[9582]: pam_radius_auth: RADIUS server 10.10.10.13 failed to respond
Aug 27 03:40:12 f5-test.local err httpd[9582]: pam_radius_auth: RADIUS server 10.10.10.13 failed to respond
Aug 27 03:40:12 f5-test.local err httpd[9582]: pam_radius_auth: All RADIUS servers failed to respond.
Aug 27 03:40:12 f5-test.local notice httpd[9582]: pam_radius_auth: Auth server unavailable, trying fallback
Aug 27 03:40:12 f5-test.local notice httpd[9582]: pam_radius_auth: pam_radius_auth: user george not authenticated
Aug 27 03:40:12 f5-test.local info httpd(pam_audit)[9582]: user=george(george) partition=[All] level=Administrator tty=(unknown) host=18.92.203.246 attempts=1 start="Sun Aug 27 03:40:00 2023" end="Sun Aug 27 03:40:12 2023".
In conclusion, it is a good practice to enable Fallback to Local in the remote authentication configuration, since you can never know when there is an authentication server failure or network outage. Also if you have multiple F5 operators, maybe with different levels of access, then Fallback to Local is better than handing out the admin credentials to all operators.
