I’m happy to announce that I’ll be one of the speakers at Zabbix Summit 2022 in Riga. As moderator of the Zabbix Telegram and Zabbix reddit community, I was asked by Zabbix to present at the premier event about Zabbix monitoring.
The 10th Zabbix Summit is taking place in Riga, Latvia on October 7-8. My speech is titled:
As a Managed Services Provider (MSP) we are responsible for dozens of customers, with hundreds of Windows Servers. Therefore, we introduced mechanisms to automatically deploy, configure, and sync Windows Zabbix agents across all these isolated environments. In this process, we leverage a tool set consisting of Azure DevOps, Git, PowerShell, and a publicly available Zabbix agent auto-updater, to provide all our customers’ servers with the latest Zabbix agent configuration possible.
Marco Hofmann is an IT-Systems Engineer with 15+ years worth of experience. He works at the german Managed Services Provider ANAXCO GmbH. His priority is the configuration and maintenance of the company’s central Zabbix environment, which is a key component in the fulfillment of the customers’ maintenance contracts. He also focuses on designing and configuring Citrix RDSH environments, fully automated with PowerShell.
Microsoft offers a nice set of security baseline GPOs, for direct use in your Active Directory environment. If you make use of the “MSFT Windows Server 2022 – Domain Controller” policy, your NPS installation might start to fail.
Introduction
Many of you probably have a Citrix NetScaler Gateway installation based on the following concept:
Those articles describe, how someone can implement Azure MFA with Microsoft Authenticator App pushOTP and an on-premises Microsoft NPS server, without making use of SAML, which is necessary, if you use the ICAProxy Gateway only license for Citrix NetScaler Gateway.
The Error
While implementing the Microsoft security baselines at a customers’ site, we also introduced the Domain Controllers security baseline called: MSFT Windows Server 2022 – Domain Controller
MSFT Windows Server 2022 – Domain Controller
A few minutes later, the external Citrix NetScaler Gateway authentication stopped working. Users would only receive the error: Unknown username or password
Citrix NetScaler Gateway – Login failed
Troubleshooting on the Citrix NetScaler Gateway through the aaad.debug showed the following error:
MS-CHAP error ERROR_AUTHENTICATION_FAILURE (691)
MS-CHAP error ERROR_AUTHENTICATION_FAILURE (691)
While the event viewer on the NPS server told us:
NPS Extension for Azure MFA: NPS Extension for Azure MFA only performs Secondary Auth for Radius requests in AccessAccept State. Request received for User Marco.Hofmann with response state AccessReject, ignoring request.
NPS Extension for Azure MFA only performs Secondary Auth for Radius requests in AccessAccept State. Request received for User Marco.Hofmann with response state AccessReject, ignoring request.
The solution
The error weren’t helpful at all. We spent quite too much time searching for an indicator of what might be the issue. We knew it must have to do with the Domain Controller security baseline. Finally, we stumbled at the following post from serverfault.com:
One of my colleagues was at a Microsoft conference having various discussions when it dawned on him that MSCHAPv2 relies on NTLM to generate the password challenges and responses. Now plain old MSCHAP and MSCHAPv2 (i.e. not EAP-MSCHAPv2 or PEAP) when used in Windows RAS services will use NTLMv1 by default.
As many of of you have already started to catch on, we, like many administrators, have disabled NTLMv1 on our DCs and as such the DCs will only accept NTLMv2 requests. This explains why the failure I continued to get was a “bad password” error. The password being sent to the DCs was in NTLMv1 format and was getting ignored.
This post then lets to a Microsoft article, with the solution:
For example, when you set this value to 5 (Send NTLMv2 response only. Refuse LM & NTLM), the DC won’t accept any requests that use NTLM authentication. When MS-CHAP or MS-CHAPv2 are configured, RAS in Windows Server 2008 R2 will default to NTLM to hash the password. Because the DC only accepts NTLMv2, the request will be denied.
Microsoft NPS on Windows Server 2019 and 2022 will use NTLMv1 by default, if you make use of MS-CHAPv2. And the MSFT Windows Server 2022 – Domain Controller security baseline will turn off NTLMv1 for your Domain Controllers, which will break your NPS server.
To enable NTLMv2 for MS-CHAPv2, you must set the following registry key on your NPS server, and restart the NPS service, and it will start working again:
Today I learned: Citrix Printer Redirection for Mac and Linux is not available Out-Of-The-Box on the Citrix Master Images I install. In this blog post, I will detail the steps I’ve taken to add this missing piece to all of my Master Images.
Introduction
Although quite rare in Germany, a few of our customers use MacBooks at home for the home office. Last week a customer created a ticket, that he couldn’t print on his home printer, although printer redirection was allowed for him in Citrix Studio. At first, I suggested the usual troubleshooting steps, like updating the Citrix Workspace app for Mac and checking the local preferences of the app. Obviously, both didn’t help. I then checked the problem myself and didn’t find an obvious problem. A quick search then led me to two Citrix KB articles, which revealed a knowledge gap on my side:
Client Printing from Linux/MAC is not available on Windows Server 2016 and 2019 (out of the box)
The following Citrix knowledge base articles explain what’s necessary to activate that feature:
An update guide for Citrix Workspace Environment Management (WEM) to the latest version 2106.
On June 16, 2021 Citrix released version 2106 of Workspace Environment Management (WEM). This is an update guide. Customer Success Services / Software Maintenance eligibility date: May 15, 2021
