Office 365 – Hybrid Wizard Fails Due To WPAD / PAC

The Office 365 Hybrid Configuration Wizard (HCW) was introduced back with SP2 for Exchange 2010; while it has been a vast improvement over the former manually-intensive process, at times it can feel more like the “Hybrid Configuration Gremlin”…
Like most aspects of Office 365, the HCW has seen continual improvement from it’s initial introduction and in many situations it runs just fine. It does, however, seem to do a good job of uncovering the somewhat odd and maybe not-so-great configurations in your environment.

Issue

I recently worked with a client where during the Exchange 2013 HCW, I received the following errors in the logs:
ERROR : System.Management.Automation.RemoteException: Federation information could not be received from the external organization.
ERROR : Subtask NeedsConfiguration execution failed: Configure Organization Relationship
Exchange was unable to communicate with the autodiscover endpoint for your Office 365 tenant. This is typically an outbound http access configuration issue. If you are using a proxy server for outbound communication, verify that Exchange is configured to use it via the "Get-ExchangeServer –InternetWebProxy" cmdlet. Use the "Set-ExchangeServer –InternetWebProxy" cmdlet to configure if needed.
While the client did have a web proxy in the environment, I was told that the hybrid servers had direct access to the Internet. We were able to confirm this was true as I was able to browse to sites that would have been otherwise restricted by the proxy.

Troubleshooting

Unleash the Potential of Power Platform With a Center of Excellence

Business innovation often comes from within. Discover how to empower innovation from non-traditional developers with the Microsoft Power Platform.

Learn More

Stepping back from the HCW, I tried to run the following command:
Get-FederationInformation -DomainName tenant.mail.onmicrosoft.com -BypassAdditionalDomainValidation: $true -Verbose
The resulting error message contained this reference to proxy authentication:
Exception=The remote server returned an error: (407) Proxy Authentication Required
Other communication seemed to work fine, I could connect to Exchange Online using Remote PowerShell and “netsh winhttp show proxy” indicated there was no proxy in use.
After doing some digging, it appears that the commands run by the HCW (and my subsequent “Get-FederationInformation”) are run under the context of “Local System”. As such, these commands are subject to the proxy settings of the “Local System” user profile.
The default value of “Automatically Detect Settings” in the Internet Options is “Enabled” and is configured on a per-user basis; this also applies to “Local System”. This default setting, combined with WPAD and the client’s PAC file, was directing “Local System” to use the proxy. Since “Local System” is not an account with credentials valid for the proxy, authentication was failing.

Resolution

Options for resolution might include allowing the server subnet to use the proxy without authentication or modify the settings in the PAC file being distributed via WPAD. Unfortunately these are larger enterprise-wide changes that would require significant time for testing.
As a workaround, the value of “Automatically Detect Settings” can be changed to “Disabled” in the “Local System” profile and the direct path to the Internet can be used as intended. There are two ways you can do this:

1. Make sure your own profile has the value disabled and export the registry key from HKCU and import to the “Local System” (HKEY_USERS\.DEFAULT) hive.

HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections

2. Use a utility like “PsExec” to launch Internet Explorer as “Local System”, disable the setting and save the changes.

psexec.exe -i -s -d "C:\Program Files\Internet Explorer\iexplore.exe"

The workaround above allowed me to successfully run the “Get-FederationInformation” command and should allow the Hybrid Configuration Gremlin Hybrid Configuration Wizard to now complete successfully.