Admin Web Site Technologies

The IT Detective agency: Outlook client is Disconnected, all else fine

Today we were asked to consult on the following problem. Some proxy users at a large company could not connect to Microsoft Outlook. Only a few users were affected. Fix it.

The details
Affected users would bring up Outlook and within a few short seconds it would simply show Disconnected and stay that way.

It was quickly established that the affected users shared this in common: they use LDAP authentication and proxy-basic-authentication. The users who worked used NTLM authentication. The way they distinguish one from the other is by using a different proxy autoconfiguration (PAC) file.

More observations
Well, actually there was almost no difference whatsoever between the two PAC files. They are syntactically identical. The only difference in fact is that a different proxy is handed out for the NTLM users. That’s it!

We were able to reproduce the problem ourselves by using the same PAC file as the affected user. We tried to trace the traffic on our desktop but it was a complete mess. I did not see any connection to the designated proxy for Outlook traffic, but it’s hard to say definitively because there is so much other junk present. Strangely, all web sites worked OK and even the web-based version of Outlook works OK. So this Outlook client was the only known application having a problem.

When the affected users put in the proxy directly in manual proxy settings in IE and turned off proxy autoconfig, then Outlook worked. Strange.

We observed the header for the PAC file was a little bit inconsistent (it was being served from multiple web servers through a load balancer). The content-tyep MIME header was coming back as either text/plain or there was no such header at all, depending on which web server you were hitting. But note that the NTLM users were also getting PAC files with this same header.

The solution

Although everything had been fine with this header situation up until the introduction of Outlook, we guessed it was technically incorrect and should be fixed. We changed all web servers to have the PAC file be served with this MIME header:

Content-Type: application/x-ns-proxy-autoconfig

The results

A re-test confirmed that this fixed the Outlook problem for the LDAP-affected users. NTLM users were not impacted and continued to work fine.

A strange Outlook connection problem was resolved in large company Intranet by adjusting the PAC file to include the correct content-type header. Case closed!

References and related information
Here’s a PAC file case we never did resolve: excessive calls to the PAC file web server from individual users.

Internet Mail

Exchange Online Protection is currently broken. Resolved.

6/24th, 3:20 PM
A lot of my outbound emails are currently stuck with this status:

Deferred: 421 4.3.2 The maximum number of concurrent server connections has exceeded a limit, closing transmission channel

For instance, to

The MX record:

> dig +short mx

So it ends in

Same for emails to The MX record:

> dig +short mx

Also ends in is another one I see.

So all these domains we can’t currently email to have an MX record ending in and so I conclude are using Exchange Online Protection.

The situation has persisted for a couple hours so this doesn’t seem to be a 99.9999% uptime type of service.

Something went seriously wrong with Microsoft’s Exchange Online Protection service today.

6/25/14 update
Apparently this affected lots of Outlook users as well. It was finally fixed last night.