Access to the Web Proxy Filter on Forefront TMG 2010 is Denied
Frequently I am asked to review Forefront TMG 2010 firewall logs for suspicious behavior. Often times a security administrator will express concerns about many instances of denied requests by clients attempting to connect to Forefront TMG’s web proxy service. On busy TMG firewalls there may be hundreds or even thousands of instances where the following access denied record appears in the Web Proxy logs:
Status: 12209 Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
On a Forefront TMG 2010 firewall where web access rules require authentication, this behavior is expected and by design. It does not indicate an attack of any type on the Forefront TMG firewall or its web proxy service. The root cause for the flood of access denied messages has to do with how the Web Proxy client behaves when accessing resources via an authenticating web proxy like the Forefront TMG 2010 firewall. When a Web Proxy client sends its initial request for a resource it will always attempt to do so anonymously. Only when prompted for authentication by the firewall will the web proxy client provide the credentials of the logged on user.
Consider a scenario where Forefront TMG is configured to only allow authenticated users to access the Internet. The firewall policy might look something like this:
Below is a network trace taken from a client attempting to access http://www.bing.com/ through a TMG firewall as configured above.
We can see that the first three packets of the trace are the TCP three-way handshake taking place between the web proxy client and the Forefront TMG firewall. Once a connection to the web proxy listener has been established, in packet 8 the client sends an HTTP GET request for http://www.bing.com/. In packet 13 you’ll see that the Forefront TMG firewall denied the request and replied with an HTTP 407 response, indicating that proxy authentication was required. This was done because the Forefront TMG firewall did not have any access rules which would allow the anonymous request. It did, however, have access rules that might apply to this request, depending on who the user is. This response also includes which authentication methods the web proxy listener is configured to accept.
In packet 15 the web proxy client again submits its HTTP GET request for http://www.bing.com/, this time indicating that it would like to use the NTLM Secure Service Provider (SSP). In packet 16 the Forefront TMG web proxy denies the request yet again and replies with another HTTP 407 response, this time including the NTLM challenge. In packet 17 the client submits an HTTP GET request for http://www.bing.com/ and supplies the credentials in the form of an NTLM response.
As you can see, each time a web proxy client requests a resource through a Forefront TMG firewall that requires NTLM authentication the client is actually denied twice during the transaction before being successfully authenticated and allowed access. If this sounds like a lot of overhead for authenticated proxy traffic, you are right. Denying each request twice consumes additional resources on the Forefront TMG firewall and introduces some latency for clients as well. In addition, the burden of authenticating the user is placed on the TMG firewall when using NTLM, as the firewall itself must contact a domain controller to authenticate the user. You can reduce the authentication load on the Forefront TMG firewall considerably by enabling Kerberos authentication. When the Forefront TMG firewall is configured to use Kerberos there is only a single denied request and HTTP 407 response. The client must then contact a domain controller and obtain a Kerberos ticket to present to the TMG firewall to gain access to the resource. Information on how to configure Microsoft ISA Server and Forefront TMG 2010 to use Kerberos authentication can be found here.
Additional information…
HTTP response codes – http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
NTLM challenge/response – http://en.wikipedia.org/wiki/NTLM
Connect
Richard Hicks' Forefront TMG Blog 
Hi Richard
Great information !!!
Thanks! Glad you found the post informative and helpful. 🙂
I’m getting this error when trying to publish to an report internal Reporting Services via Visual Studio 2005. It makes no sense as it should not be going through the proxy to get to this server.
Absolutely baffling.
If the request is for a resource for which the client does not have to use the proxy to connect to, you’ll need to configure your proxy exceptions accordingly. This can be done using TMG if your are using WPAD or the automatic configuration script, or on the client side in the web browser.
hi ,
what if i whant to allow web access for the comuters and users that are not in the active directory , is there a possipility that TMG not to request authentication.
I have TMG 2010, web proxy configured on it i have restricted sites with web access policy for my AD users acording on AD groups and it works just fine. but we have a lot of guest every day and they need to access web sites but they can’t, the authentication window shows up so i need to allow anonymus user to access websites i tryed to allow for all users but it did not help they sare still block. what shlould i do?
Yes, that is certainly possible. To configure Forefront TMG 2010 to allow access without requesting authentication, simply configure your access rule to apply to “all users”.
You will need to create another access rule on the Forefront TMG 2010 firewall that applies to “all users” and restrict access to your guest network.
I have tried it already but the browser still opens the authentication window if i cancel that window it does not lets me to enter sites. my goal is to allow guests not restrict. with none http traffic allowing “all users” allow guests without authentication but it does not work for http or https
In that case you’ll probably need to create a separate guest network. Forefront TMG 2010 does not provide a way to authenticate some requests and not others on the same network.
Hi Richard,
Thanks for the information, it´s very helpful!
I have one question about the anonymous authentication, in some cases the user authentication doesn’t work, just stays in the first denied request and to make it works we have to create one rule in TMG with permit action from “All Users” to this specific web. We have this scenario only with some specific web pages. The last case with this issue was a connection to metasploit.com:443. Do you know why this happens?
This is the log information:
Denied Connection
Log type: Web Proxy (Forward)
Status: 12232 Forefront TMG denied the specified Uniform Resource Locator (URL).
Rule:
Source:
Destination:
Request: metasploit.com:443
Filter information:
Protocol: SSL-tunnel
User: anonymous
This can happen when the request is made by a SecureNAT client, and often occurs in web proxy clients that don’t know how to handle the HTTP 407 response generated by the Forefront TMG 2010 firewall. The way to resolve this is to create an anonymous access rule (as you did) or to install the Firewall Client. I prefer the latter because I can enforce authentication for all outbound TCP and UDP-based access requests, not just web proxy requests.
HI
Nice blog
can you please provide some info for mobile devices how we need to configure with no proxy using TMg2010
The best advice I can give is to create an isolated network for mobile devices and make them SecureNAT clients.
How to creat isolated network?
This involves adding a network interface to your server and assigning a unique IP subnet to it dedicated to these clients. You’ll then need to create a network definition in Forefront TMG 2010 for it, as well as establish a network relationship (NAT or route) and create any access rules required for access.
Hi Richard,
I have created a web service and windows client which consume that web service. Web service is hosted on a web server which is visible to the outside world. I have tried to install the client on windows 7 machines outside our domain. They are able to connect to it and work fine. But one of our client has windows xp machine and he is not able to connect to it.
Below is the message that he is seeing every time he tries to connect.
The request failed with HTTP status 407: Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ).
I am not sure whats going wrong here, as some of the machines are able to access the web service and run the windows client absolutely fine, but one client is not able to run the windows client which consumes our web service. Although he is able to click on the web service URL and he sees the methods in our web service.
Please suggest how can i figure out the exact cause of the problem and eventually fix it.
The HTTP 407 is pretty self-explanatory. If you’re receiving this message it is because the access rule that allows the traffic requires authentication and the client was unable to handle it. If you are certain that this client hit the correct publishing rule on your TMG firewall, I’d suggest making sure that the rule applies to “all users” and that your back end web server does not also require authentication.
Thanks Richard, my problem was resolved by using below block in the configuration file of the client application.
Although i am not 100% sure on the reasoning behind the solution but i could infer that may be this block will force the client machine proxy to use default credentials to connect to the web server.
Please suggest if my understanding is correct here ?
I am not able to put configuration code block in the reply some how.. 😦
Hi richard thnx for this informations ; i’m a consultant continuous integration and i’m working in a project for a company and i think they are using Forefront TMG : because i have to “mavenize” a project for them and i’m using their internal computer and when i’m trying to DL jars from maven repository, i’m facing this problem :
“Not authorized by proxy , ReasonPhrase:Proxy Authentication Required ( Forefront TMG requires authorization
to fulfill the request. Access to the Web Proxy filter is denied. )”.
What have i to do to bypass this problem ? contacting the security administrator and asking what exactly ? thanks for your answer
There’s nothing you can do to bypass the issue on your side I’m afraid. You’ll have to reach out to the administrators responsible for the TMG server in question and provide them with these details. From there, they can proceed with troubleshooting as required.
thnx for your answer, i’m blocked for while right now and i’ll provide them these details, but what they will do exactly ? configuring forefront with my session and giving me the necessary credentials ? sorry but i want more informations to convince them . Thnx for the reply.
Either provide you with the necessary credentials or update their settings to allow unauthenticated access. It’s up to them. Send them the screen shot, that should be enough information for them.