Richard Hicks' Forefront TMG Blog

Microsoft System Center Data Protection Manager (DPM) 2010 is a management product that provides data protection for Windows systems. More advanced than Windows Backup, DPM uses Protection Agents that provide advanced capabilities. Getting the DPM server to communicate with the Protection Agent installed on a Forefront TMG 2010 firewall can be challenging, however. DPM server-to-agent communication takes place over several non-standard ports, and it also relies on DCOM. Unfortunately the Forefront TMG RPC filter does not fully support DCOM , so we’ll need to employ a workaround to ensure that DPM communication works correctly.

[Note: Many resources that I found on the Internet detailing how to configure TMG for DPM were incorrect and didn’t work. Those that did work didn’t explain why, involved unnecessary steps, or included very broad rule sets that allowed more access to the TMG firewall than absolutely necessary. In this post I’ll provide a definitive and comprehensive guide to preparing the TMG firewall to work with DPM and its Protection Agent, while at the same time adhering to the principle of least privilege and maintaining the lowest possible attack surface.]

As stated, DPM uses Protection Agents installed on each system it manages and protects. This agent can be deployed remotely from the DPM console or installed manually and later ‘attached’. When installing the Protection Agent on a Forefront TMG 2010 firewall it is recommended that the agent be installed manually following the instructions here. Since there is no system policy access rule for DPM, we’ll need to configure access rules to allow the required communication to and from the Protection Agent on the TMG firewall and the DPM server. Begin by creating three new protocols as follows:

DPM Agent Coordinator – TCP 5718 outbound
DPM Protection Agent – TCP 5719 outbound
DPM Dynamic Ports – TCP 50000-50050 outbound

Create a Computer or a Computer Set network object that includes the IP address of your DPM server. DO NOT add the DPM server to the Enterprise Remote Management Computers or Remote Management Computers network objects.

Next, create an access rule called DPM [Inbound]. The action will be allow and the protocols will include the three new protocols you just created, along with Microsoft CIFS (TCP) and RPC (all interfaces). The source will be the DPM server and the destination will be Local Host for all users. Now right-click on the access rule and choose Configure RPC protocol.

Uncheck the box next to Enforce strict RPC compliance and choose Ok.

Create another access rule called DPM [Outbound]. The action will be allow and the protocols will include only DPM Agent Coordinator [5718] and DPM Dynamic Ports [50000-50050]. The source will be Local Host and the destination will be the DPM server for all users. Once complete the rule set should look like this:

Next, right-click the Firewall Policy node in the TMG management console navigation tree and select All Tasks | System Policy | Edit System Policy. Under the Authentication Services configuration group highlight Active Directory. Select the General tab and uncheck the box next to Enforce strict RPC compliance.

The last step required to allow the DPM server to communicate with a Protection Agent installed on the TMG firewall involves making registry changes to restrict RPC communication to a specific range of ports. This is necessary because, as I mentioned earlier, the TMG RPC filter does not fully support DCOM and is unable to manage the dynamic port assignments required for this communication. This change must be made to both the TMG firewall and the DPM server. To make this change, open the registry editor on each system and navigate to the HKLM\Software\Microsoft\Rpc\Internet key.

Create the following new keys:

Ports – REG_MULTI_SZ, 50000-50050
PortsInternetAvailable – REG_SZ, Y
UseInternetPorts – REG_SZ, Y

You can download a .reg file and the TMG access policies here.

Note: For this limited demonstration I have chosen to restrict the dynamic port range to 50 ports. In a real production environment, you may need to increase this limit. An excellent reference article that includes a formula to determine the number of ports required can be found here.

Once the registry changes have been made, the system will have to be restarted for the changes to take effect. After both systems are back online, install the Protection Agent manually on the TMG firewall and then attach the agent in the DPM management console. You can now manage TMG firewall protection in DPM just as you would any other Windows system.