Home > Forefront TMG 2010, Utilities > Configuring Syslog on ISA and TMG with Splunk Log Management

Configuring Syslog on ISA and TMG with Splunk Log Management

April 4, 2010

[Updated July 26, 2011: You can also use the Splunk Universal Forwarder to deliver Microsoft ISA Server and Forefront TMG 2010 log files to a Splunk indexing server. More details here.]

In a recent ISAServer.org article I wrote about the enhancements made to the logging infrastructure in Microsoft Forefront Threat Management Gateway (TMG) 2010. With regard to logging, one commonly requested feature for ISA and TMG is integration with syslog. Many organizations collect log data from numerous systems and network devices in their environment and aggregate that data in a central repository. This makes auditing, reporting, and event correlation with multiple systems much easier.

Although ISA and TMG do not support syslog natively, this functionality is available by using a free third-party utility. Snare Epilog for Windows is a tool that takes data from ISA and TMG log files and exports them to a syslog server such as Splunk.

Before installing Snare Epilog for Windows you will need to change the Firewall and Web Proxy logging properties to use the text file format. Be advised that when you do this, you will no longer be able to view historical log data in the ISA or TMG management console.

Configure Logging

To configure TMG for text file logging, open the management console and highlight Logs & Reports in the console tree, then select the Logging tab.

To configure ISA for text file logging, open the management console and highlight Monitoring in the console tree, then select the Logging tab.

For both ISA and TMG, click Configure Firewall Logging or Configure Web Proxy Logging in the Tasks pane.

Select the File option and choose W3C Extended Log File Format. Do the same for Web Proxy Logging.

Configure Syslog Access Rule

To allow ISA or TMG to communicate with a remote syslog server, an access rule must be created that allows syslog traffic from the Local Host to the syslog server. This will require a new custom protocol to support syslog (UDP 514) with a direction of send.

Once completed, the access rule should look like this.

Installing and Configuring Snare Epilog for Windows

Download and install Snare Epilog for Windows on the ISA or TMG firewall. For ISA or TMG arrays, install Epilog on each array member. Installation is simple and straightforward – just accept the defaults until completed.

To configure Epilog to send log data to a remote syslog server, such as a Splunk log management server, navigate to Start/All Programs/InterSect Alliance and select Epilog for Windows. On the left side, click Log Configuration.

Click Add, and then in the drop-down box, select Microsoft ISA Firewall Logs.

In the Log File or Directory field, enter the location of the firewall logs. For TMG, the default log folder is C:\Program Files\Microsoft Forefront Threat Management Gateway\Logs. For ISA, the default log folder is C:\Program Files\Microsoft ISA Server\ISALogs. In the Log Name Format: field, enter *FWS*.w3c. Click Change Configuration when finished.

To add Web Proxy log files, repeat the steps above, selecting Microsoft ISA Web Logs as the log type and specifying *WEB*.w3c for the log name format.

Note: For demonstration purposes I have used the default location for the ISA and TMG log files. Best practices dictate that the log files be located on a separate physical disk from the system partition, however.

Click Network Configuration.

Enter the IP address and destination port for your syslog server, and then click Change Configuration.

After completing the log and network configuration, click Apply the Latest Audit Configuration to complete the process.

To confirm that Epilog for Windows is configured correctly, click Latest Events to view the current events. Epilog will display any monitored log information.

If everything is configured correctly and working properly, you should now see ISA or TMG log data in your syslog console. If you do not see log data in your syslog console, you may need to restart the Epilog service on the ISA or TMG firewall.

  1. Dawid
    May 27, 2010 at 4:46 pm

    Thanks,
    for inspiring combination of tools. I’m curious, is there a way to use System Center Configuration Manager (with “some” MPs) instead of Splunk?

  2. May 30, 2010 at 6:13 pm

    There is a management pack for Forefront Threat Management Gateway (TMG) 2010 – http://bit.ly/avbXEG – but unfortunately it does not do log aggregation or reporting.

  3. Dawid
    May 31, 2010 at 3:28 am

    Thanks Richard,

    yes, the Microsoft’s TMG MP is unsuitable for logs, that’s way I asked you about “some” MPs, which can aggregate (or even report) logs (it’d be best not only TMG logs), ’cause I’ve been investigating this kind of a SCOM functionality quite a while, but looks like it’s unviable. Maybe that’s why you ended up with Splunk, didn’t you?

  4. June 4, 2010 at 10:42 am

    You are absolutely right. WebSpy is pretty good too, but Splunk seems to be more popular, especially with the network infrastructure folks.

  5. Dawid
    June 19, 2010 at 4:27 am

    And how do you use Splunk to create similar reports like ISA Reporting with all those graphs and stuff? Don’t you miss them?

  6. June 19, 2010 at 4:42 pm

    Honestly, I have little experience using Splunk with TMG. Splunk is more popular with the network infrastructure guys, that’s what prompted the post. I prefer to use WebSpy Vantage. It is very feature-rich and there are plenty of charts and graphs, which management always seems to like. : )

  7. M G Patel
    July 19, 2010 at 11:19 pm

    Thanks for this really helpful post. Just 1 ?
    How do i configure Splunk to read the data being “pushed” from Epilog? i just installed Splunk but have no clue what the next step is to read/view the data similar to your screenshot.

  8. July 30, 2010 at 1:19 pm

    Since I’m not that familiar with Splunk, I can’t answer that question for you. You can find more information about Splunk and reporting here.

  9. February 3, 2011 at 2:40 pm

    What specs do I need on the logging machine? how much load do these logs thro on the LAN?

  10. February 3, 2011 at 2:42 pm

    There are a list of Web Spy , which one do you use or recommend for the internet monitoring?

  11. February 4, 2011 at 10:25 am

    Depends entirely on your traffic profile and current load. I can’t give a specific hardware recommendation…too many factors to consider. I’d suggest testing in a lab environment and extrapolating the results. In addition you can use Splunk’s native log forwarding capabilities instead of using Snare. That option will likely require less overhead than this solution.

  12. February 4, 2011 at 10:27 am

    WebSpy is my favorite, but there are many other alternatives. Check http://www.isaserver.org/software/ISA/Reporting/ for more suggestions.

  13. Foti
    August 19, 2011 at 10:14 am

    Richard,
    What you described is exactly what we’re trying to do… but it SEEMs the Epilog agent for windows has an issue running the Win2008/64bit platform. Can you tell me what version of Windows Server you were doing this on?

  14. August 19, 2011 at 10:29 am

    Interesting. I tested this with Forefront TMG 2010 running on Microsoft Windows Server 2008 R2 and didn’t encounter any problems.

  15. November 29, 2011 at 3:21 pm

    Foti :
    What you described is exactly what we’re trying to do… but it SEEMs the Epilog agent for windows has an issue running the Win2008/64bit platform.

    Hi Foti,

    Epilog should work fine on 2008/64bit, drop us an email via the intersectalliance.com site if you’re still having problems, and we’ll try and sort it out for you.

    Leigh (Snare Dev)

  16. March 2, 2012 at 2:52 am

    Hi Richard.
    Was there any special config you had to do apart from the firewall rule for UDP? I have a tmg2010 on win2k8r2 and a GFI win2k8r2 server and although I see my traffic leaving my firewall I dont see it arriving on GFI eventmanager. These are on the same network so it should not be problem but I remember something about UDP not being happy going through a firewall in some cases. Do you know of any tcp syslog senders?
    Regards,
    Morris

  17. March 3, 2012 at 3:01 pm

    No special configuration other than creating the access rule correctly. Be sure you have the protocol direction configured correctly (should be send). Does the Forefront TMG 2010 firewall log indicate the traffic is being allowed? Have you confirmed on the remote server that the packet is being received? You may need to confirm this using Network Monitor. I’m not familiar with GFI event monitor, so make sure it fully supports syslog.

  18. Brian Long
    June 17, 2015 at 9:25 am

    We’re still running TMG Forefront 2010. I just set up Kiwi Syslog and would love to capture logs from TMG. I followed your instructions, but it appears the Epilog product has changed. I opted for the open source version, but once installed, when I tried to launch it (http://localhost/6161) I got page cannot be displayed. I stopped/restarted the service, and un/reinstalled with the same result. Running on Server 2008 R2 Enterprise 64 bit. Any suggestions?

  19. June 22, 2015 at 7:40 am

    I haven’t used Epilog in ages, so I’m not surprised to hear that things have changed. It’s been so long since I’ve used this that I’m afraid I don’t have any suggestions for you. Sorry about that. 🙂

  1. May 24, 2011 at 2:38 pm
  2. June 12, 2015 at 5:36 am
Comments are closed.