Best practices when deploying TMG URL Filtering

Microsoft Threat Management Gateway 2010 brings a lot of new and enhanced features to edge network security. Ever since Microsoft started with Proxy Server 2.0, and then ISA Server, URL filtering has always been something which administrators have wanted. Every company has their own IT policies and most companies want to restrict their users from visiting inappropriate or unsafe websites during their office hours from company-owned machines. Microsoft TMG has a fully functional URL filtering system and it uses 70+ URL categories with millions of URLs categorized in them.

We have been involved with deploying TMG for URL filtering for many organizations, from small businesses to organizations with 10,000+ seats. We can say we have done some of the biggest URL filtering deployments in United States using Microsoft TMG. Enabling or disabling URL filtering in Microsoft TMG server is pretty easy, the tough part is to plan the URL filtering across different departments within organizations and planning the exceptions.

In this article we will discuss some of the lessons learnt in deploying URL filtering for large organizations with multiple departments, as well as things you should keep in mind when planning URL filtering.

Let’s divide this article into two parts: Planning and Deployment.

Planning URL Filtering

Following are some of the key things you should keep in mind when planning the URL filtering using the Microsoft TMG server:

1. Decide the categories to be blocked based on the IT policy of the company. If your company doesn’t have a policy in place, then this is the right time to create one.
2. Gather as much information as you can from the key stake holders in different departments. Ask relevant questions about the websites they usually access and are required for their day to day working. Also, if you think some of their URLs/domains could be part of the URL categories which are being blocked, take a note of those URLs.
3. Decide on the ways the exceptions will be created. You may either use the Active Directory groups to make exceptions or the exceptions can be made based on the machine IP address or set of IP addresses and subnets. If there are just hand full of exceptions then the best approach is to make exceptions based on IP addresses because when using the AD users/groups, you will have to either configure the client machines as Web Proxy clients or TMG clients. So, strategically plan the exceptions. In one of our deployments we chose to make IP based exceptions as the customer did not want to authenticate users and wanted to keep the access anonymous.
4. Logging could be very important part of the deployment, but for some companies logging allowed requests does not matter. Make sure if the customer needs logging enabled or not.
5. Note the kind of applications which needs internet access and if they would need anonymous access or authenticated access to the internet.
6. Microsoft TMG includes many other features such as HTTP malware exception and HTTPS inspection; gather information from the customer to see if they want to implement these features. Additional administrative work is needed to enable these features and make it to work. For more information on these topics check http://technet.microsoft.com/en-us/library/dd182018.aspx and http://technet.microsoft.com/en-us/library/ee658156.aspx respectively.
7. Gather information on how does the customer wants the user to be notified that they are blocked. There are two ways you can do so:
   1. Use the default error pages on the TMG server and customize the content (needs a bit of HTML knowledge). It’s very easy now to customize the pages in Microsoft TMG SP1. A whole new range of error pages have been released by Microsoft and they can be activated from within the TMG Management console.
   2. Host a webpage internally on a web server and then redirect the requests to that web page.
8. Gather information from customer whether they want to allow users to override denial to a specific URL/Category or the set of URLs/Categories

 
Deploying URL Filtering

1. When deploying, make sure to keep all your notes handy. You don’t want to miss an exception and block something which can bring access to that website down.
2. Activate the URL filtering with the EA license agreement number and update it to sync with the Microsoft update center. The URL filtering definitions are downloaded every 15 minutes by default.
3. Create general internet access rules first and test the internet connectivity from the test machine. Make sure all the important websites on the internet are accessible including the websites listed under exceptions in your notes.
4. Follow the Firewall Policy best practices listed at http://technet.microsoft.com/en-us/library/cc995156.aspx.
5. Create a general deny rule specifying the categories to be denied for everyone. Test the rule to make sure the websites are being blocked. Additionally, make sure you have “All users” selected under the users.
6. Add general exceptions to the deny rule at this time and make sure the exceptions are working as expected. There are couple of ways you can make an exception:
   1. Create a Domain Name Set or URL Set in the TMG and add it to the “Exceptions” under the “To” tab.
   2. Allow access to the Domain Name Set or the URL set you created and place the rule just above the general deny rule.
7. Place the rules which needs authentication as per the guidelines laid down in the TechNet article mentioned in point #3 above.
8. Finally, re-check the policies before applying the changes to ensure accuracy.

As you can see, planning sounds more tedious than the actual deployment of the URL filtering and it can become more complex if you have large number of exceptions and more departments that deal with their own set of requirements. Ultimately, the key is to carefully write down their requirements and implement them one by one in order to reduce the chance of incorrectly implementing policies on the production TMG servers.

 
nAppliance team