Website Categorization Explained

This week’s topic: Website Categorization


TL;DR (Too Long Didn’t Read):

ObserveIT can detect users visiting particular categories of websites by leveraging NetStar’s inCompass website categorization module.


What is it?

The Internet is the first thing that humanity has built that humanity doesn’t understand, the largest experiment in anarchy that we have ever had.

-Eric Schmidt, Google


The internet is a massive place. At any given time, the indexed internet contains around 4.5 billion unique web pages and billions of more subpages. Website Categorization quite simply places this extensive number of websites into an appropriate category. For Example, would be placed into the category Social Networking and would be categorized as a gambling website.


Why is it valuable?

Most organizations have an acceptable use policy in place when it comes to employees and web browsing. Taken directly from a SANS acceptable use template, the verbiage might look something like this:

3.1 Internet Services Allowed

Internet access is to be used for business purposes only. Capabilities for the following standard Internet services will be provided to users as needed:

  • E-mail: Send/receive E-mail messages to/from the Internet (with or without document attachments).
  • Navigation: WWW services as necessary for business purposes, using a hypertext transfer protocol (HTTP) browser too. Full access to the Internet; limited access from The Internet to dedicated company public servers only.

Business purposes is a very vague term, and, as you can imagine, some websites are not always what they seem. While some websites are exactly what they seem, for instance, the Website is exactly what it says it is…..

Web Categorization DMX Website


…..Other websites may be very misleading……

Web Categorization Chatroulette

There are some places on the internet you may not even know existed …

Web Categorization Deep Web

(More on the deep web in another episode of Insider Threat Tips)


Some organizations are very strict and may block access to certain websites using a web content filter. Other companies may be very loose and allow users to use their best judgment. In either practice, security and IT teams often need the ability to detect when a user is going to a website not related to business needs. It would be a nearly impossible task for teams to categorize every website in creation, so these teams really need services and products to do it for them.


Website categorization & ObserveIT

The ObserveIT website categorization module categorizes 28 billion URLs and domains.

The categories that are pre-defined per our documentation:

  • Malicious
  • Infected Malicious
  • Phishing
  • DDNS Services
  • Remote Proxies
  • Copyright Sensitive
  • Legal-Sesitive
  • Adults
  • Illegal Drugs
  • Gambling
  • Search Engines & Portals
  • Job Searching
  • Downloads
  • Music
  • News
  • Sports
  • Gaming
  • Shopping
  • Social Media Sites
  • Streaming
  • Storage
  • Counter-Productivity
  • Web Mail
  • Chats
  • Instant Messaging
  • P2P
  • Ads

This module relies on the inCompass solution by NetSTAR that maintains a huge database of URLs and their respective categories.

Here are the alerts in the Insider Threat Library that leverage website categorization:

Web Categorization Unacceptable Use Web Categorization Time Fraud

Here is what the ObserveIT metadata would look like when a user goes to a website like

Web Categorization ObserveIT Metadata


In version 6.7, ObserveIT introduced a Website Categorization module that is capable of identifying the category of a website browsed to by employee. ObserveIT can detect a user that is browsing to a website of a predefined category and then trigger an alert to the security administrator, and display an optional message to the user.

Web Categorization Stay Tuned


Leave a Reply