Content Filtering

Using the Catch-All Wildcard (*) in URLs

The asterisk symbol has two primary uses in URLs for content filtering.

• Standalone Catch-All Wildcard
  • The " * " (asterisk) symbol when used on its own line is an all-inclusive wildcard which represents all possible entries
  • When used on its own line in allow listed URL patterns, ALL URL patterns are allow listed
  • When used on its own line in blocked URL patterns, ALL URL patterns are blocked, except those that are explicitly allow listed
• In-URL Asterisk Character
  • The " * " (asterisk) symbol when used as part of a URL or in line with a URL is simply a regular asterisk symbol and is interpreted as part of the URL, NOT as a wildcard
  • Note that this is very rarely useful, except in URLs that actually require asterisk symbols, such as https://web.archive.org/web/*/meraki.com

Patterns for Blocking or Allow Listing Specific URLs

Whenever a device on the network accesses a web page, the requested URL is checked against the configured lists to determine if the request will be allowed or blocked.

1. The full URL is evaluated 
"http://www.foo.bar.com/qux/baz/lol?abc=123&true=false" is compared in its entirety and blocked or allowed if it matches an entry in either of those lists.

2. The URL is reduced down to its domain and subsequent parameters, then evaluated
"http://www." (protocol and leading 'www') is removed from the front of the URL

3. The URL will be reduced to only the domain along with its directory structure
The "foo.bar.com/qux/baz/lol?abc=123&true=false" will be reduced to "foo.bar.com/qux/baz/lol" by removing "?abc=123&true=false". 

4. A multi-step evaluation that removes pieces of the directory structure, a level at a time, starting from the last directory.
Directory structure removed and evaluated: "foo.bar.com/qux/baz" is reduced to "foo.bar.com/qux" which is then reduced to "foo.bar.com"

5. Subdomains are removed in another multi-step process.
Subdomains are removed from the left to right, eventually being reduced to the top-level domain ( .COM). In this instance, "foo.bar.com" is reduced to "bar.com" and eventually to ".com"

6. Finally, the matching mechanism will check for the single asterisk character, used as a catch-all wildcard, in both the "Allow" and "Block" lists. 
Step 6 is used to match the lone asterisk character to the "allow" and "block" lists. 

If any of the above steps produces a match, then the request will be blocked or allow listed as appropriate. The allow list always takes precedence over the blocklist, so a request that matches both lists will be allowed. If there is no match, the request is subject to the category filtering settings above.

 

URL Parsing Steps	Current version of URL at this Step	Evaluated/Processed URL(s)
Full URL	http://www.foo.bar.com/qux/baz/lol?abc=123&true=false	http://www.foo.bar.com/qux/baz/lol?abc=123&true=false
URL reduced down to its domain and subsequent parameters	http://www.foo.bar.com/qux/baz/lol?abc=123&true=false	foo.bar.com/qux/baz/lol?abc=123&true=false
Domain along with its directory structure  (Multi-step)	foo.bar.com/qux/baz/lol?abc=123&true=false	foo.bar.com/qux/baz/lol foo.bar.com/qux/baz foo.bar.com/qux foo.bar.com
Sub-Domains Removed (Multi-step)	foo.bar.com	foo.bar.com bar.com .com
Wildcard (asterisk - '*') catch-all	None	*  (lone wildcard asterisk)

Example

In the example above, the specific (longer) URL is allowed because it is the longest match, whereas any other access to foo.bar.com domain will be blocked.

Blocking all Websites Using Content Filtering

An MX Security Appliance can be used to block all web content then configured for specific websites only. This can be specifically important when needing to be in a very controlled environment such as a school. 

1. Navigate to Security & SD-WAN > Configure > Content filtering

2. Place an asterisk (*) in the Blocked URL patterns section

HTTPS filtering

HTTPS requests can also be blocked, but because the URL in an HTTPS request is encrypted, only the domain URL checks will be performed in the following order:

1. www.foo.bar.com
2. foo.bar.com
3. bar.com
4. .com
5. * (the special character for catch-all URL)

 

Example

In the example below all web pages are blocked except for http://meraki.com and https://meraki.com.

Group Policies

Once your Active Directory server settings are entered into Dashboard, you can click Refresh LDAP Groups to populate a list of user groups in your domain. You can then select individual groups and apply configured Group policies to them. For information about configuring Group policies, see the Group policies page.

Content Filtering Rule Priority

There are a number of different ways on the MX to use content filtering to block or allow access to websites. In circumstances where different filtering options contradict one another, the following priority applies (from highest to lowest priority):

1. Blocked and allow listed URL patterns.

2. Content filtering rules applied via Group Policy (using Active Directory or otherwise).

3. Global content filtering rules.

 

Practically speaking, with these rules in mind, consider the following best practices for content filtering design:

• Global content filtering rules should be designed as the "default" network experience.
• Group Policies should be used to create a "custom" network experience for users, which can be made either more or less restrictive than the default.
• URL patterns should be used to append or allow list a specific URL from the configured blocked categories.