By Chase Nebeker
Network troubleshooting can be difficult and time-consuming to narrow down issues as they come up, and at some point, everyone will blame the network. One of the best tools that you can utilize is Wireshark, a free and open-source program. Meraki provides ample opportunity to gather data through packet capture. For a quick rundown of how to get started, refer to the articles below. This guide is going to be diving into some (but not all) moderate to advanced Wireshark filters that can be used to help troubleshoot and narrow down the issue. Many times Wireshark can show the server admin that it is, in fact, NOT a network issue, but an issue where the server simply isn't responding to traffic that it's being sent.
Keep in mind that captures from Meraki equipment aren't always going to display 100% of the packets that pass the device. The device is going to give higher priority to delivering the packets than capturing them. If this is a concern, then it is recommended that you use a port mirror on the switch or use a network tap to capture data. This guide can only scratch the surface of what can be done with Wireshark. If you're troubleshooting an issue, feel free to reach out to Meraki Support, who can help you understand what you might be seeing in the captures.
Before getting started, there are some things that will help when filtering with Wireshark. All Wireshark filters are case sensitive - lowercase. The color of the filter bar lets you know if you're on the right track: green - your filter syntax is correct; yellow - proceed with caution you might get some unexpected results; red - something is not right, it might be wrong syntax or wrong input, no results will be presented. Always remember to press enter after modifying the filter, without pressing enter the filter won't be activated.
As of April 10, 2020, the current version of Wireshark <https://www.wireshark.org/download.html> is 3.2.3. This article is written based on this version.
Setting up a packet capture
Please refer to Packet Capture Overview for more information on capturing traffic from different products. As a best practice, it is recommended to start as close to the client or server as possible. Once you have found traffic patterns that you're interested in, then you can add different hops in between.
Picking the right interface
It is vital to select the best interface to capture traffic on to ensure you are getting the data you are looking for. Below are the interfaces available for each product line.
Internet 1 - Captures all traffic that is going out Internet 1 link. This traffic will be NATed to the IP on that interface.
Internet 2 - Captures all traffic that is going out Internet 2 link. This traffic will be NATed to the IP on that interface.
LAN - Captures traffic that is sent and received on the LAN interface of the MX.
Site-to-Site VPN - Captures all of the Meraki Site-to-Site VPN traffic.
Site-to-Site VPN over Internet 1 - Captures all of the Meraki Site-to-Site VPN traffic that is establish on Internet 1. These two interfaces are useful for determining if SD-WAN preferences are working correctly.
Site-to-Site VPN over Internet 2 - Captures all of the Meraki Site-to-Site VPN traffic that is establish on Internet 2.
Client VPN - Captures all traffic sent and received over the Client VPN interface.
Cellular - Captures all traffic going out of an Integrated or USB Cellular link.
Some interfaces will only show if the corresponding feature is enabled (e.g., Client VPN, Site-to-Site VPN)
Switches will only let you choose a switch and a port to capture from. It's generally best to select the switch and port combo that the device is plugged into.
As a shortcut, the troubleshooting section of a specific port will open the correct combo:
1. Navigate to Switch > Monitor > Switches and select the switch in question.
2. Select the port you wish to capture traffic on.
3. Under the Troubleshooting section select Run a packet capture on this port.
Access Points default to "All Access Points" but should not be used as it is best to choose the specific AP the client is connected to.
Wired - Captures all traffic that is sent to the switch
Wireless - Captures all traffic that is sent on the wireless radio
It is recommended to capture only on the wired interface instead of using the wireless interface. For capturing wireless traffic, please refer to Capturing Wireless Traffic from a Client Machine.
Capture filters are what the dashboard is going to use to search for packets to download. These are not to be confused with Display filters as they use a completely different syntax. The general rule here is, the more simple the better. The capture filter is going to only save traffic that you specify so don't be surprised if the resulting capture is empty. The disadvantage of excluding other traffic is that you might miss something that might be going on. However, the disadvantage can also be an advantage as you won't have too many frames to investigate and a lot of noise isn't going to be getting in your way. The fewer packets that are captured means that you can capture longer.
Generally, it is recommended that captures on the LAN side of the MX should use the computer's IP address. Captures on the Internet interface should use the destination Public IP address, as everything is going to be NATed to the IP of the MX's uplink. For everything else, it's just to leave it blank and take a look at in Wireshark. Some example filters can be found below:
host 10.92.182.6 - will capture all data to and from the computer.
host 184.108.40.206 - will capture traffic going to the Google DNS server 220.127.116.11.
ether host 00:18:0a:aa:bb:cc - will only capture for a specific mac. This will not work on interfaces where traffic has been NATed like NAT mode SSID or an Internet interface.
These will be used within Wireshark to sort through traffic.
A lot of the filters that we're going to use are based on just a couple of protocol filters.
tcp - will only display TCP packets
udp - will only display UDP packets
icmp - will only display ICMP (ping) packets
dhcp - will display DHCP packets (if you are using an old version of Wireshark you'll need to use bootp)
dns - will display DNS packets
Both tcp and udp can be extended even more to include the particular ports that are being used to communicate.
tcp.port == 80 - this will display un-encrypted TCP traffic on port 80.
tcp.port == 443 - this will only show encrypted TCP traffic using port 443.
udp.port == 53 - another way of specifying DNS traffic, this will filter off of DNS's use of UDP port 53.
One of the most common filters you're going to start with is looking at what a particular IP address is communicating with IP.
ip.addr == 10.92.182.6 - will display frames that have 10.92.182.6 in either the source or destination field.
When you're looking at communication between devices on the same vlan you can use filters using the mac address of the device. Be aware that when traffic passes a layer 3 boundary, the mac address will change. With this in mind, when you're capturing on the LAN, you usually won't have a problem with the filters, but that might not always be the case.
eth.addr == 00:18:0a:aa:bb:cc - will display frames that have 00:18:0a:aa:bb:cc in the source or destination field.
At this point, we're ready to combine the protocol and device filters to get more granular data with logical operators.
and - this will combine two (or more) filters together, which will only display results if all of the statements are true.
ip.addr == 10.92.182.6 and dns - will only show the host 10.92.182.6 and it's DNS traffic.
or - this will combine two (or more) filters together, displaying results if either one of the statements are true.
ip.addr == 10.92.182.6 or ip.addr == 172.16.7.7 - is going to display both 10.92.182.6 as well as 172.16.7.7
not - this will exclude specific elements from the filter.
not ip.addr == 172.16.7.7 - is going to exclude all traffic that has an IP of 172.16.7.7
() - parentheses can be used to combine elements together. Just like in math, the order of operations matters. () allows using both and with or in the same statement.
(ip.addr == 172.16.7.42 or ip.addr == 172.16.7.7) and dns - is going to show dns traffic for two IPs 172.16.7.42 OR 172.16.7.7
(ip.addr == 172.16.7.42 and dns) or (172.16.7.7 and icmp) - here the parentheses are being used to combine two completely different filters. DNS traffic for 172.16.7.42 as well as icmp traffic for 172.16.7.7.
Here's an example of finding communication between a computer and captive.apple.com. First, you see that the computer is performing a DNS query for captive.apple.com with a reply from the DNS server. That is covered by the dns portion. The filter tcp.port == 80 and ip.addr == 18.104.22.168 is going to find everything on TCP port 80 going to the IP of 22.214.171.124.
Tips and tricks
When filtering for web traffic be sure to check out the article Using Chrome Devtools with Wireshark, as it will make it really easy to know what port is being used by the computer to communicate with the webserver. Searching for the specific source port will immediately show the tcp connection we want to investigate.
When troubleshooting DHCP problems use the filter. Because of the way that DHCP works a lot of the time, you will not see all of the traffic if you just search for the mac address and dhcp. This method you can see all of the attempts for a client to get an IP. Remember for DHCP we want to find DORA - Discover, Offer, Request, and Ack.
dhcp.hw.mac_addr == a4:83:e7:c9:37:cd
In the packet details, if you find something that you want to filter for, you can right-click and select "Apply as Filter > Selected." Just like that, we've created our DHCP filter. Or I could have chosen the Transaction ID to find only that particular exchange. Notice in the info column it lists the Transaction ID. The DORA all has the same ID.
Right-clicking on a packet will allow you to Follow the TCP Stream. This will show only the particular TCP connection.
If you're looking for DNS queries that aren't getting responded to, you might try the following advanced filter. As Wireshark keeps track of which frame a DNS reply comes in on, this filter uses the lack of a recorded reply (!dns.response_in) combined with only looking for DNS queries (dns.flags.response == 0) that are only UDP port 53 (dns).
!dns.response_in and dns.flags.response == 0 and dns
Not officially endorsed by Cisco Meraki.
http://packetlife.net/media/library/...ay_Filters.pdf is a handy cheat sheet for other display filters that can be used.
- http://packetlife.net/media/library/12/tcpdump.pdf is a handy cheat sheet for other capture filters that can be used.
- Sharkfest is an annual Wireshark conference with advanced instruction sessions on using Wireshark. Many of the examples here are explained much more in-depth in this video.