Systems Manager Logging and Troubleshooting
This article is intended to assist in basic to intermediate troubleshooting of common Systems Manager issues, as well as expose the tools needed to troubleshoot more advanced issues/cases.
Initial Steps
When experiencing an issue with Systems Manager, the first step is usually to take a look at an example client which is exhibiting the behavior. Once you have identified an example device, its logging will describe the issue and provide suggestions on how to fix it.
Dashboard-Side Logging
The primary logging location in SM is the Client Details page, and is always the first/easiest place to look. Here you can find three primary banks of data:
- The client details bank (Magenta)
- Displays current state/stance and vital statistics such as MAC Address, Local/Public IP, profile/app update state, OS Level, Check-in time, etc.
- The Restrictions/Profiles/Apps bank (Cyan)
- The Restrictions table displays a list of Restrictions on a client device, enforced by its profile(s).
- The Profiles table lists all configuration Profiles installed on the client, including how they were installed and whether there is an update pending for them.
- The Apps table lists all Apps Either Installed or Scoped for this device (Apps which are scoped but not installed will be under the 'Missing' Tab).
- The Logging/Data Bank (Yellow)
- The Event log lists each command/response pair that has been transmitted to/received from the client device.
- The Activity log lists the actions which have recently been performed on the device as well as the completion state and any error messaging associated with it. When an error is thrown on one or more of these commands, the error text will be displayed in the status column as a hyperlink, and the link destination will open a popup with the full text of the error, including any associated error codes.
Client-Side Logging
If Dashboard-side logging isn't getting you anywhere, each OS includes logs that are local to the client devices.
It's important to remember that the majority of this logging is only temporarily available, so it's recommended to start capturing logs prior to triggering the broken behavior. Make sure to save/export these logs as soon as they're captured.
iOS
Android Logs
macOS
macOS Meraki Agent logs can be pulled from Dashboard while the device is online by using the Agent logs > fetch logs button. This can be used to debug custom app deployments, script deployments (like shown below), agent upgrades, agent connectivity, and much more.
Agent logs on MacOS can be found at the following location:
/var/log/m_agent.log
Console logs deal with system events, dialog text, errors, status, and other communications. To gather these logs on MacOS, use the "Console" application (/System/Applications/Utilities/Console.app). Click "start" to begin collecting logs, reproduce the issue, then click pause to stop the log stream.
To collect existing logs from a device, a sysdiagnose command must be run:
sudo sysdiagnose
When the sysdiagnose has finished running, a finder window should open at the location of the log file (/private/var/tmp/). Unpacking this archive file will reveal a number of files including the system log file "system_logs.logarchive" which can be opened with the console application.
The primary process whose messages may assist is:
- mdmclient: the MDM Management Daemon on macOS
Filtering can be done on the log file by entering "mdmclient" in the search bar, then selecting "process" as the search method:
Windows
Windows Meraki Agent logs can be pulled from Dashboard while the device is online by using the Agent logs > fetch logs button. This can be used to debug custom app deployments, script deployments (like shown below), agent upgrades, agent connectivity, and much more.
Agent logs on Windows vary by agent version. These logs can be found locally at the following locations:
Agents ≤ 3.6.0:
C:\Windows\temp\m_agent_service.log
Agents ≥ 3.7.0:
C:\ProgramData\Meraki\Systems Manager Agent\Logs\m_agent_service.log
Agents ≥ 4.0.0:
C:\ProgramData\Meraki\Endpoint Agent\Logs\m_agent_service.log
The C:\ProgramData directory is a hidden folder which will only be visible in the file explorer when "Show hidden files and folders" is selcted in file exporer.
System logs are found in the Event Viewer:
Windows Logs/Application and Services/Microsoft/Windows/DeviceManagement-Enterprise-Diagnostics-Provider/Admin
Using Logs to Troubleshoot
The following table describes common error messages organized by OS/Type:
OS(s) | Error Code | Error Text | Reason | Fix | Notes |
iOS/MacOS | Profile Installation Failed: An SSL Error has occurred and a secure connection to the server cannot be made. | Proxy or MITM device is manipulating the payload/traffic containing the payload | Add the client to allow list | ||
iOS | 1000 |
Domain: MCProfileErrorDomain |
Most commonly, the user/owner assigned to the device is not a member of a valid ASM Group | Assign the user to a valid group in ASM | |
iOS | 1006 | Profile could not be decrypted | Device has received an encrypted payload but is incapable of decrypting it | Remove DEP settings, set up the device, upgrade iOS, re-add DEP settings, factory reset client | |
iOS | 2003 | Domain: MCPayloadErrorDomain The field “UserName” is invalid. | EAP authentication information is incomplete for a WiFi Profile | Include a password with the Username in the EAP configuration in the WiFi Profile | This will only effect iOS 9+ Supervised devices |
iOS | 4001 | Domain: MCInstallation Error | Profile installation error general fault | Remove the faulting part of the profile before deployment | This error is generally thrown when a profile cannot be installed on a client but doesn't otherwise have a more specific error message. Examples of this include pushing a Passcode payload while modification of passcode is disallowed and pushing supervised settings to non-supervisied devices. |
iOS | 12021 | Domain:* MCMDMErrorDomain“ScheduleOSUpdate” is not a valid request type. | Device needs to be DEP Supervised to execute an OS Update command | DEP Supervise the client | |
iOS | 12023 | Domain:* MCMDMErrorDomain“ The iTunes Store ID of the application could not be validated. |
Generic 'App License Unavailable' Error message Possible causes:
|
Possible Solutions:
|
|
iOS | 12026 | Domain:* MCMDMErrorDomain“ The app “com.generic.app” is already scheduled for management. | The app is already in the client's installation queue. Generally speaking, this error is safe to ignore | Wait | If the app never installs, look for messages from the installd process |
iOS | 12064 | Domain: MCMDMErrorDomain License for app "X” could not be found. | License for the app is not available on the client | Unscope and rescope the client for this app | This message will only occur with the App configured to use Device License Assignment |
iOS, MacOS | 'Error - Activation Lock Bypass code not found with Apple. Is activation lock already disabled?' | Activation Lock Bypass command was already acknowledged by Apple's servers | Attempt to restart the client device, associate it to wifi, wait 10, restart setup assistant (home button), try again | This doesn't always work, but if these steps fail contact Apple for Activation Lock Bypass assistance | |
iOS | Error - License for app "com.generic.app" could not be found |
|
|
||
MacOS/Windows Agent | curl error 5 |
Couldn't resolve proxy. The given proxy host could not be resolved. Typically this means that something upstream is firewalling the connection |
Open TCP993 Outbound to Dashboard | This error may also mean that there are no remaining licenses on dashboard, so be sure to check to see if the dashboard has hit its license limit. | |
MacOS/Windows Agent | curl error 6 |
Couldn't resolve host. The given remote host was not resolved. Typically this indicates that something is attempting to manipulate the traffic (SSL Intercept Proxy) |
Add the client/traffic to allow list (not in dashboard, on that device whatever it is) | It's important to remember that TCP993 is also the IMAPS port, so when you are looking for what device is manipulating our datastream, also look at any Email Filtering systems or local Anti-Virus systems | |
MacOS/Windows Agent | curl error 7 or curl error 60 | Typically this occurs on config fetch when the client is attempting to contact https://cf.meraki.com | Unblock/enable HTTPS traffic to this url |