Skip to main content

 

Cisco Meraki Documentation

Troubleshooting Remote Targets

Troubleshooting Remote Targets

This document is designed to help you diagnose and resolve common issues encountered with Remote Targets. A Remote Target is a virtual appliance that enables your cloud-based workflows to securely communicate with resources inside your network that do not have direct internet access. It effectively bridges the gap between these on-premise devices and the cloud, allowing them to be seamlessly incorporated into your automation workflows.

Please note that this document focuses only on troubleshooting and does not cover the deployment process of Remote Targets. For information on how to deploy a Remote Target, refer to the Remote Setup and Deployment guide.

Many of the troubleshooting steps detailed here involve the use of standard Docker and docker-compose commands. For further information on Docker diagnostics, you can refer to the official Docker troubleshooting guide.

Before You Begin

Understand Key Terminology

This section defines key terms used throughout the document.

  • Remote Target: Refers to the virtual appliance deployed in your on-premises environment. It acts as a secure communication bridge between your cloud-hosted workflows and network resources that do not have direct internet access.

  • Remote: Refers to the configuration entry created within your Automation tenant (Automation > Targets > Remote Targets > New Remote). This entry defines and manages the settings for a Remote Target and provides the configuration files required to deploy the Remote Target virtual appliance. You must create a Remote before deploying a new Remote Target virtual appliance.

Verify Remote Targets Access Requirements

Review the following prerequisites and considerations for accessing and setting up your Remote Targets. For complete setup procedure, see Remote Setup and Deployment .

Remote Target Console Password Considerations

To ensure a successful setup and secure access to the Remote Target console, your default user password must adhere to the following rules:

  • Length: Minimum of 14 characters.

  • Composition: Must include at least:

    • 1 uppercase letter

    • 1 lowercase letter

    • 1 number

    • 1 special character

  • Restrictions: Must not contain:

    • More than 3 identical characters consecutively (e.g., 'aaa')

    • More than 3 sequential characters (e.g., '123', 'abc')

    • Your username

    • Common dictionary words

For more information, see Remote Setup and Deployment .

SSH Considerations

To generate an SSH key pair, see "Generate an SSH Key Pair" section.

  • Public Key Authentication Only: For security reasons, we recommend that the Remote Target appliance allows SSH access only via public key authentication. Password authentication is not recommended for SSH access.

  • Initial Setup Only: SSH public keys can only be registered during the initial deployment of the Remote Target. You cannot configure or add SSH public keys after the virtual appliance has been deployed. If you need to add an SSH key after deployment, you must delete the virtual appliance and create a new one.

When logging in to your remote target, specify your private key. Enter your passphrase if you specified one during key generation.

ssh -i /path/to/private_key ubuntu@<Remote FQDN or IP>

SSH Considerations Verify Remote Target Service Status

Verify that essential services (core remote, terminal adapter, and web service adapter) are active and running by using the following command on the Remote Target:

ubuntu@xdr-remote-2:~$ docker ps

Expected Output Example:

 CONTAINER ID   IMAGE                                               COMMAND                  CREATED          STATUS          PORTS      NAMES
779c8d7f8ccc   localhost:5000/cisco-sxo/ao-remote:OVA version        "/usr/bin/ao-remote …"   35 minutes ago   Up 35 minutes   8081/tcp   automation-ao-remote-1
465643e8b979   localhost:5000/cisco-sxo/ad-terminal:OVA version     "/usr/bin/activity-w…"   35 minutes ago   Up 35 minutes   8082/tcp   automation-adapter.terminal-1
8c2ed1b6f31c   localhost:5000/cisco-sxo/ad-webservice:OVA version   "/usr/bin/activity-w…"   35 minutes ago   Up 35 minutes   8082/tcp   automation-adapter.web-service-1
Collect Remote Configuration

To troubleshoot issues, review the Remote Target's configuration. The initial configuration is contained within the remotePackage.zip file, which includes information such as credentials used to connect to the Meraki Cloud, along with other data.

The cloud-config.txt file, found on the deployed Remote Target, is derived from this package and provides a clear view of the configured environment variables and network settings. To view this configuration file on the Remote Target:

  1. Switch to the root user:

  sudo su

  1. View the configuration file:

  cat /var/lib/cloud/instance/cloud-config.txt

      This output includes network settings (IP address, gateway, DNS), remote_id, remote_name, MQTT broker details, and proxy settings.

Troubleshooting Remote Target Issues

The first step to any troubleshooting is reviewing logs. The primary log files for the Remote Target services are located at:

  • /var/log/docker-services/ao-remote/service.log

  • /var/log/docker-services/adapter.web-service/service.log

  • /var/log/docker-services/adapter.terminal/service.log

  • For other adapter types, check /var/log/docker-services/adapter.<adapter_name>/service.log.

For in-depth analysis of the Remote Target, collect all relevant platform-related logs from the /var/log/docker-services/... directory and provide a corresponding timeline of the issue.

Onboarding Issues

Unable to Connect with SSH Key

Problem: Unable to connect to the Remote Target using your generated SSH key.

Cause: This indicates that the SSH public key was not successfully registered during the initial Remote Target deployment process.

Resolution:

If you encounter this issue, do the following:

  1. Delete the existing Remote Target.

  2. Deploy a new Remote Target. During the new deployment, ensure that you provide your SSH public key is provided. ( For more information, see "Remote Setup and Deployment " section)

Remote Target "Not Connected" Status (Cloud Connectivity)

Problem:

The Remote Target remains in a “Not Connected” state on the dashboard for more than 10 minutes after deployment.

Cause:

The Remote Target is unable to reach the Meraki cloud.

Resolution:

  1. Review the Remote Target’s service logs. SSH into the remote and examine the core remote service logs:

tail -f /var/log/docker-services/ao-remote/service.log

  1. Verify TCP Connectivity to Meraki Cloud. The Remote Target requires a TCPS (TCP over SSL/TLS) connection to the Meraki Cloud on TCP port 8883. The specific FQDN depends on your cloud region (e.g., North America: us-remote.workflows.meraki.com).

    • Proxy Environments: Note that only SOCKS5 proxy is supported; HTTP proxy is not.

    • Test Connectivity: Use curl to test TCP connectivity (as telnet is not available on the Remote Target). An "Empty reply from server" indicates successful TCP connectivity, even without an HTTP response.

  Example (Direct Connection):

curl -v http://us-remote.workflows.meraki.com:8883
* Trying 52.192.107.103:8883...
* Connected to us-remote.workflows.meraki.com (52.192.107.103) port 8883 (#0)
> GET / HTTP/1.1
> Host: us-remote.workflows.meraki.com:8883
> User-Agent: curl/8.1.2
> Accept: */*
>
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server <<< This indicates that connectivity itself is working
```

Look for output similar to * Connected to us-remote.workflows.meraki.com (...) port 8883 (#0) and curl: (52) Empty reply from server.
 

 Example (SOCKS5 Proxy Connection):

curl -v -x socks5h://proxy.example.com:1080 http://us-remote.workflows.meraki.com:8883
* Uses proxy env variable no_proxy == 'registry.local,localhost,127.0.0.1,::1'
* Trying 173.39.92.228:1080...
* TCP_NODELAY set
* SOCKS5 communication to us-remote.workflows.meraki.com:8883
* SOCKS5 connect to us-remote.workflows.meraki.com:8883 (remotely resolved)
* SOCKS5 request granted.
* Connected to proxy.esl.cisco.com (173.39.92.228) port 1080 (#0)
> GET / HTTP/1.1
> Host: us-remote.workflows.meraki.com:8883
> User-Agent: curl/7.68.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host proxy.esl.cisco.com left intact
curl: (52) Empty reply from server <<< This indicates that connectivity itself is working

Look for * SOCKS5 request granted. and * Connected to proxy.esl.cisco.com (...) port 1080(#0) followed by curl: (52) Empty reply from server.

Replace proxy.example.com:1080 with your SOCKS5 proxy address and port.

Deployment Specific Issues

VMware Deployment Issues

If you encounter problems during VMware deployment, review the Requirements for VMware version and VM specifications for the Remote Target. While the workflows documentation currently does not state vCenter is mandatory, some environments (especially ESXi without vCenter) may benefit from its use for successful deployment.

Remote Target Deployment Issues

For issues specific to the Remote Target appliance itself that are difficult to diagnose, it is often simpler to redeploy a new Remote Target than to perform deep troubleshooting on the existing one.

Before Deleting:

Ensure that any targets currently using the old Remote are migrated to a new Remote. This prevents service disruption. We recommend that you set up a new Remote and migrate targets to it before deleting the old one.

Steps to delete:

  1. Navigate to Automation > Targets > Remote Target in your Meraki dashboard.

  2. Select the Remote and click Delete Remote.

  3. After deleting the Remote from your dashboard, you can then safely stop and delete the corresponding Remote Target appliance in your VMware vCenter.

Once the old Remote Target is removed, proceed to set up a new Remote Target appliance.

Workflow Issues

Problem:

Workflows using a local target through the Remote are not functioning as expected.

Cause:

Connectivity issues between the Remote Target and the on-prem target. This can stem from network connectivity problems, incorrect routing, or IP address conflicts.

Resolution:

  1. Verify network reachability to the target's IP address: Use ping or curl:

ping 1.150.0.30

curl http://1.150.0.30/

  1. Use the following commands on the Remote Target to examine its network setup:

  • ip a: Displays IP addresses and network interfaces.

  • ip route: Shows the routing table.

  • ip nei: Displays the ARP/NDP cache (neighbor table).

       Important Note on Internal Network Addresses: The Remote Target uses internal network addresses (e.g., 10.42.0.0/24 or 10.43.0.0/24).         If these overlap with your customer's network, API access to Remote Target could fail. To prevent this, configure advanced network settings during the         initial Remote setup (via Automate > Target > New Remote) to avoid IP range duplication.

      Remote_Advance_Options.png

  1. Advanced Network Troubleshooting with tcpdump:

     For detailed network troubleshooting, use tcpdump to capture traffic on the Remote VM:   

 sudo tcpdump -i ens192 -w <PCAP filename> src host <Remote IP> and dst host <Target IP>

      Replace <PCAP filename>, <Remote IP>, and <Target IP> with appropriate values.

 

 

  • Was this article helpful?