ConnectWatch / Filesystem probe setup
Back to connectors

Filesystem probe setup

Core connector

ConnectWatch measures filesystem write/read latency by SSHing into your server and writing a small synthetic test file. No data is transferred from ConnectWatch — the file is created and deleted entirely on your server.

Before you start — on-demand large file tests

The 1 GB on-demand test temporarily writes up to 1 GB to your filesystem. Ensure you have at least 1.1 GB of free space at your probe path before running it. The file is deleted automatically when the test completes.

How permissions work

ConnectWatch runs as the SSH user you configure. It can only write to the path you specify — it has no root access and cannot touch anything outside that directory unless you explicitly grant it.


One-time server setup

Run these commands once on the server you want to monitor. They create an isolated probe user and a dedicated directory that ConnectWatch can write to — and nothing else.

  1. Create a dedicated probe user

    Using a dedicated user keeps ConnectWatch completely isolated from your application accounts.

    # Create a no-login system user for probing
    sudo useradd -m -s /sbin/nologin probeuser
  2. Create and permission the probe directory

    Replace /mnt/efs/connectwatch-probe with the path on your filesystem you want to test. This is the only path ConnectWatch will ever write to.

    # Create probe directory and give ownership to probeuser
    sudo mkdir -p /mnt/efs/connectwatch-probe
    sudo chown probeuser:probeuser /mnt/efs/connectwatch-probe
    sudo chmod 750 /mnt/efs/connectwatch-probe
  3. Generate an SSH key pair

    ConnectWatch uses key-based authentication — no passwords stored on your server.

    # Generate an ed25519 key (no passphrase)
    ssh-keygen -t ed25519 -f /tmp/connectwatch_key -N ""
    
    # Install the public key for probeuser
    sudo mkdir -p /home/probeuser/.ssh
    sudo cp /tmp/connectwatch_key.pub /home/probeuser/.ssh/authorized_keys
    sudo chmod 700 /home/probeuser/.ssh
    sudo chmod 600 /home/probeuser/.ssh/authorized_keys
    sudo chown -R probeuser:probeuser /home/probeuser/.ssh
  4. Copy the private key into ConnectWatch

    Print the private key and paste it into the SSH private key field when adding your environment in ConnectWatch. Delete the temp files after.

    # Print the private key — copy this entire output
    cat /tmp/connectwatch_key
    
    # Delete temp key files when done
    rm /tmp/connectwatch_key /tmp/connectwatch_key.pub
  5. Verify the connection

    Test from your local machine before adding to ConnectWatch to confirm the key works.

    # Test SSH login as probeuser (should succeed with no password)
    ssh -i /tmp/connectwatch_key probeuser@YOUR_SERVER_IP echo "ok"
    
    # Test write permission to the probe path
    ssh -i /tmp/connectwatch_key probeuser@YOUR_SERVER_IP \
      "touch /mnt/efs/connectwatch-probe/test && rm /mnt/efs/connectwatch-probe/test && echo write-ok"

What ConnectWatch can and cannot do

The table below shows the exact permissions granted when using the setup above.

ActionAllowedNotes
Write test files to /mnt/efs/connectwatch-probe/✓ YesThis is the only directory written to
Read back test files✓ YesTo measure read latency
Delete test files after probing✓ YesFiles are always cleaned up automatically
Access files outside the probe path✗ NoFilesystem permissions prevent this
Run commands as root / sudo✗ Noprobeuser has no sudo rights
Read your application data✗ NoNo access outside probe directory
Open a login shell✗ No/sbin/nologin prevents interactive login
Transfer files from ConnectWatch to your server✗ NoTest files are generated from /dev/zero locally on your server

On-demand large file test

Before running the 1 GB test

• Ensure at least 1.1 GB of free space is available at your probe path.
• Run during low-traffic hours — a 1 GB sequential write creates real I/O load. On shared NFS or EFS mounts this can slow other processes while the test runs.
• The test file is deleted automatically on completion or failure.
• ConnectWatch performs a disk space pre-flight check and will abort with a clear error if there is insufficient space.

The test works as follows:

# What ConnectWatch runs on your server (10 × 100MB segments)
dd if=/dev/zero of=/mnt/efs/connectwatch-probe/probe-large-{ts}-{n}.tmp \
   bs=1M count=100 oflag=dsync

# No data is transferred from ConnectWatch — /dev/zero is local to your server
# Each segment is read back and deleted before the next segment starts

Results returned: average write MB/s, average read MB/s, minimum write MB/s per segment, and burst credit exhaustion detection (triggered when a segment's throughput drops below 30% of the average).


Error messages and how to fix them

Error messageCauseFix
Permission denied — check that user "probeuser" has write access to /mnt/efs/... The probe user cannot write to the configured path sudo chown probeuser /mnt/efs/connectwatch-probe && sudo chmod 750 /mnt/efs/connectwatch-probe
No space left on device at /mnt/efs/... Filesystem is full or quota exceeded Free up disk space. The regular probe only needs a few KB; the 1 GB test needs 1.1 GB free.
Path not found: /mnt/efs/... — create the directory first The probe path doesn't exist on the server sudo mkdir -p /mnt/efs/connectwatch-probe && sudo chown probeuser /mnt/efs/connectwatch-probe
Filesystem is mounted read-only at /mnt/efs/... The filesystem was remounted read-only (often after an I/O error) Check dmesg for I/O errors. Remount read-write: sudo mount -o remount,rw /mnt/efs
SSH authentication failed — check that the private key matches authorized_keys Wrong key pasted into ConnectWatch, or authorized_keys was overwritten Re-run the key setup steps above and re-paste the private key in ConnectWatch.
Connection timed out — check host/IP and firewall rules Firewall blocking port 22, wrong IP, or server is unreachable Ensure ConnectWatch's egress IPs are allowed on port 22 in your security group / firewall.
Insufficient disk space — Xmb free, need at least 1126MB Pre-flight check failed before the 1 GB test started Free up space on the filesystem hosting your probe path, then retry.

Terms of service — filesystem probe

Filesystem probe terms

  1. Scope of access. By configuring a filesystem probe, you grant ConnectWatch permission to write, read, and delete synthetic test files (generated from /dev/zero) within the probe path you specify. ConnectWatch will never access, read, modify, or delete any files outside this path.
  2. No data exfiltration. Test files contain only zero bytes generated locally on your server. No data is transferred from ConnectWatch to your server, and no data from your server is transmitted to ConnectWatch beyond probe latency metrics.
  3. Customer responsibility for permissions. You are responsible for configuring the SSH user and probe path with appropriate permissions. ConnectWatch acts only within the permissions you grant. ConnectWatch is not liable for probe failures caused by permission misconfiguration.
  4. Disk space. The regular probe uses approximately 4 KB of temporary disk space per cycle. On-demand large file tests use up to the configured test size (default 1 GB) temporarily. ConnectWatch performs a pre-flight disk space check before large file tests and will abort if insufficient space is detected. You are responsible for ensuring adequate free space at the probe path.
  5. I/O impact. Probe operations — particularly on-demand large file tests — generate real filesystem I/O. On shared filesystems (NFS, EFS, CIFS) this may temporarily affect other processes. You are responsible for scheduling on-demand tests appropriately for your workload.
  6. Key security. You are responsible for protecting the SSH private key used for probe access. If you believe the key has been compromised, revoke it immediately by removing it from authorized_keys on your server and generating a new key pair.
  7. Revocation. You may revoke ConnectWatch's access at any time by removing the probe user's entry from authorized_keys or by deleting the probe user entirely. This does not affect your ConnectWatch subscription.

Questions? Contact support@connectwatch.io