Category Archives: PRTG

Push Notifications With PRTG And Pushover

PushoverIntroduction

I was looking for a way for my PRTG installation to alert me whenever I was away from home.  I initially thought of using an “email to SMS message” service, and started looking into this option.  I was then pointed to an API push service called Pushover.

Setting this up to work with PRTG is amazingly simple.  The instructions and scripts I use are taken from this blog, just cleaned up a bit and with more screenshots.

 

Prerequisites

A Pushover account,

Install Python for Windows onto your PRTG server.  Choose Python 2, not 3.

An Android or Apple device with the Pushover app installed,

store-google   store-apple

 

Pushover Instructions

Once you have created your account and verified your email address, you need to create an application in-order to get an API key.

  1. pushover - 1From your main home page,
  2. Next to Your Applications, click Register An Application,
  3. Fill out the displayed form, my entries were…
    • PRTG
    • Application
    • (blank)
    • This image (click to enlarge) – PRTG-75x75
  4. Tick the Terms Of Service check box and click Create Application.

Once the application has been created, you will be presented with an API Key.  You will need this later, along with your User Key.

 

Device Setup

Depending on your device, download the appropriate app and install it.  Log in to the app, and check the settings to see if you need to change anything, specifically the Edit Quiet Hours.

 

Python Scripts

Log on yo your PRTG server, and install Python.  This is a very straight forward installation…

python install - 1   python install - 2   python install - 3   python install - 4   python install - 5

Once done, we need to create two plain text files into the following folder…

C:\Program Files (x86)\PRTG Network Monitor\Notifications\EXE

 

File 1 – pushover.bat

Create a file called pushover.bat and enter the following code…

C:\Python27\Python "C:\Program Files (x86)\PRTG Network Monitor\Notifications\EXE\pushover.py" %*

Change the listed paths if required.

 

File 2 – pushover.py

Create a file called pushover.py and enter the following code…

import httplib, urllib, sys
 
apiKey = sys.argv[1]
userKey = sys.argv[2]
priority = sys.argv[3]
title = sys.argv[4]
msg = ''
 
for count, arg in enumerate(sys.argv):
    if count > 4:
        msg += arg + '\n'
 
conn = httplib.HTTPSConnection("api.pushover.net:443")
conn.request("POST", "/1/messages.json",
    urllib.urlencode({
        "token": apiKey,
        "user": userKey,
        "message": msg,
        "title": title,
        "priority": priority,
        "retry":"30",
        "expire":"3600"
    }),
    { "Content-type": "application/x-www-form-urlencoded" })
response = conn.getresponse()

 

PRTG Setup

Part one

  1. From your PRTG console,
  2. Click Setup > Account Settings > Notifications,
  3. Click the ‘Add New Notification‘ button on the bottom left,
  4. Add a name for your notification – PRTG – ALERT,
  5. Scroll down and check the Execute Program box,
  6. Choose pushover.bat from the dropdown,
  7. Enter the parameters as follows:

[api-key] [user-key] [priority] “%device – %name : %status” “ALERT/ERROR” “Msg: %message”

Example:  abcd1234efgh5678 ab12cd34ef56gh78 0 “%device – %name : %status” “ALERT/ERROR” “Msg: %message”

(For a list of priority values, check out the Pushover FAQs)

  1. Click Save,
  2. Click Test, and you should receive an alert on your device almost immediately.

 pushover - alarmtest

I have three alerts set up..

  • ALERT/ERROR
  • WARNING
  • ALL OK

 

Part two

Now that we have an alert or three setup, we need to make sure your devices start using them.  For my small setup, I set my notifications at the top level, at the Local Probe…

  1. From the PRTG console, select the Local Probe device (or other device if you want more control),
  2. Select the Notifications tab along the top,
  3. Click Add State Trigger,
  4. From the new trigger then appears, choose the options that bests suits you and your environment.  The no notification drop down should list the alerts you setup above.

prtg notifications - 1

The values I have are shown below…

prtg notifications - 2

 

Errors / Issues / Troubleshooting

When I initially set this up the alerts were not coming though.  In order to troubleshoot this issue, I ran the pushover.py script manually.  Do do this…

  1. Open a command prompt on your PRTG server,
  2. Type the following command into the window…
C:\Python27\Python "C:\Program Files (x86)\PRTG Network Monitor\Notifications\EXE\pushover.py" [api-key] [user-key] 0 "Manual Test" "Ignore This Message"

…making sure to enter your API Key and User Key in the correct locations

  1. You should see no output if everything works OK.

In my case, I got SSL handshake errors.  To fix this, I just visited the Pushover API site (https://api.pushover.net) in order for my server to download the SSL certificate.

 

More Pushover Configuration

I have added another post about using Pushover notifications with my Synology DiskStation and VMware vCenter Alerts.  Read about it here.

NTP Time Service

Introduction

Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems…  NTP is intended to synchronize all participating computers to within a few milliseconds of Coordinated Universal Time (UTC).

Taken from Wikipedia

Making sure you have a reliable and accurate time source can help with mitigating issues on your network.

 

Problem

One of my ESXi hosts was replaced recently, and using my automatic build script I had the new replacement up and running quite quickly.  What my script doesn’t do however, is configure the NTP settings with the host and because of this, it’s internal clock was about 6 hours ahead of the rest of the network.

My PRTG Network Monitoring system is a VM that, via DRS, can move freely amongst my three hosts depending on the current workload.  At 5:00 am the other morning, my monitoring server was moved to the offending host.  This screwed up its monitoring results.

As the VM was migrated to the new host, the time suddenly jumped forward by about 6 hours.  The OS then took over sometime after and jumped back to the correct time.  This time flip-flop caused large gaps in the monitoring results, and left me scratching my head for a while figuring out why.!

 

How to fix the above issues

There were two issues at play here, the host NTP settings were not correct, and the guest VM was set to synchronise it’s time with the host.

 

Configure Host NTP Settings

To enable configure each hosts time setting…

  1. Select your host (if you have more than one, you will need to do this on each one in turn),
  2. Select the Configuration tab along the top of the main section,
  3. Down the left hand menu, near the middle, select Time Configuration,
  4. Click Properties at the top right,
  5. From the first window that appears, make sure the NTP Client Enabled option is selected,
  6. Click Options,
  7. From the General item, choose the middle option, choose Start and stop with host,
  8. Select the NTP Settings item, click Add, and enter the IP address or host name of your chosen NTP host,
  9. Tick the option Restart NTP service to apply changes.

NTP-1

NTP-2   NTP-3

If you don’t have an internal NTP time source, the best option is to use 0.pool.ntp.org1.pool.ntp.org2.pool.ntp.org, and 3.pool.ntp.org.  See the link below for more information, or for geographically local sources…

http://support.ntp.org/bin/view/Servers/NTPPoolServers

 

Disable ‘synchronise guest time with host’

I had my home-lab domain configured to use a specific time source, and to push that out to all the Windows machines.  This is a good idea to configure.  What I had done wrong, was allow the virtual machines guests to have their clocks synchronised with the hosts.  This is generally a bad idea.

  1. Right-click a virtual machine, choose Edit Settings…,
  2. Choose the Options tab along the top,
  3. Select the VMware Tools settings,
  4. Un-tick the Synchronize guest time with host option.

While you are there, it’s also a good idea to tick the other option : Check and upgrade Tools during power cycling.

 

If you have a lot of hosts to change, this is going to be a very tedious task.  Thankfully, there is a script that will handle this for you.

The script below will set both options for you:

  • Line 12: Check and upgrade Tools during power cycling,
  • Line 13: Synchronize guest time with host.

Save the script as a .PS1 file, and execute it within PowerCLI.  Remember to change the first line to enter your vCenter server name or IP address.

Connect-VIServer "ENTER VCENTER SERVER HERE"

# Query for the VM guests
$VMGuests = Get-VM

# Loop through your VM guests, set the VM Tools upgrade checkbox and the Sync Time checkbox to true
ForEach ($VMGuest in $VMGuests) {
    $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $spec.changeVersion = $VMGuest.ExtensionData.Config.ChangeVersion
    $spec.tools = New-Object VMWare.Vim.ToolsConfigInfo
    
    $spec.tools.toolsUpgradePolicy = "upgradeAtPowerCycle"
    $spec.tools.syncTimeWithHost = $false

    # Apply the changes
    $MyVM = Get-View -Id $VMGuest.Id
    $MyVM.ReconfigVM_Task($spec)
}

So, there you have it.  Make sure all your VM hosts and guest are using the correct time, and are synchronised with a reliable time source.

PRTG Network Monitor

Paessler PRTG Network MonitorIntroduction

If you want to monitor your home lab, and the devices connected to it, printers, switches, routers, etc, you can’t go wrong with the monitoring solution from Paessler, called PRTG Network Monitor.

 

Monitoring

The PRTG software has a few licencing levels, one of which is a free 3o sensor licence.  Not a free time limited demo, a free forever licence.

30 sensors might sound like a small amount when you find out what a sesnor is…

The licensing options are based on the number of sensors (not on the number of devices or IPs). We define one (1) sensor as any particular, individual monitoring entity. One sensor monitors one network service, one URL, one network connection, one port of a switch, one NetFlow export stream, one CPU load, one disk drive, etc. Please have a look at the list of supported sensor types. Monitoring one item from this list counts as one sensor.

Taken From http://www.paessler.com/support/faqs#e1913

…however, I find that 30 is enough for what I want to monitor on my network.  There are some sensors that “bundle” more that one metric into them.  I could use more if I had them though  🙂

Network Port Monitoring

For example: monitoring a single network port counts as one sensor, but that port could return several metrics.  The HP switch I use returns the following information: Downtime, Transmitted, Packets, Broadcast Packets, Multicast Packets, CRC Errors, Undersize Packets, Oversize Packets, Fragments, Jabbers, Collisions, Packets <= 64 Byte, Packets 65 – 127 Bytes, Packets 128 – 255 Bytes, Packets 256 – 511 Bytes, Packets 512 – 1023 Bytes, Packets 1024 – 1518 Bytes, Drop Events.

That’s a lot of information from one sensor, but expected for a network port.

ESX Performance Monitoring

Another type of “bundled” sensor is the ESX Performance Monitoring one.  This gives much more useful and varied information: CPU usage, Datastore total ReadLatency, Datastore total WriteLatency, Disk read, Disk usage, Disk write, disk.deviceLatency, disk.kernelLatency, Downtime, Memory active, Memory consumed, Memory consumed, Memory swap used, Network received, Network transmitted, Network usage, Power

This is a varied collection of useful information, all collected in one sensor.  It covers the four main metrics: CPU, Memory, Disk, Network.  All of these metrics are available in graph form too, so you can see the history of this information over various time periods, from as little as the last two hours, up to a year ago.

You don’t even need to see all these metrics either.  If you are not interested in the “Power” metric for example, a few clicks later and it’s gone from all your graphs.  The data is still being collected, just not shown to you.

 

My Home Network Monitoring

I use it on my home network to monitor and graph the throughput of my internet router, my wireless router, and my Synology Diskstation 1813+ NAS.  I also have monitors for my three NUC ESXi hosts.

With all the data, this tool collects, you might think it’s hard to find what you are looking for, or just want a way of displaying the data to make it easier to look at.  For this, PRTG has Network Maps.  Below is my network map for my home network…

PRTGNetworkMap

The data shown in the graphs are updated every 30 seconds or so (depending on your settings).  You can see that one of my ESXi hosts is offline at the moment, and that some of my critical virtual machines are listed.

The network map layout can be in almost any layout you can think of.  It doesn’t have to be as pretty as mine.