Pattoo Agents¶
pattoo
agents collect IoT data for a centralized pattoo
server.
Visit the Pattoo Agents GitHub site to see the code.
Introduction¶
General information about the project, including the the prerequisite steps to get it operational on your system.
About Pattoo¶
pattoo
allows you to use your web browser to chart your organization’s constantly changing data.
It was inspired by the need to collect and visualize data from various DevOps, network, industrial PLC controllers, electro-mechanical and enterprise systems on a single web dashboard.
This data is collected by pattoo
agents. There are standard agents for:
- Linux
- SNMP
- Modbus TCP
- Bacnet/IP
- OPC UA
With programming skill, you can create your own custom agents if needed.
Operational Overview¶
pattoo
has a number of inter-related components. You can see how they all work together on the pattoo web page.
The Palisadoes Foundation¶
pattoo
is based on the original infoset
code created by the Palisadoes Foundation as part of its annual Calico Challenge program. Calico provides paid summer internships for Jamaican university students to work on selected open source projects. They are mentored by software professionals and receive stipends based on the completion of predefined milestones. Calico was started in 2015.
Basic Installation¶
This section covers some key steps to get you started.
Prerequisites¶
There are some software components that need to be installed prior to starting.
- Install the prerequisite packages for the
easysnmp
python pip package. Instructions can be found here. pattoo
only runs on Python 3.6 or higher
Let’s install the software.
Installation¶
Follow these steps.
Install
git
on your system.Select and create the parent directory in which you want to install
pattoo-agents
.$ mkdir -p /installation/parent/directory $ cd /installation/parent/directory
Clone the repository to the parent directory using the
git clone
command. You can also choose to downloading and unzip the file in the parent directory. The repository can be found at: https://github.com/PalisadoesFoundation/pattoo-agents$ cd /installation/parent/directory $ git clone https://github.com/PalisadoesFoundation/pattoo-agents.git
Enter the
/installation/parent/directory/pattoo-agents
directory with thepattoo-agents
files.Install the required packages using the
pip_requirements
document in thepattoo-agents
root directory$ pip3 install --user --requirement pip_requirements.txt
Use the Configuration Guide to create a working configuration.
Follow the configuration steps for each daemon as explained in the Agent Documentation.
Configuring systemd Daemons¶
You can also setup all the pattoo-agents
agents as system daemons by executing the setup/systemd/bin/install_systemd.py
script.
You have to specify a --config_dir
defining the configuration file directory.
Note The daemons are not enabled or started by default. You will have to do this separately using the systemctl
command after running the script.
$ sudo setup/systemd/bin/install_systemd.py --config_dir ~/GitHub/pattoo-agents/etc
SUCCESS! You are now able to start/stop and enable/disable the following systemd services:
pattoo_agent_os_spoked.service
pattoo_agent_snmpd.service
pattoo_agent_os_autonomousd.service
pattoo_agent_os_hubd.service
$
Configuration Guide¶
After installation, you will need to create a configuration file in a directory dedicated to pattoo
.
Setting the Configuration Directory Location¶
You must first set the location of the configuration directory by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo
applications will read the configuration files located in this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running pattoo
agent daemons or scripts.
Configuration Options¶
There are two ways to configure pattoo
. These are the:
- Quick Method
- Expert Method
Quick Method¶
Use the quick method if you are new to pattoo
.
Run the setup/configure.py
script. It will prompt you for all configuration parameters. The defaults should be sufficient in most cases.
Here’s the command to run:
setup/configure.py
Next Steps:
- Run the installation script next as outlined in the Basic Installation guide.
- You will now need to configure each agent individually. See the Agent Documentation file for details on how to configure each type of agent.
Expert Method¶
This section goes into configuration parameters in great detail.
Setting the Configuration Directory Location¶
You must first set the location of the configuration directory by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo
applications will read the configuration files located in this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running pattoo
agent daemons or scripts.
Copy the Template to Your Configuration Directory¶
You can create your first pattoo.yaml
configuration file by copying the template file in the examples/etc
directory to the PATTOO_CONFIGDIR
location.
NOTE: If a /path/to/configuration/directory/pattoo.yaml
file already exists in the directory then skip this step and edit the file according to the steps in following sections.
$ cp examples/etc/pattoo.yaml.template \
/path/to/configuration/directory/pattoo.yaml
The next step is to edit the contents of pattoo.yaml
Edit Your Configuration¶
Take some time to read up on YAML
formatted files if you are not familiar with them. A background knowledge is always helpful.
The pattoo.yaml
file created from the template will have sections that you will need to edit with custom values. Don’t worry, these sections are easily identifiable as they all start with PATTOO_
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items (if applicable).
pattoo:
log_level: debug
log_directory: PATTOO_LOG_DIRECTORY
cache_directory: PATTOO_CACHE_DIRECTORY
daemon_directory: PATTOO_DAEMON_DIRECTORY
system_daemon_directory: PATTOO_SYSTEM_DAEMON_DIRECTORY
language: en
pattoo_agent_api:
ip_address: 192.168.1.100
ip_bind_port: 20201
This table outlines the purpose of each configuration parameter
Section | Config Options | Description |
---|---|---|
pattoo |
This section defines the locations of key directories for both operation and troubleshooting | |
log_directory |
Path to logging directory. Make sure the username running the daemons have RW access to files there. | |
log_level |
Default level of logging. debug is best for troubleshooting. |
|
cache_directory |
Directory of unsuccessful data posts to pattoo |
|
daemon_directory |
Directory used to store daemon related data that needs to be maintained between reboots | |
system_daemon_directory |
Directory used to store daemon related data that should be deleted between reboots. This should only be configured if you are running pattoo daemons as systemd daemons. The systemd daemon installation procedure automatically adjusts this configuration. This parameter defaults to the daemon_directory value if it is not configured. |
|
language |
Language spoken by the human users of pattoo . Defaults to en (English) |
|
pattoo_agent_api |
This section provides information needed by pattoo agent clients when contacting the pattoo server |
|
ip_address |
IP address of remote pattoo server |
|
ip_bind_port |
Port of remote pattoo server accepting agent data. Default 20201. |
Agent Configuration¶
You will now need to configure each agent individually. See the Agent Documentation file for details on how to configure each type of agent.
Configuring systemd Daemons¶
You can also setup all the pattoo
related daemons located in this GitHub repository as system daemons by executing the setup/systemd/bin/install_systemd.py
script.
The script requires you to specify the following parameters. Make sure you have a username and group created for running your pattoo
services.
usage: install_systemd.py [-h] -f CONFIG_DIR -u USERNAME -g GROUP
optional arguments:
-h, --help show this help message and exit
-f CONFIG_DIR, --config_dir CONFIG_DIR
Directory where the pattoo configuration files will be located
-u USERNAME, --username USERNAME
Username that will run the daemon
-g GROUP, --group GROUP
User group to which username belongs
Note The daemons are not enabled or started by default. You will have to do this separately using the systemctl
command after running the script.
$ sudo setup/systemd/bin/install_systemd.py --user pattoo --group pattoo --config_dir /etc/pattoo
SUCCESS! You are now able to start/stop and enable/disable the following systemd services:
pattoo_api_agentd.service
pattoo_apid.service
pattoo_ingesterd.service
$
Backup and Restoration¶
Always take precautions. Backup your data as you’ll never know when you’ll need to restore it.
Backup¶
It is strongly advised that you backup your agents to protect you in the event of catastrophe.
The following directories need to be saved periodically.
- The
PATTOO_CONFIGDIR
directory which contains your configuration - The
daemon_directory
location defined in your configuration. This area stores important authentication information. - The
pattoo-agents
directory which contains your source code.
We’ll discuss data restoration next.
Restoration¶
It’s important to follow these steps in this order when restoring pattoo-agents
after a disaster.
- FIRST make sure all the
pattoo
agents are stopped. - SECOND restore the contents of the
daemon_directory
location defined in your configuration. This area stores important authentication information. - Restore the
PATTOO_CONFIGDIR
directory which contains your configuration - Restore
pattoo-agents
directory which contains your source code.
You should now be able to restart your agents without issue.
Periodic Jobs¶
You will need to configure some jobs to improve pattoo
performance and troubleshooting.
Logrotate Configuration¶
The default pattoo
debug logging mode can quickly create large logging files. The logrotate utility can automatically compress and archive them.
- Copy the the
examples/logrotate.d/pattoo
file to the/etc/logrotate.d
directory. - Edit the file path accordingly.
Read up on the logrotate utility if you are not familiar with it. The documentation is easy to follow.
Agent Setup¶
How to get the daemons running to collect data.
Agent Documentation¶
pattoo
comes with a number of standard agents, but you can also create your own custom agents to meet your needs. Both approaches are described here.
pattoo
Standard Agents¶
Here is a description of currently supported pattoo
agents.
Agent | Description | Documenatation |
---|---|---|
pattoo_agent_modbustcpd |
Python3 based daemon that polls remote ip_devices for Modbus data over TCP. |
Documentation can be found here. Pattoo ModbusTCP Agent |
pattoo_agent_os_spoked |
Python3 based daemon that presents pattoo data via a web API URL. This data can be regularly polled from a central server |
Documentation can be found here. Pattoo Hub and Spoke Operating System Agents |
pattoo_agent_os_hubd |
Python3 based daemon that polls pattoo_agent_os_spoked APIs for data. |
Documentation can be found here. Pattoo Hub and Spoke Operating System Agents |
pattoo_agent_os_autonomousd |
Python3 based daemon that posts pattoo to a central server. |
Documentation can be found here. Pattoo Operating System Autonomous Agent |
pattoo_agent_snmpd |
Python3 based daemon that polls remote ip_devices for SNMP data. |
Documentation can be found here. Pattoo SNMP Agents |
pattoo_agent_snmp_ifmibd |
Python3 based daemon that polls remote ip_devices for SNMP ifMIB data. |
Documentation can be found here. Pattoo SNMP IfMIB Agent For Network Devices |
Creating Custom Agents¶
Please visit the Pattoo Shared documentation site to see how it is done.
Pattoo Operating System Autonomous Agent¶
pattoo_agent_os_autonomousd
gathers performance data from the operating system on which it is running and reports it to the pattoo
server.
The pattoo_agent_os_autonomousd
has a number of advantages over using a combination of pattoo_agent_os_hubd
and pattoo_agent_os_spoked
.
pattoo_agent_os_autonomousd
can the used where the remote client is allowed to initiate connections to thepattoo
server, but not vice versa.- Many more
pattoo_agent_os_autonomousd
clients can be supported as the centralpattoo_agent_os_hubd
daemon can get overloaded if it needs to poll a large number of remote devices.
If this describes your needs, then continue reading!
Installation¶
These steps outline what needs to be done to get pattoo_agent_os_autonomousd
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_os_autonomousd
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_os_autonomousd.yaml
configuration file. Details on how to do this follow. - Start the desired daemons using the commands below. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_os_autonomousd
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_os_autonomousd
client will read its own pattoo_agent_os_autonomousd.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_os_autonomousd.yaml
¶
Let’s get started on configuring pattoo_agent_os_autonomousd.yaml
.
pattoo_agent_os_autonomousd
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_os_autonomousd:
polling_interval: 300
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_os_autonomousd |
|||
polling_interval |
The pattoo_agent_os_autonomousd will report to the pattoo server every polling_interval seconds |
Polling¶
Use pattoo_agent_os_autonomousd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_os_autonomousd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_os_autonomousd.py --help
usage: pattoo_agent_os_autonomousd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_os_autonomousd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo Hub and Spoke Operating System Agents¶
The pattoo_agent_os_hubd
and pattoo_agent_os_spoked
daemons operate together to report on system performance.
- The
pattoo_agent_os_spoked
runs on a remote server where it provides system performance data on a simple web page. - The
pattoo_agent_os_hubd
polls one or morepattoo_agent_os_spoked
enabled devices for data and reports this to thepattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_os_hubd
and pattoo_agent_os_spoked
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_os_hubd
andpattoo_agent_os_spoked
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_os_hubd.yaml
and apattoo_agent_os_spoked.yaml
configuration file to manage each daemon. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_os_hubd
and pattoo_agent_os_spoked
are standard pattoo
agent and need their configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_os_hubd
and pattoo_agent_os_spoked
clients will read respective pattoo_agent_os_hubd.yaml
and pattoo_agent_os_spoked.yaml
configuration files located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring the Hub Daemon¶
The pattoo_agent_os_spoked
is configured using the pattoo_agent_os_spoked.yaml
file. Let’s see how it is done.
pattoo_agent_os_spoked Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_os_spoked:
ip_listen_address: 0.0.0.0
ip_bind_port: 5000
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Config Options | Description |
---|---|---|
pattoo_agent_os_spoked |
Note: Only required for devices running pattoo_agent_os_spoked |
|
ip_listen_address |
IP address on which the API server will listen. Setting this to 0.0.0.0 will make it listen on all IPv4 addresses. Setting to "0::" will make it listen on all IPv6 configured interfaces. It will not listen on IPv4 and IPv6 addresses simultaneously. You must quote all IPv6 addresses. The default value is 0.0.0.0 |
|
ip_bind_port |
TCP port on which the API will listen |
Operating the Spoke Daemon¶
The pattoo_agent_os_spoked
creates a web page on the device it runs to report on the device’s operating status.
You will need a pattoo_agent_os_spoked.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_os_spoked.py --help
usage: pattoo_agent_os_spoked.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_os_spoked
daemon to start at boot.
Testing¶
If you are running pattoo_agent_os_spoked
on your local system, then you can test it by pointing your browser to http://localhost:5000/pattoo-agent-os/300
to view the system data. In this case 300
is a reference to the polling interval of the polling device. On a Linux system you should be able to see the results by using this command curl http://localhost:5000/pattoo-agent-os/300 | json_pp
or curl http://localhost:5000/pattoo-agent-os/300
if you don’t have JSON Pretty Print installed.
Configuring the Hub
Daemon¶
The pattoo_agent_os_hubd
is configured using the pattoo_agent_os_hubd.yaml
file. Let’s see how it is done.
pattoo_agent_os_hubd Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_os_hubd:
ip_devices:
- ip_address: 127.0.0.1
ip_bind_port: 5000
- ip_address: 127.0.0.2
ip_bind_port: 5000
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_os_hubd |
Note: Only required for devices running pattoo_agent_os_hubd |
||
ip_devices |
Sub-Section providing a list of IP addresses or hostnames running pattoo_agent_os_spoked that need to be polled for data. You must specify an ip_address and TCP ip_bind_port for each of these devices. |
||
ip_address |
The IP adrress of the remote ip_device . |
||
bind_port |
The TCP port on which the remote ip_device is listening. |
Polling From Hubs to Spokes¶
Use pattoo_agent_os_hubd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_os_hubd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_os_hubd.py --help
usage: pattoo_agent_os_hubd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_os_hubd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo SNMP Agents¶
pattoo_agent_snmpd
polls data on any SNMP enabled system and reports it to the pattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_snmpd
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_snmpd
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_snmpd.yaml
configuration file. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_snmpd
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_snmpd
client will read its own pattoo_agent_snmpd.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_snmpd.yaml
¶
Let’s get started on configuring pattoo_agent_snmpd.yaml
.
pattoo_agent_snmpd
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_snmpd:
polling_interval: 300
polling_groups:
- group_name: TEST 1
ip_devices:
- ip.address.of.device1
- ip.address.of.device2
oids:
- address: .1.3.6.1.2.1.2.2.1.10
multiplier: 8
- address: .1.3.6.1.2.1.2.2.1.16
multiplier: 8
- group_name: TEST 2
ip_devices:
- ip.address.of.device3
- ip.address.of.device4
oids:
- address: .1.3.6.1.2.1.2.2.1.10
multiplier: 8
- address: .1.3.6.1.2.1.2.2.1.16
multiplier: 8
auth_groups:
- group_name: CISCO
snmp_authpassword: null
snmp_authprotocol: null
snmp_community: public
snmp_port: 161
snmp_privpassword: null
snmp_privprotocol: null
snmp_secname: null
snmp_version: 2
ip_devices:
- ip.address.of.device1
- ip.address.of.device2
- group_name: Juniper
snmp_authpassword: null
snmp_authprotocol: null
snmp_community: notpublic
snmp_port: 161
snmp_privpassword: null
snmp_privprotocol: null
snmp_secname: null
snmp_version: 2
ip_devices:
- ip.address.of.device3
- ip.address.of.device4
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_snmpd: |
|||
polling_interval |
The pattoo_agent_snmpd will report to the pattoo server every polling_interval seconds |
||
polling_groups: |
List of groupings of ip_devices that need data from a shared set of SNMP OIDs. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
group_name: |
Unique name for a group of ip_devices that share the same SNMP parameters |
||
ip_devices: |
List of ip_devices to poll for OID data |
||
oids: |
OIDs to poll for data from for the ip_devices . Each address must be an OID. The multiplier is the value by which the polled data result must be multiplied. This is useful in converting byte values to bits. The default multiplier is 1. |
||
auth_groups: |
List of groupings of ip_devices that share SNMP authentication parameters |
||
group_name: |
Unique name for a group of ip_devices that share the same SNMP parameters. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
snmp_authpassword: |
SNMPv3 authpassword | ||
snmp_authprotocol: |
SNMPv3 authprotocol | ||
snmp_community: |
SNMPv2 community string | ||
snmp_port: |
SNMP used by ip_devices |
||
snmp_privpassword: |
SNMPv3 privpassword | ||
snmp_privprotocol: |
SNMPv3 privprotocol | ||
snmp_secname: |
SNMPv3 secname | ||
snmp_version: |
SNMP version | ||
ip_devices: |
List of ip_addresses or hostnmae to poll |
Polling¶
Use pattoo_agent_snmpd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_snmpd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_snmpd.py --help
usage: pattoo_agent_snmpd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_snmpd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo SNMP IfMIB Agent For Network Devices¶
pattoo_agent_snmp_ifmibd
polls SNMP IfMIB data from SNMP enabled systems and reports it to the pattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_snmp_ifmibd
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_snmp_ifmibd
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_snmp_ifmibd.yaml
configuration file. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_snmp_ifmibd
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_snmp_ifmibd
client will read its own pattoo_agent_snmp_ifmibd.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_snmp_ifmibd.yaml
¶
Let’s get started on configuring pattoo_agent_snmp_ifmibd.yaml
.
pattoo_agent_snmp_ifmibd
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_snmp_ifmibd:
polling_interval: 300
polling_groups:
- group_name: TEST 1
ip_devices:
- ip.address.of.device1
- ip.address.of.device2
oids:
- address: .1.3.6.1.2.1.2.2.1.10
multiplier: 8
- address: .1.3.6.1.2.1.2.2.1.16
multiplier: 8
- group_name: TEST 2
ip_devices:
- ip.address.of.device3
- ip.address.of.device4
oids:
- address: .1.3.6.1.2.1.2.2.1.10
multiplier: 8
- address: .1.3.6.1.2.1.2.2.1.16
multiplier: 8
auth_groups:
- group_name: CISCO
snmp_authpassword: null
snmp_authprotocol: null
snmp_community: public
snmp_port: 161
snmp_privpassword: null
snmp_privprotocol: null
snmp_secname: null
snmp_version: 2
ip_devices:
- ip.address.of.device1
- ip.address.of.device2
- group_name: Juniper
snmp_authpassword: null
snmp_authprotocol: null
snmp_community: notpublic
snmp_port: 161
snmp_privpassword: null
snmp_privprotocol: null
snmp_secname: null
snmp_version: 2
ip_devices:
- ip.address.of.device3
- ip.address.of.device4
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_snmp_ifmibd: |
|||
polling_interval |
The pattoo_agent_snmp_ifmibd will report to the pattoo server every polling_interval seconds |
||
polling_groups: |
List of groupings of ip_devices that need data from a shared set of SNMP OIDs. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
group_name: |
Unique name for a group of ip_devices that share the same SNMP parameters |
||
ip_devices: |
List of ip_devices to poll for OID data |
||
oids: |
OIDs to poll for data from for the ip_devices . Each address must be an OID. The multiplier is the value by which the polled data result must be multiplied. This is useful in converting byte values to bits. The default multiplier is 1. |
||
auth_groups: |
List of groupings of ip_devices that share SNMP authentication parameters |
||
group_name: |
Unique name for a group of ip_devices that share the same SNMP parameters. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
snmp_authpassword: |
SNMPv3 authpassword | ||
snmp_authprotocol: |
SNMPv3 authprotocol | ||
snmp_community: |
SNMPv2 community string | ||
snmp_port: |
SNMP used by ip_devices |
||
snmp_privpassword: |
SNMPv3 privpassword | ||
snmp_privprotocol: |
SNMPv3 privprotocol | ||
snmp_secname: |
SNMPv3 secname | ||
snmp_version: |
SNMP version | ||
ip_devices: |
List of ip_addresses or hostnmae to poll |
Polling¶
Use pattoo_agent_snmp_ifmibd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_snmp_ifmibd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_snmp_ifmibd.py --help
usage: pattoo_agent_snmp_ifmibd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_snmp_ifmibd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo BACnet/IP Agents¶
pattoo_agent_bacnetipd
polls BACnet Analog Value
data from BACnetIP enabled systems and reports it to the pattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_bacnetipd
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_bacnetipd
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_bacnetipd.yaml
configuration file. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_bacnetipd
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_bacnetipd
client will read its own pattoo_agent_bacnetipd.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_bacnetipd.yaml
¶
Let’s get started on configuring pattoo_agent_bacnetipd.yaml
.
pattoo_agent_bacnetipd
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_bacnetipd:
polling_interval: 300
polling_groups:
- group_name: GROUP 1
ip_devices:
- ip.address.of.device1
- ip.address.of.device2
points:
- address: 162
- address: 181
- address: 1
- address: 2
- address: 3
- group_name: GROUP 2
ip_devices:
- ip.address.of.device3
- ip.address.of.device4
points:
- address: 134
multiplier: 8
- address: 136
multiplier: 10
- address: 144
- address: 158
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_bacnetipd: |
|||
polling_interval |
The pattoo_agent_bacnetipd will report to the pattoo server every polling_interval seconds |
||
polling_groups: |
List of groupings of ip_devices that need data from a shared set of BACnet points (For example the same manufacturer’s make and model). Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
group_name: |
Unique name for a group of ip_devices that share the same BACnet parameters |
||
ip_devices: |
List of ip_devices to poll for data |
||
points: |
BACnet Analog Value point to poll for data from for the ip_devices . Each address must be a BACnet point. The multiplier is the value by which the polled data result must be multiplied. This is useful in converting byte values to bits. The default multiplier is 1. |
Polling¶
Use pattoo_agent_bacnetipd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_bacnetipd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_bacnetipd.py --help
usage: pattoo_agent_bacnetipd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_bacnetipd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo ModbusTCP Agent¶
pattoo_agent_modbustcpd
polls data from ModbusTCP enabled systems and reports it to the pattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_modbustcpd
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_modbustcpd
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_modbustcpd.yaml
configuration file. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_modbustcpd
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_modbustcpd
client will read its own pattoo_agent_modbustcpd.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_modbustcpd.yaml
¶
Let’s get started on configuring pattoo_agent_modbustcpd.yaml
.
pattoo_agent_modbustcpd
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_modbustcpd:
polling_interval: 300
polling_groups:
- group_name: TEST 1
ip_devices:
- test1.modbus.tcp.device.net
input_registers:
- address: 30123
multiplier: 1
- 30789
multiplier: 1
holding_registers:
- address: 40123
multiplier: 1
- address: 40456
multiplier: 1
unit: 0
- group_name: TEST 2
ip_devices:
- test2.modbus.tcp.device.net
input_registers:
- 30387
- 30388
holding_registers:
- 40123
- 40456
unit: 0
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_modbustcpd: |
|||
polling_interval |
The pattoo_agent_modbustcpd will report to the pattoo server every polling_interval seconds |
||
polling_groups: |
List of groupings of ip_devices that need data from a shared set of Modbus registers |
||
group_name: |
Unique name for a group of ip_devices that share the same Modbus parameters. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
ip_devices: |
List of ip_devices to poll for data |
||
input_registers: |
List of Modbus input registers that we need data from for the ip_devices . Each address must be an OID. The multiplier is the value by which the polled data result must be multiplied. The default multiplier is 1. |
||
holding_registers: |
List of Modbus holding registers that we need data from for the ip_devices . Each address must be an OID. The multiplier is the value by which the polled data result must be multiplied. The default multiplier is 1. |
||
unit: |
Modbus unit number to poll. If not present or blank, the default is ‘0’ |
Polling¶
Use pattoo_agent_modbustcpd
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_modbustcpd.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_modbustcpd.py --help
usage: pattoo_agent_modbustcpd.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_modbustcpd
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Pattoo OPC UA Agents¶
pattoo_agent_opcuad
polls Analog Value
data from OPC UA enabled systems and reports it to the pattoo
server.
Installation¶
These steps outline what needs to be done to get pattoo_agent_opcuad
working.
- Follow the installation steps in the Basic Installation file.
- Configure the
pattoo.yaml
configuration file following the steps in Configuration Guide. This file tellspattoo_agent_opcuad
, and all other agents, how to communicate with thepattoo
server. - Create a
pattoo_agent_opcuad.yaml
configuration file. Details on how to do this follow. - Start the desired daemons as explained in sections to follow. You may want to make these
systemd
daemons, if so follow the steps in the Basic Installation file.
Setting the Configuration Directory Location¶
pattoo_agent_opcuad
is a standard pattoo
agent and needs its configuration directory defined by using the PATTOO_CONFIGDIR
environmental variable. Here is how to do this from the Linux command line:
$ export PATTOO_CONFIGDIR=/path/to/configuration/directory
pattoo_agent_opcuad
client will read its own pattoo_agent_opcuad.yaml
configuration file located this directory when PATTOO_CONFIGDIR
is set.
You can automatically set this variable each time you log in by adding these lines to your ~/.bash_profile
file.
export PATTOO_CONFIGDIR=/path/to/configuration/directory
Make sure that files in this directory are readable by the user that will be running standard pattoo
agent daemons or scripts.
Configuring pattoo_agent_opcuad.yaml
¶
Let’s get started on configuring pattoo_agent_opcuad.yaml
.
pattoo_agent_opcuad
Section¶
Here is a sample of what should be added. An explanation follows.
NOTE: The indentations in the YAML configuration are important. Make sure indentations line up. Dashes ‘-‘ indicate one item in a list of items.
pattoo_agent_opcuad:
polling_interval: 300
polling_groups:
- group_name: GROUP 1
ip_target: server-01.opcua.net
ip_port: 4840
username: opcua_username
password: opcua_password
nodes:
- address: ns=1;s=[OPCUA_SERVER_1]DischargehAirTemp.PV
- group_name: GROUP 2
ip_target: server-02.opcua.net
ip_port: 4840
username: opcua_username
password: opcua_password
nodes:
- address: ns=1;s=[OPCUA_SERVER_2]DischargehAirTemp.PV
Configuration Explanation¶
This table outlines the purpose of each configuration parameter
Section | Sub-Section | Config Options | Description |
---|---|---|---|
pattoo_agent_opcuad: |
|||
polling_interval |
The pattoo_agent_opcuad will report to the pattoo server every polling_interval seconds |
||
polling_groups: |
List of groupings of ip_devices that need data from a shared set of OPC UA nodes. Make this the first entry in the configuration sub-section. Make sure it starts with a dash ‘-‘ which indicates the beginning of a new grouping. |
||
group_name: |
Unique name for the set of parameters required to poll an OPC UA ip_device |
||
ip_device: |
The ip_device to poll for data |
||
ip_port: |
The ip_port on which the ip_device is listening for data |
||
username: |
The OPC UA username to use when querying the ip_device |
||
password: |
The OPC UA password to use when querying the ip_device |
||
nodes: |
OPC UA Analog Value node to poll for data from for the ip_devices . Each address must be a OPC UA node. The multiplier is the value by which the polled data result must be multiplied. This is useful in converting byte values to bits. The default multiplier is 1. |
Polling¶
Use pattoo_agent_opcuad
to poll your devices. The daemon has a simple command structure below.
You will need a pattoo_agent_opcuad.yaml
configuration file in the PATTOO_CONFIGDIR
directory before you start.
$ bin/pattoo_agent_opcuad.py --help
usage: pattoo_agent_opcuad.py [-h] [--start] [--stop] [--status] [--restart]
[--force]
optional arguments:
-h, --help show this help message and exit
--start Start the agent daemon.
--stop Stop the agent daemon.
--status Get daemon daemon status.
--restart Restart the agent daemon.
--force Stops or restarts the agent daemon ungracefully when used with --stop or
--restart.
$
General Operation¶
Use these commands for general operation of the daemon.
Start Polling at Boot¶
Configuration Guide provides information on how to get the pattoo_agent_opcuad
daemon to start at boot.
Troubleshooting¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
Miscellaneous Information¶
Technical background information on the project.
Troubleshooting Pattoo Agents¶
Troubleshooting steps can be found in the PattooShared troubleshooting documentation
JSON Formatting for pattoo-agents¶
JSON data formatting can be found in the PattooShared data documentation
Pattoo Terminology¶
A complete glossary of terms can be found in the Pattoo Shared glossary documentation
Developers¶
How To Contribute¶
Start contributing today!
Introduction¶
Below is the workflow for having your contribution accepted into the pattoo-agents
repository.
Create an Issue or comment on an existing issue to discuss the feature
If the feature is approved, assign the issue to yourself
Fork the project
Clone the fork to your local machine
Add the original project as a remote (git remote add upstream https://github.com/PalisadoesFoundation/pattoo-agents, check with: git remote -v)
Create a topic branch for your change (git checkout -b
BranchName
)you may create additional branches if modifying multiple parts of the code
Write code and Commit your changes locally. An example of a proper
git commit
message can be seen below:Make the example in CONTRIBUTING imperative and concrete ... Without this patch applied the example commit message in the CONTRIBUTING document is not a concrete example. This is a problem because the contributor is left to imagine what the commit message should look like based on a description rather than an example. This patch fixes the problem by making the example concrete and imperative. The first line is a real life imperative statement with a ticket number from our issue tracker. The body describes the behavior without the patch, why this is a problem, and how the patch fixes the problem when applied. Resolves Issue: #123 See also: #456, #789
When you need to synch with upstream (pull the latest changes from main repo into your current branch), do:
git fetch upstream
git merge upstream/master
Check for unnecessary white space with
git diff --check
.Write the necessary unit tests for your changes.
Run all the tests to assure nothing else was accidentally broken
Push your changes to your forked repository (git push origin
branch
)Perform a pull request on GitHub
Your code will be reviewed
If your code passes review, your pull request will be accepted
Code Style Guide¶
For ease of readability and maintainability code for all pattoo
projects must follow these guidelines. Code that does not comply will not be added to the master
branch.
- All
pattoo
projects use the Google Python Style Guide for general style requirements - All
pattoo
python projects use the The Chromium Projects Python Style Guidelines for docstrings. - Indentations must be multiples of 4 blank spaces. No tabs.
- All strings must be enclosed in single quotes
- In addition too being
pylint
compliant, the code must be PEP8 and PEP257 compliant too. - There should be no trailing spaces in files
Guidelines to remember¶
- Always opt for the most pythonic solution to a problem
- Avoid applying idioms from other programming languages
- Import each module with its full path name. ie: from pack.subpack import module
- Use exceptions where appropriate
- Use doc strings
- Try not to have returns at multiple points in a function unless they are failure state returns.
- If you are in the middle of a development session and have to interrupt your work, it is a good idea to write a broken unit test about what you want to develop next. When coming back to work, you will have a pointer to where you were and get back on track faster.
Commits¶
The pattoo
projects strive to maintain a proper log of development through well structured git commits. The links below offer insight and advice on the topic of commit messages:
Sample .vimrc File for Compliance¶
You can use this sample .vimrc file to help meet our style requirements
" Activate syntax
syntax on
" set number
" Disable automatic comment insertion
autocmd FileType * setlocal formatoptions-=c formatoptions-=r formatoptions-=o
" Delete trailing whitespace
autocmd BufWritePre * :%s/\s\+$//e
" Convert tabs to spaces
set expandtab
" Set tabs to 4 spaces
set tabstop=4
" Set the number of spaces for indentation
set shiftwidth=4
" Switch on highlighting the last used search pattern when the terminal has colors
if &t_Co > 2 || has("gui_running")
set hlsearch
endif
" Tell vim to remember certain things when we exit
" '10 : marks will be remembered for up to 10 previously edited files
" "100 : will save up to 100 lines for each register
" :20 : up to 20 lines of command-line history will be remembered
" % : saves and restores the buffer list
" n... : where to save the viminfo files
set viminfo='10,\"100,:20,%,n~/.viminfo
" Function for viminfo to work
function! ResCur()
if line("'\"") <= line("$")
normal! g`"
return 1
endif
endfunction
" Function for viminfo to work
augroup resCur
autocmd!
autocmd BufWinEnter * call ResCur()
augroup END