ProFTPD module mod_statsd

## ProFTPD module `mod_statsd`

The `mod_statsd` module instruments the ProFTPD code to emit metrics directly to a `statsd` server, avoiding the need for custom log file parsing.

This module is contained in the `mod_statsd` files for ProFTPD 1.3.x, and is not compiled by default. Installation instructions are discussed here. More examples of `mod_statsd` usage can be found here.

The most current version of `mod_statsd` can be found at:

      https://github.com/Castaglia/proftpd-mod_statsd

## Author

Please contact TJ Saunders <tj at castaglia.org> with any questions, concerns, or suggestions regarding this module.

## Directives

- StatsdEngine
- StatsdExcludeFilter
- StatsdSampling
- StatsdServer

### StatsdEngine

Syntax: StatsdEngine on|off
Default: off
Context: server config, `<VirtualHost>`, `<Global>`
Module: mod_statsd
Compatibility: 1.3.6rc1 and later

The `StatsdEngine` directive enables or disables the emitting of metrics to the configured `statsd` server.

### StatsdExcludeFilter

Syntax: StatsdExcludeFilter regex|"none"
Default: None
Context: server config, `<VirtualHost>`, `<Global>`
Module: mod_statsd
Compatibility: 1.3.6rc1 and later

The `StatsdExcludeFilter` directive configures a regular expression filter that is applied to every command. Any command which matches the configured regular expression will not be sampled by `mod_statsd`.

Example:

      # Exclude SYST commands from our metrics
      StatsdExcludeFilter ^SYST$

### StatsdSampling

Syntax: StatsdSampling percentage
Default: 100
Context: server config, `<VirtualHost>`, `<Global>`
Module: mod_statsd
Compatibility: 1.3.6rc1 and later

The `StatsdSampling` directive configures the percentage of events that `mod_statsd` will sample sample, and thus send to `statsd`. With very busy systems, sampling 100% of the metrics can place undue stress on the metrics collection, while sampling a smaller percentage of the events can still provide a statistically relevant view of the system.

The configured percentage value must be between 1 and 100.

Example:

      # Sample only 10 percent of the metrics
      StatsdSampling 10

### StatsdServer

Syntax: StatsdServer [scheme://]address[:port] [prefix] [suffix]
Default: None
Context: server config, `<VirtualHost>`, `<Global>`
Module: mod_statsd
Compatibility: 1.3.6rc1 and later

The `StatsdServer` directive is used to configure the IP address/port of the `statsd` server that the `mod_statsd` module is to use. For example:

      StatsdServer 1.2.3.4:8125

or, for an IPv6 address, make sure the IPv6 address is enclosed in square brackets:

      StatsdServer [::ffff:1.2.3.4]:8125

By default, the `mod_statsd` module will use UDP when sending metrics to the configured `statsd` server. For more reliable (but slower) transmission of data using TCP, use the optional scheme prefix, e.g.:

      # Use TCP instead of UDP
      StatsdServer tcp://1.2.3.4:8125

or, to explicitly declare that you want to use UDP:

      # Use UDP, with an IPv6 address
      StatsdServer udp://[::ffff:1.2.3.4]:8125

The `StatsdServer` directive also supports optional prefix and suffix values. These are strings which will be used as prefixes and suffixes to the metric names. For example:

      StatsdServer udp://1.2.3.4:8125 proftpd.prod ftp03

This will use the prefix "proftpd.prod", and the suffix "ftp03", to generate counter/timer metric names such as `proftpd.proftpd.command.PASS.230.ftp03`, instead of the default `command.PASS.230`. Other examples:

      # Use a prefix but no suffix for the metrics
      StatsdServer udp://1.2.3.4:8125 proftpd.prod

      # Use a suffix but no prefix for the metrics
      StatsdServer udp://1.2.3.4:8125 "" ftp03

## Installation

To install `mod_statsd`, copy the `mod_statsd` files into:

      proftpd-dir/contrib/

after unpacking the latest proftpd-1.3.x source code. For including `mod_statsd` as a staticly linked module:

      $ ./configure --with-modules=mod_statsd

To build `mod_statsd` as a DSO module:

      $ ./configure --enable-dso --with-shared=mod_statsd

Then follow the usual steps:

      $ make
      $ make install

## Usage

An example configuration:

      <IfModule mod_statsd.c>
        StatsdEngine on
        StatsdServer 127.0.0.1:8125
      </IfModule>

Metrics
The `mod_statsd` module emits one counter, and one timer, for each command processed. For example, a `USER` command which receives a 331 response code would generate a counter and a timer whose name is:

      command.USER.331

Optional metric prefixes and/or suffixes can be configured via the `StatsdServer` directive, e.g.:

      StatsdServer udp://1.2.3.4:8125 prod ftp01

would result in command metric names like:

      prod.command.USER.331.ftp01

For each connection, `mod_statsd` will emit one counter and one gauge using a metric name of:

      connection

Protocol-specific connection metrics are also emitted: a counter, a timer, and a gauge, all using a metric name of "protocol.connection", thus:

- `ftp.connection`
- `ftps.connection`
- `sftp.connection`
- `scp.connection`

In addition, `mod_statsd` increments counters when the following timeouts are encountered:

- `TimeoutIdle`
- `TimeoutLogin`
- `TimeoutNoTransfer`
- `TimeoutSession`
- `TimeoutStalled`

using a metric name of "timeout.name", like:

      timeout.TimeoutLogin

TLS-Specific Metrics
For FTPS connections, `mod_statsd` emits some TLS-specific metrics. The metric name `tls.handshake.ctrl` is used for both a counter and a timer, for tracking number of successful TLS handshakes and how long they take. For failed TLS handshakes, the following two counters are used:

- `tls.hansdshake.ctrl.error`
- `tls.hansdshake.data.error`

Counters on the TLS protocol versions and ciphers used by FTPS clients are also available. Note that these TLS-related counter metrics are only available when FTPS is used, and when the `StdEnvVars` `TLSOption` is enabled:

      TLSOptions StdEnvVar

The counters in question are:

      tls.cipher.cipher
      tls.protocol.protocol

thus, for example, you would see:

      tls.cipher.ECDHE-RSA-AES128-SHA
      tls.protocol.TLSv1

Logging
The `mod_statsd` module supports trace logging, via the module-specific log channels:

- statsd
- statsd.metric
- statsd.statsd

Thus for trace logging, to aid in debugging, you would use the following in your `proftpd.conf`:

      TraceLog /path/to/ftpd/trace.log
      Trace statsd:20

This trace logging can generate large files; it is intended for debugging use only, and should be removed from any production configuration.

© Copyright 2017 TJ Saunders
All Rights Reserved