---
agentTools:
  projectIndex: https://docs.project-fifo.net/llms.txt
---

# Monitoring

There are multiple things to do to monitor FiFo.

# General

The most basic way thing to monitor is the same as for any application, CPU, memory, IO, network saturation, and disk usage.

# Services

## FiFo-Zone

In the FiFo zone the following SMF services can be checked in every installation:

* `sniffle` - main management service
* `snarl` - AAA service
* `howl` - API endpoint
* `epmd` - Supporting Erlang service

In addition if other services are installed the following services might exist:

* `fifo_dns`- DNS endpoint
* `kennel` - Docker endpoint
* `dalmatiner/db` - DalmatinerDB
* `dalmatiner/fe` - Dalmatiner Frontend
* `tachyon` - Tachyon aggregator
* `nsqadmin` - NSQ Admin node
* `nsqd` - NSQ daemon

## Global Zone

In the global zone the following SMF services can be checked in every installation:

* `fifo/zlogin` - FiFo zlogin service
* `chunter` - Zone manager

In addition if other services are installed the following services might exist:

* `tachyon-meter` - Tachyon
* `nsqadmin` - NSQ Admin node
* `nsqd` - NSQ daemon

# Services

## RiakCore based services (sniffle, snarl, howl)

All those services share some common commands to get status from them, they are all called `-admin`, so `sniffle-admin`, `snarl-admin`, and `howl-admin`. For the following section `snarl-admin` will stand in as a default example.

### Status

This is a very generic command that will give a short information if the cluster is in a good state or not. It can be used by running `snarl-admin status`.

[block:code]
{
  "codes": [
    {
      "code": "[root@75f04bc5-a457-4351-ce32-e3a5f2ace850 ~]# snarl-admin status\nThe cluster is fine!",
      "language": "text"
    }
  ]
}
[/block]

### Handoffs

Handoffs mean that data is transferred from one node to another, this happens after communication between nodes was lost, a node went down/was restarted or nodes were added or removed.

Generally handoffs are part of normal operations not not a reason for concern, however them appearing frequently can indicate underlaying problems as crashes, network partitions or very high overload.

[block:code]
{
  "codes": [
    {
      "code": "[root@75f04bc5-a457-4351-ce32-e3a5f2ace850 ~]# snarl-admin handoff summary\nEach cell indicates active transfers and, in parenthesis, the number of all known transfers.\nThe 'Total' column is the sum of the active transfers.\n+----------------------+-----+---------+------+------+------+\n|         Node         |Total|Ownership|Resize|Hinted|Repair|\n+----------------------+-----+---------+------+------+------+\n|  snarl@192.168.1.43  |  0  |         |      |      |      |\n|  snarl@192.168.1.44  |  0  |         |      |      |      |\n|  snarl@192.168.1.45  |  0  |         |      |      |      |\n|  snarl@192.168.1.46  |  0  |         |      |      |      |\n+----------------------+-----+---------+------+------+------+",
      "language": "text"
    }
  ]
}
[/block]

Handoff status can be checked by running `snarl-admin handoff status`

### Cluster membership

It is possible to check the cluster membership of all nodes with the `sniffle-admin member-status` command, it will give an idea of data distribution between nodes and indicate if all nodes agree on the cluster layout. Also leaving and joining nodes can be seen here.

[block:code]
{
  "codes": [
    {
      "code": "[root@75f04bc5-a457-4351-ce32-e3a5f2ace850 ~]# snarl-admin member-status\n================================= Membership ==================================\nStatus     Ring    Pending    Node\n-------------------------------------------------------------------------------\nvalid      25.0%      --      'snarl@192.168.1.43'\nvalid      25.0%      --      'snarl@192.168.1.44'\nvalid      25.0%      --      'snarl@192.168.1.45'\nvalid      25.0%      --      'snarl@192.168.1.46'\n-------------------------------------------------------------------------------\nValid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0",
      "language": "text"
    }
  ]
}
[/block]

### Ring Status

It is possible to gather general information about the distribution mechanism with the `snarl-admin ring-status` command.

[block:code]
{
  "codes": [
    {
      "code": "[root@75f04bc5-a457-4351-ce32-e3a5f2ace850 ~]# snarl-admin ring-status\n================================== Claimant ===================================\nClaimant:  'snarl@192.168.1.43'\nStatus:     up\nRing Ready: true\n\n============================== Ownership Handoff ==============================\nNo pending changes.\n\n============================== Unreachable Nodes ==============================\nAll nodes are up and reachable",
      "language": "text"
    }
  ]
}
[/block]

# Ports

All ports are configurable the following table will list only the defaults. Additional ports are allocated dynamically via the `epmd` service.

[block:parameters]
{
  "data": {
    "h-0": "Service",
    "h-1": "Ports",
    "0-0": "Snarl",
    "1-0": "Sniffle",
    "2-0": "Howl",
    "3-0": "Chunter",
    "2-1": "80, 443, 4240, 8499",
    "0-1": "8099, 4200, 4201",
    "1-1": "8199, 4210",
    "3-1": "4200"
  },
  "cols": 2,
  "rows": 4
}
[/block]

# LeoFS

LeoFS can be checked by running the [status command](http://leo-project.net/leofs/docs/admin_guide/admin_guide_2.html#status-command) on one of the leofs managers.