Snarl is Project FiFo's RBAC service. Snarl allows for fine grained RBAC style permissions.

Snarl has multi-datacenter that allows for synchronization of user data between multiple datacenters.

Snarl also has multi form factor authentication support through the use of Yubikeys.

Snarl has the following properties:

• No SPOF when using multiple nodes.
• Data is stored using LevelDB.
• Provides authorization and authentication for user actions inside Project-FiFo and Project-FiFo controlled vms.

Configuration

Configuration file

As with Version 0.8.0 and up Snarl's configuration file is located in /data/snarl/etc/snarl.conf

Deprecated

Snarl's configuration file is located in /opt/local/fifo-snarl/etc/snarl.conf. It is automatically generated on the first install and will not be overwritten on updates.

Nonetheless the newest version of the file is always located in /opt/local/fifo-snarl/etc/snarl.conf.example.

The configuration file is documented in-line but we'll go over some more interesting settings here.

Active Anti Entropy (AAE)

AAE is riak's mechanism of background synchronization of systems to ensure a higher data consistency. It was ported to FiFo to offer increased stability in multi node setups. AAE increases the required resources considerably and does not have much use with a single system so it is disabled by default.

🚧

When having more then one system it strongly recommended to enable AAE!

## enable active anti-entropy subsystem
anti_entropy = on

Database

FiFo uses LevelDB as it's back-end database. LevelDB has many different tune-ables. Some of the more important ones are:

ring_size

The number of VNodes used by FiFo, this setting can only be changed before the system is booted the first time so choose it carefully, it defaults to 64 but with more then one systems it is very sensible to increase this number higher, generally ~10 vnodes per physical node are a good rule of thumb. The ring_size needs to be a a power of two (2, 4, 8 ... 64 ...).

leveldb.mmap_size

The chunk size of each mmaped file, this has a huge impact of the memory requirements. Since FiFo does not store lots of data a setting of 1MB is a valid value for small and medium installations, the settings can be increased as long as it is ensured that enough memory is present.

Multi DC support

The multi DC support in FiFo is based in Snarl. It works by synchronizing user data between all data-centers. Synchronization happens on a point to point system meaning that each Snarl instance in DC1 synchronizes with one system in DC2.

By default the synchronization subsystem is disabled. It needs to be enabled and configured in order to properly propagate data.

sync

Either set to on or off enabling or disabling the system, a restart is required to enable it but it can be changed at any time after the installation.

sync.ip

The IP/Port this server will listen to for incoming sync connections. The syntax is <IP>:<Port>, this needs to be matched on the remote side.

sync.partner.$name

One Sniffle node is able to sync to multiple remote nodes, usually it should have exactly one parter per data-center, having multiple partners in the same data-center is unsupported can have unforeseen results. The syntax is <IP>:<Port>.

sync.build_interval

For each subsystem Snarl keeps a list of hashes, the build_interval determines how often the system rebuilds this from scratch. Aside of the full rebuild the hashes are dynamically updated.

sync.read_delay

During a full rebuild reads from the Snarl system are delayed to prevent slowing down the database during the rebuild this value specifies the delay between reads.

sync.interval

This defines how often it should be checked if differences exists between local and remote nodes. Please keep in mind that sync should be set up in both directions this syncs will be performed about twice as often as specified here.

sync.recv_timeout

If a command is send to a sync partner and this threshold is exceeded the connection will attempt to bounce and be reestablished.

Global configuration

In addition to the config files that apply on a per node level there are global configurations that can be changed from one system and are applied globally. Unless otherwise noted all those settings can be changed during runtime.

snarl-admin config show

Shows a list of all settings in the global configuration.

snarl-admin config set <key> <value>

Sets a global config value please see the following sections for valid settings.

defaults.users.inital_role

If this is set every user created gets placed into the role given. The value must be the UUID of the role.

Initial roles/orgs

This settings apply per realm, unless changed the realm will be default.

📘

Default role

The initial_role parameter is especially helpful when using a default Users role that cover basic permissions grated for every user.

YubiCo parameters

Administration

The Snarl admin command is snarl-admin but many commands can also be accessed via fifoadm command. Please keep in mind that fifoadm is not designed as an every day command but only as a last fall-back when commands are not available through the API.

General Management

Snarl uses the SMF to manage it's running state so it is restarted in the case of crashes and booted accordingly on system start. Snarl can be enabled, disabled and restarted via: svcadm enable snarl, svcadm disable snarl and svcadm restart snarl

Cluster management

snarl-admin cluster join <nodename>@<ip>

Joins a Snarl cluster, please note that all data on the joining (not the joined) node is deleted.

snarl-admin cluster leave

Cleanly removes a node from the ring. This is helpful when nodes get moved and the cluster downsized

snarl-admin cluster leave <nodename>@<ip>

Forcefully removes a node from the ring. This can be used after fatal node cluster.

snarl-admin cluster plan

Shows the current plan on how to change the cluster.

snarl-admin cluster commit

Executes the current cluster plan.

snarl-admin member-status

Lists the status of each node and the distribution of data over the ring nodes.

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid     100.0%      --      'snarl@172.16.0.254'
-------------------------------------------------------------------------------
Valid:1 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

snarl-admin ring-status

Gives an extended report on the ring including hand-offs and downed nodes.

================================== Claimant ===================================
Claimant:  'snarl@172.16.0.254'
Status:     up
Ring Ready: true

============================== Ownership Handoff ==============================
No pending changes.

============================== Unreachable Nodes ==============================
All nodes are up and reachable

snarl-admin status

A simple command that returns the overall cluster status. It returns a proper return code and is useful for scripted rolling updates.

snarl-admin aae-status

Gives a detailed status on the AAE status of the system.

Log Files

New with Version 0.8.0 and up:
FiFo uses extensive logging to make debugging issue and understanding behavior. The log files are located in /data/snarl/log/. There are multiple log files with various severities.

Depreceated: (Version 0.7.x and before)
FiFo uses extensive logging to make debugging issue and understanding behavior. The log files are located in /var/log/snarl/. There are multiple log files with various severities.

debug.log

By default the debug log is disabled, it is very verbose and should not be enabled in production systems to enable it uncomment the following line in the snarl.conf

log.debug.file = /var/log/snarl/debug.log

console.log

This file contains logs of the level info and above, usually all interesting logs can be found here.

error.log
This files contains errors, it usually should be mostly empty but please keep in mind that failing is not a uncommon practice to deal with unexpected behavior so sporadic entries might just be fine.

General tasks

snarl-admin users|role|org add <realm> <name>

Adds a user or role, especially helpful when no users exist yet.

snarl-admin users|role grant <realm> <name> <permission> [<permission>]

Grants a user or role permissions. A permission can have multiple elements so: snarl-admin users grant user some special permissions would grant some->special->permissions to user. ... and _ are special cases and not taken as strings but as the wildcards for the permission section where _ means this level matches and ... means everything below this level matches.

snarl-admin users passwd <realm> <username> <password>

Changes the password for a user.

snarl-admin users|roles|orgs list <realm>

Lists all users, roles or organisations.

snarl-admin users|roles|orgs delete <realm> <name>

Deletes a given user, role or organisation.