Setting up superseriousstats 5.1

All commands in this guide are based on Unix, for their Windows equivalents please check corresponding documentation. superseriousstats depends on certain third party software to be installed and configured properly. To keep this guide clear and manageable I won't go into too much detail on how to set up said software.

Requirements

In order to parse IRC logs the PHP CLI >= 5.3 must be available. Two modules need to be compiled in: mysqli and mbstring. Optionally, if you want to be able to parse gzipped logs, the zlib module must be enabled as well.

Example: compile PHP 5.4 with all mentioned modules enabled. ./sapi/cli/php is what you need.

$ ./configure --disable-all --with-mysqli --enable-mbstring --with-zlib
$ make

Equally important is storing the processed data. For this superseriousstats uses MySQL 5.1. Later versions of MySQL might work as well. The MySQL user should have all table related rights within the chosen database, such as CREATE, INSERT, DELETE, DROP etc.

Finally, you will need a Webserver to serve the static main stats page (HTML) and optional dynamic user and history stats (PHP). The latter ofcourse requires that your Webserver incorporates PHP >= 5.3 with mysqli compiled in.

Supported logfile formats

superseriousstats can currently handle a reasonable selection of logfile formats, which are listed below. Keep in mind that only the default logging syntax of the client or bot is supported. To test whether the parser works properly set outputbits in sss.conf to include debug messages and do an import of your log directory. Log lines that are not being recognized by the parser will then be shown and can be used to improve the parser code.

Regardless of which logfile format you use, the following requirements must be met: one logfile per day with the full date in its filename, e.g. #chatroom.log.20090328. Secondly, all log lines should be prefixed with a timestamp including hour, minutes and optionally also seconds.

Eggdrop is fully supported.

HexChat is supported when using the default Month Day Time timestamping format.

Irssi is fully supported. Note that themes may affect the logging syntax.

LimeChat is fully supported.

mIRC is supported although action lines are intentionally not being picked up by the parser. The way mIRC logs action lines is broken, the syntax cannot be properly distinguished from various other events. There is a workaround for this by means of loading a little mIRC script (mirc6hack) as seen below (copied from the PISG project). It prefixes actions with double asterisks instead of the default single one. Problem solved!

# 2004-11-21 by coaster

alias me {
  if ($1) {
    .describe $active $1-
    echo $color(own) -qt $active ** $me $1-
  }
  else {
    echo $color(info) $active * /me: insufficient parameters
  }
}

on ^*:ACTION:*:*:{
  echo $color(action) -lt $iif($chan,$chan,$nick) ** $nick $1-
  haltdef
}

muh2 is fully supported using the same parser as ZNC.

nodelog is supported even though there is not much being logged.

Smuxi is fully supported using the same parser as Irssi.

Supybot is fully supported.

Textual is fully supported.

ZNC is fully supported.

Getting a release and configuring it

It is recommended to download a packaged release since these are tested to work properly. Let's say you grabbed version 5.0 and you extracted it in ~/sss-5.0/. Next, open up the MySQL console and initialize a database for your channel:

$ mysql -u user -p
***
create database sss;
use sss;
source ~/sss-5.0/empty_database_v5.sql;
quit;

This will create all the tables needed by superseriousstats. You can also run the above commands to reset your database to its initial state if you ever desire to do so.

It is important to note that the major version number of the release should be the same number the empty database scheme has. In this case 5. Releases that are no longer backward compatible will have their major version number increased. A full reimport of all logfiles using the new scheme is necessary when upgrading to such a release.

Finally, go over the settings in ~/sss-5.0/sss.conf and edit them to fit your needs and preferences. sss.conf is the default configuration file the sss.php program will look for. Alternatively, you can create a config file by a different name and in a different location and use the -c flag to use it.

Importing logs

Logs can be imported by specifying a single file or a directory after the -i flag. When pointing to a directory, superseriousstats will look up the parse history for your channel and continue parsing all fresh activity from where it left off previously. This is the recommended way of importing logs.

Example 1: import all logs from specified directory.

$ php sss.php -i /path/to/logs/

Example 2: import just one log and use a specific config file.

$ php sss.php -c /etc/mychannel.conf -i /path/to/logs/log.20110326.txt

When using the single file method of importing logs make sure to issue them in chronological order starting with the oldest one first.

Creating the stats page

Before creating the stats page you will need to copy the contents of ~/sss-5.0/www/ to your Webserver. To be specific: sss.css and optionally favicon.ico. If you plan to serve user and history stats you will also need to copy user.php, history.php and vars.php and edit the settings in the vars.php file. You can then browse to the user and history stats via the main stats page.

$ php sss.php -o /path/to/www/index.html

Automate with cron

You can schedule a cron job to create the stats page at any interval you desire as long as these jobs don't overlap, and possibly interfere, when executing. Depending on how fast your machine is, a generally safe interval would be 15 minutes or more.

Example: run once every hour.

0 * * * * /usr/local/bin/php /path/to/sss.php -i /path/to/logs/ -o /path/to/www/index.html

Automatic linking of nicks

By default superseriousstats looks for nicks that are near-identical and automatically links these nicks together. It uses a very conservative approach in doing so, and tries to eliminate any false positives. To learn the exact mechanics of this feature it is best to have a look at the source code. The automatic linking of nicks can be disabled by setting autolinknicks to false in sss.conf. To manually link nicks read on below.

Different nicks, same user!

All unique nicks are treated as separate users on the stats page. A user may however have multiple nicks belonging to him or her. Take the following example:

<@Jane> OMG house on fire!
-!- Jane is now known as Jane^brb
<@Jane^brb> quick AFK

Both Jane and Jane^brb belong to the same user but aren't linked automatically. This can be done manually but first, let's go over the four different types of nicks that superseriousstats uses:

Unlinked nicks are the default type. These nicks have no aliases linked to them and are not an alias themselves either.

Alias is a type of nick that belongs (is linked) to another nick.

Registered is the type used to indicate nicks that have aliases, or are reserved so they cannot become an alias themselves. If a user has multiple aliases, the alias with the most lines typed will automatically become the registered nick.

Bot is essentially the same as a registered nick but specifically marked as a bot. You may be familiar with the term bot, a service providing IRC client, like the network's NickServ or an Eggdrop. Bots are handled differently from other nicks when creating stats, they are generally left out of most stats tables yet there are a few specific tables just for them.

Exporting nicks

The first step when we start with linking nicks, and for the purpose of making a backup, would be to export all nicks that are currently in the database:

$ php sss.php -e allnicks.txt

The textfile may look something like this:

1,Bill,Bill_,Bill^away,CrazyBill
1,Charles,Charlie
*,Anonymous,Billzzz,HAL9000,Jane,Jane^brb

The first line shows all linked nicks for the user Bill. The number 1 in front of the list of aliases tells us we are dealing with a registered user. The last line starts with an asterisk, indicating that all the following nicks (on the same line) are unlinked.

Linking nicks

If we want to manually link nicks we should edit the textfile we just exported. There are a few conditions when doing so: we are not allowed to use any nicks which are not already in the file, nor use wildcards of any sort. We cannot link nicks to multiple users, the last occurence will take effect. All nicks we leave out of the file when doing an import will be unlinked. All input is case insensitive.

Now, let's create an entry for Jane:

1,jane,jane^brb

And while we're here, also update the aliases of Bill:

1,Bill,Bill_,Bill^away,CrazyBill,billzzz

The complete file so far:

1,Bill,Bill_,Bill^away,CrazyBill,billzzz
1,Charles,Charlie
*,Anonymous,Billzzz,HAL9000,Jane,Jane^brb
1,jane,jane^brb

Tagging bots

Some people run bots in their channel. Linking nicks to a bot is identical to what we did for normal users, the only difference is we use the number 3 instead of 1 to indicate that the nick is, or belongs to, a bot:

3,HAL9000

The complete file, after safely omitting the list of unlinked nicks, now looks like this:

1,Bill,Bill_,Bill^away,CrazyBill,billzzz
1,Charles,Charlie
1,jane,jane^brb
3,HAL9000

Importing nicks

The final step is to import the file we just saved back into the database:

$ php nicklinker.php -n allnicks.txt

For each user or bot the first nick in the list will initially be their main nick and all following nicks will be aliases. This may change when the maintenance script is run, which will automatically make the alias with the most lines typed the main nick.