Spiderwiz Logging System

General

Spiderwiz features a powerful logging mechanism. Logs are recorded in a file system that allows easy search for every event log. Most of the logging is done by the framework automatically, freeing programmers to concentrate on their tasks. If needed, custom logs can be written in the appropriate log file with a one-line API.

In this page we will explain the log system file structure, the various log options and how to configure them, and custom logging.

Log System File Structure

All the log files are arranged under the main log folder. By default it is the folder named logs under the application root folder, but it can be configured to another location as described in Spiderwiz Configuration.

Log files are arranged by days and hours. There are daily folders, named yy-mm-dd, where yy are two digits of the year, mm are two digits of the month and dd are two digits of the day. Under the day folder there are hourly log files, named aphh.txt where ap is am or pm and hh is the two-digit hour.

The main log folder contains daily log folders for general events, such as the start and the termination of the application, connections and disconnections of communication lines, code exceptions etc. This is also where custom logs are recorded.

Besides the daily log folders for general events, the main log folder may contain the following folders:

Producers - where events related to applications connected to this application as producers (i.e. this application acts as a consumer) are recorded.
Consumers - where events related to applications connected to this application as consumers (i.e. this application acts as a producer) are recorded.
Imports - where events related to import channels are recorded.

Each of these folders contains another level of sub-folders, each for every connection. For producers and consumers, the names of the sub-folders reflect the names of the connected applications (after necessary character escaping to make them valid folder names). For imports, the names of the sub-folders reflect the configured names of the import channels as described in Spiderwiz Configuration. These sub-folders contain the daily folders that contain the hourly log files as described above.

Example:

Assuming we are running a car navigation application. Our application consumes VehicleLocation objects that contain information about car locations and produces VehicleDirection objects that instruct drivers where to go. In order to determine best routes, our application consumes also TrafficConditions objects, which contain information about traffic jams etc. in various route segments. It also needs TrafficStatistics objects to get traffic statistics for various route segments at various time windows of the day.

So let's look at the network topology. We connect as a consumer to an application called VehicleReportCollector to get VehicleLocation objects. We also connect as a consumer to TrafficConditionReporter. Our output goes to an application called DriverManager, so we connect to it as a producer. Finally, we import traffic statistics from an import channel that we name StatisticManager.

We let our application run 3 hours, from 11pm on December 31, 2019 until 2am on January 1, 2020. Assuming the log root folder name is the default logs and there was something to log for every connection on every hour (definitely not necessarily the case in every scenario), then at the end of the period the log file system would look like:

• logs
  • 19-12-31
    • pm11.txt
  • 20-01-01
    • am00.txt
    • am01.txt
  • Producers
    • VehicleReportCollector
      • 19-12-31
        • pm11.txt
      • 20-01-01
        • am00.txt
        • am01.txt
    • TrafficConditionReporter
      • 19-12-31
        • pm11.txt
      • 20-01-01
        • am00.txt
        • am01.txt
  • Consumers
    • DriverManager
      • 19-12-31
        • pm11.txt
      • 20-01-01
        • am00.txt
        • am01.txt
  • Imports
    • StatisticManager
      • 19-12-31
        • pm11.txt
      • 20-01-01
        • am00.txt
        • am01.txt

Features and Options of the Log System

Logs of execution events, such as application start and termination, connections and disconnections of communication lines, data loss detection and code exceptions caught by the framework, are automatically generated by the framework. In addition, the application can be configured to log every input or output line that goes over communication lines.

With respect to full logs of compressed data, you can choose the type of the logs as follows:

raw - input lines are logged as they are, i.e. with the objects compressed.
full - input lines are expanded to show uncompressed objects.
verbose - every line is logged twice - raw and full.

This kind of logging can be configured generally for every connection, or individually for specific connections. See details in Application Configuration.

Additionally, when configuring a specific connection, you can use the flushlogs keyword as one of the configuration parameters (no assignment is needed) to indicate that the file that logs the activity on this connection shall be flushed on each transaction. Very useful for debugging, but should be used with care since it slows performance down dramatically.

Log Line Structure

Every log line in the log files consists of the log time in milliseconds followed by the log text, in the format:

hh:mm:ss:SSS log-text

Raw input lines are recorded as follows:

hh:mm:ss:SSS <- input-line

Expanded input lines are recorded as follows:

hh:mm:ss:SSS <= input-line

Raw output lines are recorded as follows:

hh:mm:ss SSS -> output-line

Expanded output lines are recorded as follows:

hh:mm:ss SSS => output-line

There is one exception - exception. Code exception stack traces are recorded as a series of lines without the timestamp.

Custom Logging

Adding custom logs to the main log folder is easy. You need first to get the application log object with Main.getLogger()and then use any of the methods of the returned ZLog object. For instance:

Main.getLogger().logf("Finished counting. Counted %d votes in total", votes);

Alternatively you can create your own ZLog object and use it to create custom log folders. For instance:

    ZLog myLogger = new ZLog(Main.getInstance().getRootFolder(), Main.getConfig());
    myLogger.init("my log folder");
    myLogger.logEvent("Did it! This is my first custom log");