What is Spiderwiz Data Compression?

One of the main motivations behind the Spiderwiz framework is to make data sharing robust, fast and easy. When a programmer commits a data object, she shall not worry about the time it takes to the object to reach its destination even if the way there is through a relatively slow modem connection, neither shall he worry about the amount of bytes the serialized object consumes. But the framework's runtime does. To achieve its goal, the runtime applies a two-tier compression algorithm.

The lower tier is physical compression, namely GZIP. The upper tier is object-based logical compression. The idea behind it is that when a data object changes it is enough to send over the network only the discrepancies between the current state of the object and its previous version, rather than sending the entire object content. This is what Spiderwiz does, coupled with a mechanism for detection of data loss and recovering full key-frames when necessary. (The term key-frame is taken from video compression, which is based on the same idea).

For object-based compression to be efficient, objects need to be identified. Obviously two successive reports from the same car in a navigation system, to take an example (and even more two successive instructions to the same driver), differ much less than data from and to different cars. That's why Spiderwiz introduces object hierarchy and object IDs and developers are encouraged to use it as much as possible, even if the object is disposable.

How to Control Data Compression

You can use Application Configuration to control data compression for individual communication channels. The compress parameter determines how compression is done on the related channel. The options are:

no (or anything that starts with it) - no compression (the default for disk file input/output).
zip - physical compression using GZIP.
logical - object-based compression.
full - both physical and logical compression (the default for all communication types except disk file input/output).

Additionally, as a programmer developing a communication plugin, you can override Channel.isCompressable() to determine whether or not compression (in both tiers) applies to the channel by default.

Note that compression is always a full-duplex operation. If any type of compression is used in one communication direction, it must also be used in the other direction. Therefore if compression configuration tags are used as we have just explained and different tags are used for each direction, the actual compression that will be performed is the common denominator. For instance, if zip compression is configured in one direction and logical compression is configured in the opposite direction, no compression will take place.