Up till now we demonstrated the concept of live data object sharing. Data flows between applications that are up, running and connected. If any application goes offline, it issues Reset requests when it comes back and peer applications respond by re-sharing the objects that they have created. This works with indisposable objects, but disposable objects that are not consumed are, well, disposed.
There are, however, cases when disposable objects are not necessarily dispensable (who said food?). In these cases we do not want to lose data even when the consumer application is disconnected or inactive. There are excellent tools in the market for handling these scenarios, most notably Apache Kafka (and a Kafka plugin for Spiderwiz is on the way). Here we will describe a simpler solution that is built into the Spiderwiz framework. It is not as capable and trustable as Kafka – for instance it does not guarantee the order of the data feed – but it is sufficient in many cases, and in some cases even preferable because lost data objects are recovered in the background without interfering with the prompt delivery of live data.
In fact the Spiderwiz solution is so simple that it requires only one extra character. All you have to do in order to get lossless data objects is to append the ‘+’ character to the Object Code that you list in Main.getConsumedObjects(). We are going to show an example of it.
As a use case for this technology we picked the Chat Archiver of Lesson 8.
Recall that this application listens to the entire chat activity, records it
and replies to history requests by the chat applications. The way we did it,
chat activity cannot be recorded when the Archiver is not up and running. We
are going to amend the system so that ChatMessage
objects will be considered “lossless”, which requires a backup mechanism to
keep the objects when the Archiver is not active and recover them when it is up
again.
As we have just mentioned, in theory all we have to do is to
append the ‘+’ sign to ChatMessage.ObjectCode
that is an element in the list that getConsumedObjects()
returns. But before doing that there is a small architectural problem we need
to tackle. The way we did it before, the Archiver joins every room that it is
made aware of so that the routing mechanism of the chat applications includes
it as a destination for every message. Obviously in order to join a room the
Archiver needs to be active. Since now we want it to be a destination for all
chat messages even when it is down, we need to find another solution.
Our solution is to use the appParams
parameter of filterDestination() like we did in Lesson 4. The Archiver will override getAppParams() to map “role
” to “admin
”. The chat applications will route messages to applications with this mapping whether or not they are in the same room.
One more issue we have to remember is that in the previous
lesson we added ChatApp
between ChatRoom
and Chatter
and used the user name as the object ID of Chatter
. The Archiver shall be amended
accordingly.
So here is ArchiverMain
:
package org.spiderwiz.tutorial.lesson12.archiver; import java.util.HashMap; import java.util.List; import java.util.Map; import org.spiderwiz.core.DataObject; import org.spiderwiz.tutorial.objectLib.ChatApp; import org.spiderwiz.tutorial.objectLib.ChatHistoryQuery; import org.spiderwiz.tutorial.objectLib.ChatMessage; import org.spiderwiz.tutorial.objectLib.ChatRoom; import org.spiderwiz.tutorial.objectLib.Chatter; import org.spiderwiz.tutorial.objectLib.ConsoleMain; /** * Provides the entry point of the application. Initializes and executes the Spiderwiz framework. */ public class ArchiverMain extends ConsoleMain { private static final String ROOT_DIRECTORY = ""; private static final String CONF_FILENAME = "archiver.conf"; private static final String APP_NAME = "Chat Archiver"; private static final String APP_VERSION = "Z1.01"; // Version Z1.01: Initial version static final String MY_USERNAME = "_chat_message_archiver"; // Use this as the "chatter" name when joining a room /** * Class constructor with constant parameters. Call super constructor and create the user map. */ public ArchiverMain() { super(ROOT_DIRECTORY, CONF_FILENAME, APP_NAME, APP_VERSION); } /** * @return the class instance as ArchiverMain type. */ public static ArchiverMain getInstance() { return (ArchiverMain)ConsoleMain.getInstance(); } /** * Application entry point. Instantiate the class and initialize the instance. * * @param args the command line arguments. Not used in this application. */ public static void main(String[] args) { new ArchiverMain().init(); } /** * @return an empty list */ @Override protected String[] getProducedObjects() { return new String[]{}; } /** * @return the list of consumed objects */ @Override protected String[] getConsumedObjects() { return new String[]{ ChatMessage.ObjectCode + ChatMessage.Lossless, // Listens to every message for archiving.in lossless mode ChatHistoryQuery.ObjectCode // Replier of this query. }; } /** * Add implementation classes to the object factory list of this application. * @param factoryList */ @Override protected void populateObjectFactory(List<Class<? extends DataObject>> factoryList) { super.populateObjectFactory(factoryList); factoryList.add(ChatRoom.class); factoryList.add(ChatApp.class); factoryList.add(Chatter.class); factoryList.add(ChatMessageArchiver.class); factoryList.add(ChatHistoryQueryReply.class); factoryList.add(ArchivedUser.class); factoryList.add(ArchivedChatItem.class); } /** * @return a map with 'role'->'admin' mapping */ @Override public Map<String, String> getAppParams() { return new HashMap<>() { { put("role", "admin"); } }; } /** * Process an input line - nothing to process in our case. * @param line The input line * @return true */ @Override protected boolean processConsoleLine(String line) { return true; } }
Chatter
is not
produced any more (line 51) because we do not need to report that the
application joins a room. For the same reason ChatRoom
is not consumed (line 59). Also since we can now fetch the
chatter name from the Chatter
object
ID, we can figure it out from the ChatMessage
identification hierarchy and we do not need to consume Chatter
any more.
The specification of ChatMessage.ObjectCode
+ ChatMessage.Lossless in line 61 means that ChatMessage
is consumed lossless (the constant value of ChatMessage.Lossless
is the plus symbol
mentioned above).
We need to register the new ChatApp
class in populateObjectFactory() (line 74).
And finally as we said, we override getAppParams() (line 86) to map “role
” to “admin
”.
There are two tiny modifications in ChatMessageArchiver
:
package org.spiderwiz.tutorial.lesson12.archiver; import org.spiderwiz.tutorial.objectLib.ChatMessage; import org.spiderwiz.zutils.ZDate; /** * Implementation class of ChatMessagee for the archiver. */ public class ChatMessageArchiver extends ChatMessage { /** * Every arriving message is encapsulated in ArchivedChatItem and archived. * @return true */ @Override protected boolean onEvent() { try { ArchivedChatItem item = ArchiverMain.createTopLevelObject(ArchivedUser.class, getParent().getObjectID()). createChild(ArchivedChatItem.class, null); item.archive(ZDate.now(), getParent().getParent().getParent().getObjectID(), getMessage()); } catch (Exception ex) { ArchiverMain.getInstance().sendExceptionMail(ex, "When trying to archive a chat item", null, false); } return true; } }
In line 19 we take the chatter name from the object’s parent
ID rather than from its name
property.
In line 21 we take the room name from the object’s great-grandparent instead of
its grandparent.
We do not need ChatMessageArchiver
anymore so we delete it.
That’s all for Chat Archiver. There is no change in its configuration file.
The chat application from the previous lesson remains almost
the same, except of ChatMessageImp
:
package org.spiderwiz.tutorial.lesson12.chat; import java.util.Map; import java.util.UUID; import org.spiderwiz.tutorial.objectLib.ChatApp; import org.spiderwiz.tutorial.objectLib.ChatMessage; import org.spiderwiz.tutorial.objectLib.ChatRoom; import org.spiderwiz.tutorial.objectLib.Chatter; /** * Consumer implementation of the ChatMessage class */ public class ChatMessageImp extends ChatMessage { /** * Called when a chat message is received. Check if the message has been sent in the room we are in, * then print the sender name and the message. * @return true to indicate that the event has been handled. */ @Override protected boolean onEvent() { if (ChatMain.getInstance().isSameRoom(getParent().getParent().getParent().getObjectID())) { String name = ((Chatter)getParent()).getName(); if (name == null) name = getParent().getObjectID(); System.out.printf("%1$s: %2$sn", name, getMessage()); } return true; } /** * Restrict message sending to: * 1. applications that are not the current application. * 2. applications that are on the same chat room as the current user, or: * 3. applications whose 'appParams' contains the mapping 'role'->'admin' * @param appUUID destination application UUID * @param appName destination application name * @param userID destination user ID * @param remoteAddress remote network address * @param appParams application parameters of the destination application * @return true if the object should be sent to this destination */ @Override protected boolean filterDestination(UUID appUUID, String appName, String userID, String remoteAddress, Map<String, String> appParams ) { return !ChatMain.getInstance().getAppUUID().equals(appUUID) && ( isMemberOfMyRoom(appUUID) || appParams != null && "admin".equals(appParams.get("role")) ); } /** * Check if the user running the application whose UUID is given is on the same room as this message is sent to. * @param appUUID Application UUID to check * @return true if the application is in the same room as us. */ private boolean isMemberOfMyRoom(UUID appUUID) { try { // The room is the great-grandparent of the message ChatRoom chatRoom = (ChatRoom)getParent().getParent().getParent(); return chatRoom.getChild(ChatApp.class, appUUID.toString()) != null; } catch (NoSuchFieldException | IllegalAccessException ex) { // Arriving here if a ChatApp does not contain ObjectCode static field or the field is not public. // In this case, send a command exception message. ChatMain.getInstance().sendExceptionMail(ex, "Cannot instantiate ChatApp", null, false); return false; } } }
The change is in filterDestination() (line 43). As mentioned, we check if appParams
parameter is not null
and contains the “role
” -> “admin
” mapping. If it does, the message is delivered to the application even if its user is not in the same room.
Before running the test we need to modify the configuration files of the chat applications, as follows:
[application name]Chat 1 [log folder]/tests/Chat1/Logs [backup folder]/tests/Chat1/Backup [history file]/tests/Chat1/history.dat [producer-1]websocket=localhost:90/MyHub
[application name]Chat 2 [log folder]/tests/Chat2/Logs [backup folder]/tests/Chat2/Backup [history file]/tests/Chat2/history.dat [producer-1]websocket=localhost:90/MyHub
[application name]Chat 3 [log folder]/tests/Chat3/Logs [backup folder]/tests/Chat3/Backup [history file]/tests/Chat3/history.dat [producer-1]websocket=localhost:90/MyHub
Each of the files contains two new properties – backup folder
and history file
. The first specifies the folder where lossless objects
that are destined to offline applications are saved. The second specifies the
name of the file where information about peer applications is saved so that it
is available when any of them becomes offline. The defaults of these
properties, if they are not explicitly defined, are backup
and history.dat
respectively under the application root folder. In our case we run multiple
instances of the chat application and we want each instance to have its own backup
and history location, so we define them accordingly.
To test the implementation do the following:
- Run the Chat Archiver.
- Run the chat applications.
- Type a few messages in each chat application.
- Terminate (by typing
exit
) the Archiver. - Type more chat messages.
- Terminate one or more chat applications.
- Restart it.
- Type more messages.
- Restart the Archiver.
- Type more chat messages.
- Type
!history
in one or more chat applications.
You will see that full message history is displayed including messages that were typed when the Archiver was offline. They may not be in the correct order, though, but they will be there and that is what we wanted to demonstrate.
With this lesson we have fully covered the Spiderwiz Programming Model. In the next chapter we will take a look at some general features that help programmers and software maintenance staff get the most out of it – monitor execution, discover bugs, get notifications and solve problems.