Package org.spiderwiz.core

Class DataObject

java.lang.Object
org.spiderwiz.core.DataObject
Direct Known Subclasses:
QueryObject
@WizObject
public abstract class DataObject
extends Object
A base class for all Data Objects.

The Spiderwiz framework paradigm is based on the idea that all the data that an application needs is already shared to it through the network. The shared data has a 3-dimensional structure. The atomic unit is a Data Object. Data objects are arranged in a hierarchy. Data objects may change along the time. The applications that are connected through the network, we call them Data Nodes, may manipulate the object tree or modify data objects (i.e. act as Producers), and the changes are automatically and instantly shared throughout the network, reaching data nodes that declare themselves as Consumers of specific data object types.

Typically, classes that extend DataObject define Properties, specified as class fields. Property fields shall be serializable and annotated by @WizField (see the annotation description for what it takes to be serializable). Producers change data object properties and commit the changes. Consumers respond to events, examine the (modified) properties and act accordingly.

Object Code

Classes that extend this class must define a public static String field named ObjectCode that identifies the data object type. The value assigned to this field must be unique across the service mesh. Object codes may contain any character except comma (','). ObjectCode values are used in methods such as Main.getProducedObjects(), Main.getConsumedObjects() and getParentCode().

A class may extend a data object class that has a defined type without redefining the ObjectCode field. The subclass may even add property fields. The two classes are considered as two versions of the same data object type, which is valid. However multiple versions of the same data object type shall be defined in a row, i.e. a version must contain all the property fields of the previous version. Versions cannot fork in a way that two subclasses extend the same superclass and each of them adds different property fields. If this happens each of the forking versions must have a unique ObjectCode value.

The class has methods for:

  • Getting general properties of the object.
  • Manipulating the object tree.
  • Committing object changes.
  • Overriding by subclasses to provide event handlers.
  • Custom object serialization/deserialization.
  • Some other actions that can be done on a data object.

  • Field Details

    • Lossless

      public static final String Lossless
      Lossless delivery marker.

      Append this value to an object code you return in Main.getConsumedObjects() to signal that lossless delivery is required for that object type.

      When lossless delivery is requested, Spiderwiz framework constructs a buffering/acknowledgment mechanism that guarantees lossless object delivery of the specific data object type from producers to the consumer that requests it. When a producer detects that acknowledgments for one or more objects have been skipped, it resends the skipped objects. The mechanism works end-to-end between the producer application and the consumer application, regardless of the route it takes between them.

      Lossless mechanism remains active even when the consumer application goes offline. If the producer notices that the consumer of a lossless object has gone offline, it keeps buffering the produced objects until the consumer application comes back to life. The mechanism expires and data is lost after the consumer application disappears for over 24 hours.

      Note that the lossless mechanism is orderless. Skipped objects may be sent out of order. If you need the data in guaranteed order, use an appropriate transport mechanism such as Apache Kafka (for which you would need a Channel based plug-in).

      Example: 

          @Override
    protected String[] getConsumedObjects() {
        return new String[]{
            MyLossyObject.ObjectCode, MyLosslessObject.ObjectCode + DataObject.Lossless, 
        };
    }
      See Also:
      Constant Field Values

  • Constructor Details

  • Method Details

    • getParentCode

      protected abstract String getParentCode()
      Returns the object code of the parent of this data object.

      An abstract method that must be implemented to return the Object Code of the parent of this object. If this is a top level object, return null.

      Returns:
      the object code of this object's parent, or null if this is a top level object.

    • isDisposable

      protected abstract boolean isDisposable()
      Returns true if the object is disposable.

      An abstract method that must be implemented to determine whether this data object is disposable, i.e. shall not be stored in the application's data object tree. Disposable objects are discarded after their onEvent() and onAsyncEvent() methods are executed.

      Returns:
      true if and only if the object is disposable.

    • isUrgent

      protected boolean isUrgent()
      Returns true if this is an urgent data object.

      Override this method to mark the object as urgent. Urgent objects have priority over other items circulated by the framework, and they are also flushed to the network immediately when transmitted. The default implementation returns false.

      Returns:
      true if and only if this is an urgent object.

    • isCaseSensitive

      protected boolean isCaseSensitive()
      Returns true if the object ID is case sensitive.

      Override this method if IDs of this data object type are case insensitive, i.e. if their case shall be ignored in object lookups. By default object IDs are case sensitive.

      Returns:
      true if and only if the object ID is case sensitive (the default).

    • getThreadAllocation

      protected int getThreadAllocation()
      Get the number of execution threads to allocate for asynchronous event processing of objects of this type.

      Upon application startup, the framework allocates execution threads, separately for each Data Object type, for asynchronous event processing of objects of this type. By default the number of threads for each type is the number of CPUs in the machine. You can override this method to return a different value. Possible return values are: a negative value to use the default, zero to eliminate asynchronous processing and process each object synchronously, and a positive value to set the number of threads.

      Note that if multiple threads are allocated then the order of event processing cannot be guaranteed. For this reason, if a certain object type is consumed in lossless mode, the number of threads that are allocated to process it will never be more than one, regardless of the value returned by this method, since lossless delivery mandates orderly processing.

      Returns:
      a negative value to use the default, zero to eliminate asynchronous processing and process each object synchronously, and a positive value to set the number of threads.

    • getOriginUUID

      public final UUID getOriginUUID()
      Returns the UUID of the object producer.
      Returns:
      the UUID of the application that originated the object.

    • getUserID

      public String getUserID()
      Returns the user ID attached to the object.

      A data object may have an associated user ID. The user ID may be set manually by setUserID() or may be set by the framework when the object is received over a network channel that has an associated user ID (see application's configuration file how to associate a user ID to a network channel). Use this method to get the associated user ID and deal with it programmatically.

      Returns:
      a user ID or null if none is associated.

    • setUserID

      public void setUserID​(String userID)
      Attaches a user ID to the object.

      A data object may have an associated user ID. The user ID may be set manually by this method or may be set by the framework when the object is received over a network channel that has an associated user ID (see application's configuration file how to associate a user ID to a network channel).

      Parameters:
      userID - the user ID to attach to the object.

    • getObjectID

      public String getObjectID()
      Returns the object ID.

      An object ID identifies an object among all children of a specific parent object that are of the same type. By default the ID of a data object is an empty string. You can override this method to return null if no object of this type has a key.

      Note that object IDs can be case sensitive or case insensitive. By default they are case sensitive, but you can override isCaseSensitive() to make them insensitive.

      Returns:
      the object ID.

    • getParent

      public final DataObject getParent()
      Returns the parent object of this object.

      Get the parent of this object in the data object tree. If this is a top level object, the method returns null.

      Returns:
      the parent object of this object or null.

    • getChild

      public <T extends DataObject> T getChild​(Class<T> type, String id) throws NoSuchFieldException, IllegalAccessException
      Returns a child of this object with specific class type and object ID.
      Type Parameters:
      T - class type of the object you are looking for.
      Parameters:
      type - the Class of the object you are looking for. The class must contain or inherit a public static String field named ObjectCode.
      id - the ID of the child you are looking for.
      Returns:
      if a child object with the specified class type and ID is found then return the object, otherwise return null.
      Throws:
      NoSuchFieldException - if the class type does not contain a static ObjectCode field.
      IllegalAccessException - if ObjectCode field of class type is not public.

    • createChild

      public <T extends DataObject> T createChild​(Class<T> type, String id) throws NoSuchFieldException, IllegalAccessException
      Creates a child data object.

      The method instantiates an object of the specified type, assigns it the specified ID and adds it to the object tree as a child of the current object. The object is added locally. To share it on the network call commit() on the newly created object.

      Type Parameters:
      T - class type of the object you want to create.
      Parameters:
      type - a Class object that must contain or inherit a public static String field named ObjectCode that indicates the type of the created object. The exact type that will be created is determined by your implementation of Main.populateObjectFactory().
      id - the ID of the required child, or null if this is a single child.
      Returns:
      the created object, or null if the object could not be created or its parent code is not equal the current object code. If a child of this type and ID already exists return the existing object.
      Throws:
      NoSuchFieldException - if the class type does not contain a static ObjectCode field.
      IllegalAccessException - if ObjectCode field of class type is not public.

    • remove

      public DataObject remove()
      Removes the object.

      Call this method to remove the object from the data object tree. The method removes the object locally. To share the removal with consumers on the network call commit() after calling the method.

      Returns:
      the removed object. If the object is not removable because it is disposable or it has already been removed then return null.

    • rename

      public DataObject rename​(String newID)
      Renames the object.

      Call this method to modify the object ID. If an object with the required ID already exists, the current object is not renamed and the method returns null. If the object is renamed successfully, the method returns an obsolete object that retains the old ID of the renamed object. The method renames the object locally. To share the action with consumers on the network call commit() on the returned object.

      Note that if you modify the object after renaming it you will need to call commit() on the renamed object in order to share the modifications on the network after you call the method on the obsolete object to rename it.

      Parameters:
      newID - new object ID
      Returns:
      the obsolete renamed object or null if the object could not be renamed.

    • getFilteredChildren

      public <T extends DataObject> List<T> getFilteredChildren​(Filter<T> filter)
      Returns a collection of all object offspring of a given type that adhere to given rules.

      This method is used to traverse the data object branch under this object and collect objects that are of specific type and adhere to specific rules. The type and rules are determined by the Filter object provided as a parameter to this method.

      The method works as follows. First, it checks if the object has direct children of the specified type. If it does, it walks through all of them, applies the filtering methods defined in the Filter object and collects the objects that pass through it. If the current object does not have direct children of the specified type, the method is applied recursively on all the children regardless of their type.

      Hint: while the object offsprings are traversed using your Filter implementation, you can manipulate the objects that are walked through even if they do not pass through the filter.

      Type Parameters:
      T - class type of the specified filter.
      Parameters:
      filter - an implementation-specific extension of the Filter object.
      Returns:
      a collection of objects of the type of the filter that pass through the filter.

    • commit

      public void commit()
      Commits object changes and distributes it to all consumers.

      Call this method after you modify object properties, add a new object with createChild(), remove it with remove() or rename it with rename().

      By default the committed object is distributed to all the consumers of this object type. However, you can restrict the distribution by implementing filterDestination().

    • commit

      public void commit​(String destinations)
      Commits object changes and distributes it to the provided list of destinations.

      Use this version of the Commit method to target distribution to applications whose UUIDs are included in the list defined by the destinations parameter.

      If destinations is an empty string, the object will not be distributed to other applications but will still be exported to foreign systems if exportObject() is overridden and returns a non-null value.

      Note that you can restrict distribution further by implementing filterDestination().

      Parameters:
      destinations - a list of UUID values concatenated by a semicolon (';'). A null value indicates unrestricted distribution.

    • filterDestination

      protected boolean filterDestination​(UUID appUUID, String appName, String userID, String remoteAddress, Map<String,​String> appParams)
      Filters a destination of object distribution.

      Override this method if you want to restrict object distribution by matching object and destination properties. The method is called once for each potential consumer application. You can examine application information and determine whether or not to approve the distribution to the candidate application. The information is provided in the method parameters as follows:

      Parameters:
      appUUID - application UUID.
      appName - application name.
      userID - the user IDs attached to the network channel through which the destination application is connected to the current application. (see application's configuration file how to associate a user ID to a network channel). In case of multiple user IDs (when connecting the same application through multiple connections with different user IDs), the value of this parameter is a comma-separated concatenations of all IDs. If no user ID has been specified then the value is null.
      remoteAddress - remote address of the destination application.
      appParams - application parameter map as set by Main.getAppParams() method of the destination application. May be null if the destination application did not define any parameters.
      Returns:
      true to approve distribution to this application, false to deny it. The default implementation of the method always returns true.

    • preDistribute

      protected void preDistribute()
      A hook for performing an action before the object is distributed.

      Override this method if you want to perform an action, e.g. lock a resource, before the object is distributed over the network.

    • postDistribute

      protected void postDistribute()
      A hook for performing an action after the object is distributed.

      Override this method if you want to perform an action, e.g. unlock a resource, after the object is distributed over the network.

    • onEvent

      protected boolean onEvent()
      Handles an object event.

      An object event occurs when a producer commits the object. If you are a consumer of the object, override this method to handle the event.

      This method handles the event synchronously, i.e. on the same thread that reads the object from the communication line. Override onAsyncEvent() to handle the event asynchronously.

      Returns:
      true if the object is handled successfully and further asynchronous handling shall not take place. The default implementation of this method returns false.

    • onAsyncEvent

      protected boolean onAsyncEvent()
      Handles an object event asynchronously.

      An object event occurs when a producer commits the object. If you are a consumer of the object, you may override this method to handle the event.

      This method handles the event asynchronously, i.e. on a different thread than the one that reads the object from the communication line. Override onEvent() to handle the event synchronously.

      Returns:
      true if the object is handled successfully, false if not. In the latter case, if the object is consumed in lossless mode its reception will not be acknowledged and that will cause the producer to resend it.

    • onNew

      protected void onNew()
      Handles a new object event.

      This method is called when a new object is added to the data object tree. Override it if you need to do something in this case.

      The method is called in addition to onEvent() and onAsyncEvent() that are called whenever an object is committed.

    • onRemoval

      protected boolean onRemoval()
      Handles an object removal event.

      This method is called when the object is removed from the data object tree. Override it if you need to do something in this case.

      Returns:
      true to confirm the removal. If for any reason you want to refuse it and keep the object in this application's space.

    • onRename

      protected void onRename​(String oldID)
      Handles an object rename event.

      This method is called when the object ID has been changed. Override it if you need to do something in this case.

      Parameters:
      oldID - the old object ID.

    • serialize

      protected String serialize​(boolean resetting) throws Exception
      Serializes the object.

      Object serialization is the process of converting object properties into a string that can be delivered over the network. You may customize the default serialization process by overriding this method.

      If you override this method you may also need to override deserialize().

      Parameters:
      resetting - true when the object is serialized during a reset process.
      Returns:
      a string that represents object properties.
      Throws:
      Exception

    • deserialize

      protected DataObject deserialize​(String fields) throws Exception
      Deserializes the object.

      Object deserialization is the process of restoring object properties from a string that was delivered over the network. You may customize the default deserialization process by overriding this method.

      If you override this method you may also need to override serialize().

      Parameters:
      fields - a string that represents the serialized object properties.
      Returns:
      this object if deserialization was done successfully and an object event shall be fired, null otherwise.
      Throws:
      Exception

    • onlyForMe

      protected boolean onlyForMe()
      Returns true if and only if a received object shall not be propagated to other consumers.

      Override this method to return true if you process the object exclusively and do not want it to be forwarded to other data nodes.

      Returns:
      true if and only if a received object shall not be propagated to other consumers. The default implementation always returns false.

    • exportObject

      protected Object exportObject​(ImportHandler channel, String newID)
      Exports the object.

      Exporting a data object is done by converting the object to a form that can be interpreted by the handler of a specific export channel. If, for instance, the handler expects data that is serialized into strings, the exporting involves serialization into a format recognized by the handler. Override this method to implement object conversion that is specific to the channel governed by the channel parameter, which you can use to retrieve information about the channel.

      This method is called when an object is committed. The newID parameter indicates the situation that triggered the commit. If it is null then this is a normal commit. If it contains an empty string then the object has been removed. If it contains a non-empty string then the object has been renamed and the parameter contains the new name.

      Parameters:
      channel - the handler of the channel the object will be exported to.
      newID - null if this object is active, empty string if it has been removed, non-empty string if it has been renamed.
      Returns:
      the converted object to export, or null if the object shall not be exported over the given channel.

    • importObject

      protected String[] importObject​(Object data, ImportHandler channel, ZDate ts) throws Exception
      Imports the object from foreign data.

      When data is imported from a foreign system, the framework creates a temporary instance of each data object type that the application produces and calls this method to let the instance determining whether the data is relevant for this type. If it is, the method sets object properties from the imported data and returns a non-null value, which tells the framework to merge the object with the shared object tree.

      Override this method to implement object conversion that is specific to the import channel governed by the channel parameter.

      If the imported data is relevant to this object type then the method shall return a string array that contains the key hierarchy of the object. The first element shall contain the ID of the top ancestor of the object (excluding the root object) and the last element shall contain the ID of the object itself. If the IDs of the object as well as all its ancestors are null then you can return an empty array.

      If the imported data is not relevant to this object type then return null.

      If the imported data marks the deletion of an object with a certain key hierarchy then call remove() on this object and return the key hierarchy.

      Parameters:
      data - the imported data object.
      channel - the handler of the channel the data is imported from.
      ts - the timestamp attached to the data by the channel handler.
      Returns:
      the key hierarchy of the imported object, or null if the imported data is not relevant to an object of this type.
      Throws:
      Exception

    • getArchivePath

      protected String getArchivePath()
      Returns the pathname pattern of the files where objects of this type are archived.

      Spiderwiz framework has a mechanism for archiving and restoring data objects. Object data is compressed and stored in local disc files when you call the object's archive() method and retrieved by calling the restore() method. Note that object archiving is a background operation, designed to work efficiently without hindering the main application tasks.

      In order to use data object archiving, this method must be overridden to return a path that specifies the location in which the archived data will be stored. The returned value shall be of the form "folder/file.ext", where folder is a sub folder (direct or nested) under the archive folder defined in the application's configuration file, file is the file name and ext is an optional file name extension (the default is arc). Any name extension is valid, but if it is either .text or .txt its content is stored as plain text, otherwise it is compressed (GZipped).

      The path may contain parameters, specified by the pound character ('#') followed by the parameter type, that determine how the archived data is aggregated as follows:

      #0, #1, #2 etc.: refer to the object key where #0 is the object ID, #1 is the object's parent ID, #2 is the grandparent ID etc.

      #y, #m, #d, #h: refer to the object update time where the letter that follows the pound sign determines the time component - year, month, day and hour.

      Example:

      Assuming we develop a car navigation system. We define a data object called Vehicle that contains properties such as GPS location, speed, heading etc. We track the vehicle through its driver's cellphone, so we use the phone number to identify the vehicle. Phone numbers are grouped by country codes, so we define a data object called Country and make Vehicle a child of Country by implementing: 

          @Override
    protected String getParentCode() {
        return Country.ObjectCode;
    }

      We want to archive every object update so that we will be able to reconstruct the entire vehicle journey. So we implement for that class: 

          @Override
    protected boolean onEvent() {
        try {
            archive();
        } catch (Exception ex) {
            Main.getInstance().sendExceptionMail(ex, "when archiving", null, false);
        }
        return true;
    }

      We will now define the archive path pattern: 

          @Override
    protected String getArchivePath() {
        return "vehicles/#1/#0/#y#m#d/#h;
    }

      Let's say we run the application on January 20, 2017, from noon till 3pm, the phone number of one of the drivers that used the application was "917.756.8000", country code "1". Assuming that we used the default "archive" folder name, then after running the application all this driver's moves during this time would have been recorded in the following folders and files:

      • archive
        • vehicles
          • 1
            • 917.756.8000
              • 170120
                • 12.arc
                • 13.arc
                • 14.arc
      Returns:
      a pathname pattern or null if archiving is not used for this object.

    • archive

      public boolean archive() throws Exception
      Archives the object.

      Archives the object using the pathname pattern returned by getArchivePath().

      Returns:
      true if the object has been archived, false if an archive pathname pattern is not defined.
      Throws:
      Exception - if archiving failed.
      See Also:
      getArchivePath()

    • restore

      public static int restore​(String objectCode, Object associated, ZDate from, ZDate until, String... ids)
      Restores data objects from an archive.

      A static method used to restore a range of archived objects of a specific type from the archive folder set for that type by getArchivePath(). Each restored record will activate onRestore() on the object. Override that method if you want to process the restored record.

      The parameters of this method specify the object type and the range of the restored records. The objectCode parameter identifies the object type. The from and until parameters specify the record time range. The ids parameter list lets you select only specific records for restoration by specifying the ID of the restored object or the ID of an ancestor of a group of objects. The first ID in the list refers to the top ancestor of the object. The following IDs refer to objects down the object tree, until the object itself is referred. A null value for any ID in the list means that all objects of that level should be restored.

      Parameters:
      objectCode - the Object Code of the objects to restore.
      associated - a general associated with the restoration that will be provided in subsequent onRestore() calls. Can be null if it is not relevant.
      from - starting record time for restoration. A null value means all records until 'until'.
      until - ending record time for restoration. A null value means "now".
      ids - identify objects or object branches to restore.
      Returns:
      the number of records that were restored by this call or -1 if restoration was aborted by a value 'false' from onRestore().
      See Also:
      getArchivePath()

    • onRestore

      protected boolean onRestore​(Object associated)
      Handles a record restored from the object archive.

      This method is called when an object record is restored from the object's archive during a call to restore(). Override it if you want to handle the object that was reconstructed from the record.

      Parameters:
      associated - An object associated with the restoration by the restore() method.
      Returns:
      true if restoration should continue, false if it shall abort.

    • deleteFromArchive

      public static boolean deleteFromArchive​(String objectCode, ZDate from, ZDate until, String... ids)
      Deletes archive files.

      A static method used to delete all or selected files from a data object's archive folder, set by getArchivePath().

      The parameters of this method specify the object type and the range of the records to delete. The objectCode parameter identifies the object type. The from and until parameters specify the record time range. The ids parameter list lets you select only specific records for deletion by specifying the ID of the deleted object or the ID of an ancestor of a group of objects. The first ID in the list refers to the top ancestor of the object. The following IDs refer to objects down the object tree, until the object itself is referred. A null value for any ID in the list means that all objects of that level should be deleted.

      It is important to notice that this method deletes whole files, not individual records. If records marked for deletion comprises an entire archive file, the file will be deleted, but if only a partial file is marked for deletion, the file will remain untouched and no records will be removed from it.

      Parameters:
      objectCode - the Object Code of the objects to delete.
      from - starting record time for deletion. A null value means all records until 'until'.
      until - ending record time for deletion. A null value means "now".
      ids - identify objects or object branches to delete.
      Returns:
      true if file deletion is successful.

    • getRawCommand

      public final String getRawCommand()
      Returns the raw string from which this object was deserialized.

      When the object is received from a peer application on the network, this method returns the serialized object values. You may want to use it for debugging purposes.

      Returns:
      the serialized object values as delivered on the network, or null if the object was created locally.

    • getCommandTs

      public final ZDate getCommandTs()
      Returns the time that was stamped on the object by a peer application.

      When the object is received from a peer application on the network, this method returns the time that was stamped on the serialized object by a peer application.

      Returns:
      the timestamp that was attached to the object on the network, or null if the object was created locally.

    • cleanup

      public void cleanup()
      Cleans up object resources.

      This method is called upon application termination or whenever a data object is removed from the data object tree. You can override it if you want to do custom cleanup or data flushing before the object is removed. If you do so, do not forget to call super.cleanup().