Typically, most processing requires, that the input stream in temporal sorted. Odysseus can also handle out of order processing.

Access

The access operator can have a flag, that indicates, that the data could be out of order. 


dnsinput = ACCESS({
                transport = 'File',
                wrapper = 'GenericPull',
                protocol = 'Line',
                datahandler = 'tuple',
                source = 'dnsinput',
                options =[['directory','${DIRECTORY}']],
                schema=[['line','String']],
                OUTOFORDER = 'true',
                REALTIMEDELAY = 1000,
                APPLICATIONTIMEDELAY = 1000 
               }                        
            )

When setting OUTOFORDER=true: this means there could be elements delivered by the source that are not in temporal order.

Because sometimes order is required, this leads automatically to the introduction of a Heartbeat operator after the access operator. This operator is configured with some default behaviour regarding REALTIMEDELAY and APPLICATIONTIIMEDELAY. If you want different values here, you could configure them in the Access operator, too (as in the example above).

So the output of this simple operator leads to:

Operators

Odysseus automatically handles operators that cannot cope with out of order elements (e.g. aggregations) and would insert a ReOrder operator.

Each operator can be configured for out of order processing:

  • assureOrder: Default is true, can be set to false, if the operator should not care for order. In this case, e.g. the Join operator or Union operator, can process elements with less latency, as the elements need not wait until they are sure in order again.
  • No labels