RJKSolutions

No calculation timestamp for outputs of Abacus

Discussion created by RJKSolutions on Nov 7, 2013
Latest reply on Nov 11, 2013 by RJKSolutions

I've talked on and off for a bit with Steve on this topic but I still find myself wanting something that Abacus isn't providing yet the Performance Equation scheduler does. A timestamp option for outputs of the time of calculation, and not the timestamp of the input. A bit worried I'm sat in my own bubble and nobody else cares about this flexibility so looking for everyone else's opinion. Steve hope its okay to post this...

 

Right now if I create an Analysis that is naturally scheduled then it will use the timestamp of the trigger input for the output timestamp to keep results aligned to inputs. What if you don't want that behaviour though, you just want the timestamp of when the calculation occurred - for backfilling I can accept using the input timestamps but as soon as it catches up I would expect it to revert to the calculation timestamps.

 

If your calculation results are being used in real-time that a connected application then needs to react to such results then any post-analysis would show a difference between the calculation timestamp and the "reaction" timestamp of the connected application. So late arriving data (buffered for example) would always appear based on timestamps to have calculated perfectly aligned with the inputs. There will be occasions where you need to know when a calculation did execute, and I don't know right now if we even have any logging to switch on (or sufficient performance counters) to capture calculation cycle overruns, frequently late arriving data ...

 

Whilst I'm ranting, I don't see a way to configure a time out period for the 'Any Input' arrival of data for inputs that are not synchronised - I believe there is a default embedded of 0.5 seconds, right? Similar to how StreamInsight can facilitate late arriving data. Whilst talking about StreamInsight, that does provide the flexibility to use the input timestamps or the calculation timestamp

Outcomes