When following a statistical course with R, I encountered the scatterplot matrix for the first time. The new custom symbol feature allowed me to create a similar statistic tool for Coresight. In addition this Hackathon is a great way to contribute to the community.
This tool will give you a visual overview for detecting data that might be related. Questions it will answer are:
- Is there a visual pattern that suggest correlation? How strong is this correlation?
- What does a regression line for XY look like, and what is the slope and intercept?
- What are the basis statistics for the data added to the plot? (mean, standard deviation, skewness)
- What effect has filtering the outliers on all above this?
Just for demonstrating purposes (not for the data) I picked the Mineral Processing example in AF, see screenshot below:
- No dependencies on external services or tooling like R or Azure ML or some external Rest API. It just works directly in Coresight with Asset Framework (or direcly PI data archive ) data exposed by the PIWebAPI.
- Remove outliers with the Mahalanobis distance and choose how strong this filter should be applied.
- Fit a linear regression line for each scatter plot and show the formula in a tooltip with the slope and the intercept
- Calculate the Pearson correlation and or the Spearman correlation
- Show basic statistics
- Auto scale the plot based on the width and the number of data sources added.
- Interpolation of data over given time range and interval. (In this example above the time range of 24 hours is equally divided by 184)
- Show tooltip with the data source path and data values
The above features in action are shown in the following video (Best viewed full screen with quality 1080p):
Future extensions that could be interesting:
- Add more advanced data retrieval functions
- Add Hodrick Prescott filter for detrending the data
- Add Kendall correlation
I hope you like this symbol and if you have more idea's please leave a reply.
Senior Software Engineer
Magion Industrial Software Solutions