Creating a Natural User Interface to access PI and PI AF data with Kinect and BING maps

Blog Post created by MichaelvdV@Atos on Sep 16, 2011

If you don't want to read on the background of this demo, and go right to a demo video: click here. (watch in 720P, fullscreen, audio on)


About two months ago, I published this blogpost. After a blogpost by Rhys, where he said that it would be very important for us to be able to wave at PI :)... So, an onofficial competetion started up (still waiting for you Rhys.... ;-) ).


This is a second version of a little research project of integrating Kinect, PI and PI AF. You can find the first demo here. This demo also makes use of the BING Maps Geocode service, and the BING Maps WPF Control.


The scenario is that there is a fictional Renewable Energy company that operates windfarms in different countries across Europe. This application lets an employee navigate the different countries, and dive into the operations of the windfarms up to the turbine level. By pointing left, right, up and down you can navigate and quickly get an overview of the operation of all the countries, windfarms and turbines.


Here is a general overview off the application:




The caroussel control on top navigates trough the different countries by pointing left and right. There are KPI's displayed of the selected item. If you select a country and point down, you drill down and can view the different windfarms in that country. Every selected asset is being displayed on the map control, and the map automatically zooms in and navigates to the location of that asset.


Here is an example of drilling down to a windfarm in the Netherlands, located near the city of Groningen.




 And here is an example of drilling down to the wind turbine level




 At the heart of this application is an AF database, with an element tree representing our different assets




As you can see, the countries are on the top level, they contain windfarms, and windfarms contain turbines. There is no 'depth' limit build into the application logic, so it is possible to create hierarchies that are very deep. The application also does not contain any logic specific to windfarms, it would be easy to configure this application to support oil platforms, solar, chemical plants, etc. just by changing the AF configuration.


The KPI's displayed in the application are inferred from the attributes on the element. There are 3 mayor catagories that are now supported. Attributes decorated with the 'History' category will show up as trends, Attributes decorated with the 'ViewInTable' category will be displayed in a table.


The 'DisplayConfiguration' category supports two attributes: Image and Location. Image should have a path to an imagefile (local file or URL) which represents the current asset (country flag, picture of the windfarm or turbine). Location supports a location line, which can be geolocated using the BING geolocation service. The geolocation service will translate it to longitude and latitude information, which can be placed on the map using the pushpins.


Here is a video of the application at work (with comments by your truly): http://www.youtube.com/watch?v=2E2nELzEWqY (watch in 720P, fullscreen with audio on).


I'm working on a 'general' library to support Kinect, and abstract some of the SDK logic to make it more easy for us to start developing and researching NUI (Natural User Interface) applications for PI. Stay tuned, I'm hoping to provide an update on this soon...