Has anyone tried to connect PI Web API Channels with Apache Spark's (i.e. Databricks) Structured Steaming (Structured Streaming Programming Guide - Spark 2.3.0 Documentation ) successfully?
We're thinking to use Databricks for all PI data ETL process which is currently done by a Python program, but I cannot figure it out how to feed PI data (via Web API Channels) to Databricks streaming.
I found OSIsoft published a Python client for PI Web api (GitHub - osimloeff/PI-Web-API-Client-Python: PI Web API client library for Python generated using the Swagger specificat… ). Not sure if I could leverage this library with Databricks' streaming.
Much appreciate if anyone could shed some light on this topic.