Skip navigation
All Places > Security > Blog > 2016 > December

Welcome to the final post of the Killer Robots series!  If you don’t already know about Killer Robots, Inc., see the December 7th post for an introduction.  The December 14th post provided last year’s system architecture and some PI security resources.  Last week’s post introduced this year’s environment and modeled the traversal to the web server with an early flag.


This week, as foreshadowed, we must go deeper.  The web server access following the compromised user was possible due to poor credential theft defenses in the environment.  Credential theft was the path of least resistance due to other defenses in place, such as authenticated access from normal users being limited to the client application since the admin site is implemented separately, strict import folder access preventing access to modify imported displays and IIS hardening to ward off attacks to the platform.


To control the impact of this user’s credentials falling into the hands of our competitors, the environment is set up with delegation, point level permissions and limited access assigned to the user.  Just what they need to do their job.  Or maybe slightly more…


While we took the above precautions, in the event we underestimated our competitors or overestimated our defenses, we also have snapshots of the environment, offline backups and a disaster recovery plan to restore the environment to them with minimal downtime to other competitors.


When putting together your toolkit for the CTF, you’ll want to include the following for taking on the PI challenges:

  • A machine or VM running a Windows operating system with PowerShell available
  • Windbg or your preferred tool for analyzing memory dumps
  • Your favorite network capture tool
  • Link to key PI security references


This concludes the PI and the Killer Robots, Inc. Environment series.  For any taking on the challenge, best of luck and I’ll see you in Miami January 10th!

Hopefully you are here for more of the inside scoop on the S4x17 CTF competition!  Last week’s post provided a primer on PI Security and the system architecture for last year.  This week you will get a sneak peek at the updated environment. If you don’t already know about Killer Robots, Inc., see the December 7th post for the first installment of this series to get up to speed.

When examined from the perspective of the functional layers of the PI System, the S4x17 CTF environment utilizes the operations and business scenario, where the data from one or many disparate sites is sent to a business network. This adds another collection layer in an intermediate DMZ as well as additional management and delivery on the business side.

The diagram below shows an architecture loosely based off the Life Sciences Reference Architecture.  The blue boxes represent assets that are deployed in the virtual environment for the CTF competition and the red flags indicate the quantity of relevant flags for each component.  As indicated, the challenges span the entire environment, from OT to IT.  So many f1r3w@llz, so little time!

But, as we all know, f1r3w@llz != security, they only represent one of many aspects. Our S4x16 environment architecture had 6 steps between the external users and the control system.  The new architecture, with proper network segmentation, lengthens the attack path by 2 layers.  The added components, PItoPI and the site PI Data Archive are framed below in green.  At each of these layers there are many potential threats, and our CTF competitors need to see which ones are missing barriers that allow them to traverse to the next layer.

A skilled attacker will follow the path of least resistance.  To provide a tangible example, let’s look at Bow Tie diagrams for the first two steps of the chain to illustrate the additional dimensions that can be successfully attacked when a system does not have a well-rounded security strategy. 


In the S4x16 CTF competition last year, the competitors needed to gain access to the web server. To do so, they first needed to compromise a user account as the first flag.  While there are many methods they could use to compromise a user, enumerated in the Bow Tie on the left below, there were defenses in place for almost all of them. The environment machines had application whitelisting enabled with AppLocker, firewall rules in place to limit traffic, antivirus, all recent OS patches, among other defenses.  Although none of these defenses are absolute, they increased the difficulty of other attacks such as infecting the system with malware.  The path of least resistance was insecure communication to the web server.  Competitors got a capture of snooped network traffic, where they found a connection to a web server using Basic authentication without TLS, which is of course a huge faux pas. 


Once the credentials were obtained from the network capture, the competitors could access the web server simply by logging in.  No need to find a 0-day and the firewall allows the requests since they appear as normal traffic.  The credentials obtained were for a non-administrative user, but the competitor could now see data and some configuration information that they could recon as they worked towards pivoting to the next layer.

While basic or forms authentication without TLS is an extreme example, it does happen, and the same concept applies to other attack vectors such as phishing or mismanagement of credentials. So, even with gratuitous firewalls, an environment is not impenetrable if other aspects are ignored. To make matters worse for the Killer Robots in S4x17 (or better for humanity if you’re the “glass half-full” type) the CTF network will be exposed to the players as a cross-section, so the CTF competitors will have direct access to several endpoints to attack… 

If you enjoyed reading this penultimate post, tune in next week for the final installment where we’ll dive a layer deeper into some of the specific barriers targeted by flags and give some tips for putting together your toolkit!  In the meantime, you can also check out the S4CTF twitter feed for updates about the other challenges.

Are you looking for an edge in the PI challenges of the S4x17 CTF competition?  If so, you’ve come to the right place!  Herein lie PI security principles and architecture information that will benefit any taking on the challenge.  If you want a refresher on what the CTF event is all about and the Killer Robots, Inc. environment, see last week’s post which gave background on the competition and what is at stake.


PI Security Primer

While there have been volumes written on PI security, we have provided a distilled view of the most pertinent and frequently requested information.  The one-stop shop for quick access to all things PI security is the PI System Cyber Security page on the support site.  If you work with (or in the case of the CTF competition, against) PI, then I highly recommend bookmarking this page, which connects you to documentation, KBs, tools, alerts and more.


For a prospective competitor looking for a crash course on priority defensive measures for a PI System, KB00833 – Seven best practices for securing your PI Server is a great place to start.  In this KB and across our resources you’ll find that Windows Integrated Security is a focal point.  Adoption of this authentication protocol to the PI Data Archive over trusts and explicit login provides stronger authentication and enables features such as transport security.  You may also want to add KB01295 - Risks of using the PI System as an input for a control system to your short list for review, to familiarize yourself with the risks and defensive measures associated with output points.  The coveted Golden PI flag may not be possible otherwise....



With respect to architecture, the PI System can be logically represented as three functional layers based on the roles of components: Collection, Management and Delivery.  Each layer is represented graphically in Figure 1 below with the relevant software components at each layer.

  • Collect - PI Connectors and PI Interfaces pull in data from many disparate sources.
  • Manage - PI Server (PI Data Archive, PI Asset Framework) store, aggregate and contextualize data in a common format.
  • Deliver - PI Coresight and other access tools allow users to visualize data.

Figure 1: Functional layers of the PI System.


What a lovely graphic... but let's get to what this audience cares about.  Now that you have the functional layers and the related components, the next logical question is where they reside in a deployed system.  The OSIsoft environment submitted to the CTF competition for the S4x16 Conference in 2016 was modeled after a traditional operation scenario for the PI System, where the data is collected, managed and delivered in the OT space to be consumed by operators and engineers at a plant or facility.

Figure 2: PI System Roles in an Operations Scenario


An example architecture using this paradigm is represented below in Figure 3.  This more tangible example was taken from the Life Sciences Industry Reference Architecture.  Note that there are other reference architectures for several industries available here on PI Square, compiled based on best practices for the PI System and the needs of the specific industry.


Figure 3: Operations PI System deployment from Life Sciences reference architecture.


The highlighted components in Figure 4 are the minimum representative components that the OSIsoft team chose for the simplified architecture in the S4x16 CTF environment.  A few notes about each of the components are included below.  The flags superimposed on the image correspond to the placement of challenge flags throughout the environment.  Some of the flags deeper in the network could only be exposed after capturing flags upstream. 

  • Terminal server: provide access to PI utilities and thick clients such as PI ProcessBook.
  • Web server: host the PI Coresight web application for visualization of PI data.
  • PI Data Archive: provide access to historical and real-time data
  • PI AF Server: model physical assets to provide context to data streams, including PI AF Server as well as the PI Analysis Service
  • PI Interface node: feed data to the historian through the PI OPC DA Interface and a local OPC Server to act as a mock data source to represent information from the control system


In S4x17, we are expanding the environment to include the business network as well.  In this Operations and Business scenario, data from one or many sites, presumably separate plants or facilities, is sent to a centralized location for consumption. This architecture spans both the IT and OT networks.  The next post will show how the CTF environment architecture was updated for this scenario and start talking about kill chains


Figure 4: PI System Roles in an Operations and Business


If you are a PI System administrator, System Integrator or an otherwise security focused professional, you may be interested in the PI System environment at Killer Robots, Inc., the misanthropic company competitors will attempt to compromise at the S4x17 ICS Security Conference Capture the Flag (CTF) competition in a battle for the survival of mankind.  Aside from aiding in the struggle to liberate humanity from the merciless machines, this 3 day event is also a unique training opportunity, as it allows competitors to learn about PI security by going on the offensive on a live PI System environment submitted by OSIsoft alongside other vendors.


The OSIsoft team sought to create a PI System environment that highlights common mistakes, misconfigurations and misuse in a way that is both informative, and hopefully jarring. Inspiration was drawn from case studies, security engineering, 3rd party reports and our own experiences with vulnerability disclosure, so the exercises are grounded in practical application.


What does the CTF offer for a PI System administrator?

  • Exercise your skills in a simulated environment:  Not everyone has a development system or sandbox environment, so this is your opportunity to get a hands on experience exploring a PI System.
  • Cross-train with both IT and OT technologies: The CTF flags include targets against client applications such as the PI Coresight web application in the corporate zone, all the way back to output points associated with a PI OPC Interface in the plant network.
  • See some of the latest security features in action: Features such as transport security will be on display as well as the impacts when they are not.
  • Explore PI System internals: There are some gems hidden in the environment for the most avid PI geeks, such as some exploration into the SQL back end of PI Coresight.
  • Interact with developer technologies: Some exercises will require leveraging developer technologies such as the PI Web API.
  • Work on your OPSEC: Since many attendees are well versed in the art, you may even pick up some social engineering tricks to improve your OPSEC skills.


Throughout the month of December, we will discuss the philosophy, methodology and motivation behind the creation of the Killer Robots, Inc. PI System environment.  To a clever reader, these posts could provide a valuable primer for the competition, but to a clever competitor, the event should provide a better understanding of the PI System security.  The PI System challenges in the CTF will require a breadth of skills and knowledge from the competitors related to basic network packet and memory dump analysis, RESTful web services, PowerShell scripting, arcane ciphers, and most prominently, PI System administration. 


Tune into next week’s post for a survey of reference architecture, security engineering and best practices (or lack thereof) that informed the deployment of our target virtual PI System.  In the meantime, if this post has piqued your interest, check out the S4x17 Conference Site for more information or to register for the event.


PI Challenges at the S4x17 CTF

Posted by hpaul Employee Dec 5, 2016

Brian Bostwick posted earlier about the S4x17 ICS Security Conference (original post here) and I'd like to elaborate on the OSIsoft CTF environment.


The S4x17 Killer Robots CTF environment is designed to be an interactive, fun source of industrial security challenges.  After all, CTF is a great way to explore and defeat ‘forever’ day configuration issues. This year the OSIsoft team has improved and expanded the PI System environment, planting flags inspired by case studies, new security features and threat models.


Below we have a summary of the PI challenges from last year. OSIsoft provided 11 of the 43 total flags for the competition.  There were 5 flags left standing at the end of the competition and 4 flags that were only solved by one team.  The most successful competitor captured 450 of the possible 2025 points from the PI challenges.



Reviewing the logs in our environment revealed that many teams did perform reconnaissance, but did not progress.  Perhaps the low success rate of the competitors has gone to our heads, so this year we are upping the ante.  The first (if any) team that captures the mysterious, illustrious “Golden PI” flag, will win the opportunity to deliver ~3.14 pies to the faces of the OSIsoft security advisory team in attendance.  You heard right, this is your opportunity to exact sweet revenge on a vendor!


Want to learn more? Every Wednesday in December we’ll give an inside look at the CTF environment on the PI Square Security Forum, providing background and perhaps even a few hints along the way.  Search for the S4x17 tag to get all posts related to the event in the coming weeks.


Edit: First post in the series is out. PI and the Killer Robots, Inc. CTF environment, Part 0x01