I was watching an interesting presentation the other day from the guys at SCADA Strangelove. 

https://events.ccc.de/congress/2013/Fahrplan/events/5582.html

From the names and accents they appear to be a Russian group, or at least Eastern European that find SCADA vulnerabilities.  They seem to be a pretty good group of guys as they do find vulnerabilities but also seem to encourage responsible disclosure and responsible and responsible testing on production systems.

Something that caught my eye in the presentation was this slide.

At first I thought wow.. Invensys is really sucking at getting popped for 0-days.  But then if you look closer you see that they are the only vendor that has actually FIXED all of their 0-days.  This is very consistent with the story I get from people at Invensys.  They will freely admit that some outside researchers have found 0-days.  The difference, however, is that they have taken a very proactive and responsible approach to work with the researchers to identify, study, and fix the 0-days. 

If you write software you will have bugs and holes… guaranteed.  The difference is more in how you respond to other finding these security holes.  In that respect I think Invensys has done quite well.

As a parting item, here is one specific link from the Strangelove blog where they compliment Invensys on fixing an issue quickly.

http://scadastrangelove.blogspot.tw/2013/03/icsa-13-067-02invensys-wonderware-win.html

-andy

https://github.com/aaOpenSource/aaEncryption

-andy

The Problem
We should all be backing up our galaxies regularly.  Back a long long time ago when the Archestra.biz site was up and running they posted a command line app for calling galaxy backups.  This let you schedule them or call them from other apps (like SQL Server).  Having command line access was great but it was a bit limited.

The Motivation
After looking into the GRAccess tools I saw that performing this complete backup of a galaxy was really dirt simple.  Without decoration and error handling it could probably be distilled down to less than 10 lines of code.  I thought this was nice but there are times when I want to do different kinds of backups.  So I did what any geek should do, I rolled my own.

The Delivery
So here is what you get.  You get a command line tool that allows you to make numerous different kinds of backups; single CAB, single AAPKG, specific objects into a single csv or single aapkg, specific objects into multiple csv or multiple aapkg, etc.

What can you use it for
I’ll let you decide how helpful it is but here are a few ideas.

1) Daily backup of all objects separately so if you need to restore from an old version you don’t have to mount an entire backup galaxy

2) Automated tool to export a specific set of objects on a routine basis, maybe for coordination with another galaxy (yes that’s a tease of something else I’m working on)

3) Write some kind of SQL Server script to first query all the objects that have changed since a particular timestamp.  Generate a list of objects and use that in the command line call for your daily exports (hint… I’d like to get that feature baked in on the next version too)

Stability and Is It Done?
Yeah right.. done.. not even close.  Stable?  Seems to work OK for me.  In all honesty I’m taking a cue from some reading I’ve been doing lately around the software development process.  The idea is that once you get something with decent stability and at least the core functions go ahead and get it out there for people to beat on.  Admittedly this works a lot better with a website instead of a windows app but I’m giving it a try

How do I get it?
https://github.com/aaOpenSource/aaBackup

Go to this site.  Pull the source and start to play – note you need to provide your own GRAccess DLL and IDE license.

What’s left to do?
In a word or two, a lot!  Need to add good logging, good error handling, and a whole host of additional features.

What do I need from you?

Feedback, feedback feedback.  Did it work, did it break, did it break something else, feature suggestions, etc.  Said another way, participation is what I would like from the community.

Enjoy and please let me know how it goes!

- andy

http://archestranaut.avidsolutionsinc.com/2013/01/recent-gotchas/

See item #5.  The gist (if you don’t feel like clicking) is that now with the ShowGraphic() function it is possible to dynamically call a graphic from a script.  What this means on the backend is that Archestra has no clue what graphics you may call so it has no clue what graphics it should include when it deploys.  I can only assume that as an optimization technique they probably checked all the graphic references before deployment and only send down the ones that were actually needed.  Well with dynamic references and the ShowGraphic function this is not possible now.

Enter the Associate Graphics checkbox (accessible by right clicking the InTouch view app).  I saw this today and had no clue what it did. 

Straight from the help file

Associating All Galaxy Graphics with an InTouchViewApp
Associating all Galaxy graphics with an InTouchViewApp template enables deployed and published InTouch applications to execute "show graphic" requests made of any graphic in the Galaxy without having to embed them in the application.

•  The ShowGraphic() function uses the graphic as a parameter. Associating all Galaxy graphics ensures that the graphic is deployed and available at run time whether or not it is referenced by InTouchViewApp. 

•  Associating all Galaxy graphics ensures that template symbols referenced by the ShowGraphic() function are deployed and available at run time.

Note: The term "graphic" includes any symbol or client control present in the Graphic Toolbox, and any symbols owned or inherited by templates and instances.

Voila, problem solved.  I haven’t tested it yet but hey it’s a feature so it has to work right?

I would suspect this makes for a little more bloated app but it’s easier than the alternative (presented by David in his blog post).  Obviously there might be times when you want a lightweight deployment so keep this in mind when you are trying to trim a few bytes in the future.

- andy

Back to the point of the post. How many times are you troubleshooting an issue and all you really want is a clean log viewer screen so you can start seeing the new log entries. For those of us in the know we typically use the Filter function and set the filter to only show messages after the current time. Well this is a pain especially if you are doign this over and over. The trick is to use CTRL+R when you are in the log viewer. Maybe R stands for “Right Now”? What this does is automatically set the filter to only show messages after the current timestamp, yielding a clean log screen that only shows the newest messages.

Pretty slick and a nice time saver.

Speaking of time savers with the log viewer a little birdy showed me something the other day that “might” be coming to a support package near you. I don’t want to give any details at this point because most of that is up in the air but if it does happen the people who maintain large systems will really really like it.

your loyal archestranaut,
andy

http://situation-awareness.com/

I know Wonderware has had a big push on the situational awareness concept with the upcoming release of 2014 and the Situational Awareness library.  Well it turns out this is the URL for the new E-Zine as I call it, “The Wonderware/HMI Times”.

It appears to be a collection point for lots of different content that would be interesting to a Wonderware user.  Looks pretty good to me so far.

To be honest I felt kinda bad with those guys driving some traffic my way but me not giving any love in return.. so here is the love in return

- andy

The new trick I have learned involves keeping popups visible when you change windows.

Do this quick test. 

1) Create an object with a script and a UDA that just counts from 0 to 100. 
2) Create a graphic with a simple square background and display this value.
3) On another graphic on the object create a button, using the ShowSymbol animation to show this graphic as a popup. 
4) In Intouch put the button down on a window.  
5) Using the Intouch functions,  create a button to navigate to another window, making sure the original window closes. 
6) Now start InTouch and launch your popup from the first window.  Now navigate away from that first window, making sure it closes.

What happens?  You will see that your popup closes automatically.  This behavior can be traced back all the way to the early 3.0 days, the first release with Archestra Graphics.  Originally when you navigated away from the window that the popup was launched from the popup graphic just went dead.  It stayed on the screen and looked pretty but everything just stopped working.  I have to assume that is something related to the parent or launching element not being on the screen anymore.  The solution, at the time, was to simply force all popups to close if the parent element went away.  If I remember that far back I think it was originally a hotfix but then later on became a mainline feature that just worked behind the scenes. 

The first question is why would you care if a popup stayed active during navigation.  Aren’t you going to look at a different screen with different systems anyway?  If you have ever been in a large chemical facility you know that the original screen designers do their best to group logical items together on a screen.  However, with so many complex interactions in a plant there are always times when you may want the popup for a valve or PID loop (popup/faceplate is the method by which you actually control the device) from one system to be visible and accessible while you are looking at a different upstream or downstream unit.

So if you are still with me I’ll show you a cool feature in 2014 (sorry not ready for primetime yet but it’s getting close) that fixes this issue… or at least appears to at first blush.

So let’s go back to that object we created and make another button.  This time configure the button with an action script instead of ShowSymbol.

Without going into a lesson on ShowGraphic the basic idea is that you create a new object of type GraphicInfo.  Then set some properties and launch the graphic using the built-in ShowGraphic function.   Before you continue notice the fact that my WindowType is Modeless and my KeepOnMonitor = True.  A huge thanks to autocomplete or I would have never found this little jewel.  The Modeless window is important because it allows you to click things that are not on the topmost window.

Lay this button down on your InTouch graphic and repeat the test from above.  What do you see?  Magic! The popup stays up and continues to function.  The reason I made a UDA with a script was to make sure the graphic continued getting updates from the actual engine and not just some local value on the graphic itself.

Hopefully this solves some issues that may have been nagging you.  Before I discovered this little trick I had some hairbrain schemes for allowing users to set which popups to keep displayed and keeping an array …… it was going to be really hard.  With this solution all that work is unnecessary.

Any cool ideas for some other neat uses for this new feature?

- Andy

Well I’m here to change that, in a very small way.  This past weekend I worked on a total rewrite of my LDAP authentication engine.  One of the items I was never crazy about was how long a password hung around in the UDA’s in clear text before getting cleared out.  So I did something about it.

Using some code I had laying around from another project (aaBackup) I re-purposed some of it and rolled a dll to implement AES encryption.  Before you jump me and say hey Andy, you don’t know nuthin’ ‘bout that thar encryption..you are correct.  I spent a long time looking high and low for some of the best implementations I could find, with lots of positive reviews.  The most important thing, however, was that it had to be easy to use.  I think this fits the bill.

The pseudocode goes something like this.

1) Create some kind of random string
2) Instantiate the aaEncryption object, feeding it the random string as a seed value
3) Call the .EncryptString function to take a string and create an encrypted version of it
4) Call the .DecryptToString to take the encrypted value and decrypt it.

That’s it.  Very Simple, very straightforward.

So, how do you the local junior archestranauts fit in?  In the spirit of open source I want to share both the C# code for the DLL as well as my example Archestra objects for testing it out.

What do I want you to do with it? Beat on it, tear it to shreds, recommend better ways to do things.. all the while keeping in mind this thing needs to stay dirt simple.  So what are some of my ideas on things to make better?

1) Have the DLL automatically add itself to the app domain when you instantiate it.  Right now I have an initialization routine that creates the object and stashes it in the app domain.
2) Clean up some dead code.  You will see some code that used a bunch of calls to create a unique fingerprint of the PC it was running on.  This was cool but it failed in two areas.  First, it used the same data to create encryption keys for two encryption instances running on different engines, but the same host.  I didn’t like this.  Second, it was really slow.  On average it took about 2 seconds to initialize, which I didn’t like.
3) Create a routine in the DLL to generate a new random seed if no seed is passed
4) Remove, or make private, some of the extra calls that we won’t use in our implementation… keep it simple
5) Find the limits on the System Platform side.  I have my SP2014 engine a number of times trying to enter really long strings to see if I could successfully encrypt then decrypt.  I should probably drop back to this testing in C# but I was feeling on the lazy side.
6) Write some good unit tests.  Maybe my hard core software friends could handle this part for us.  All I know how to do is throw together a little windows form to poke some values and push some buttons.
7) I thought about performance tests but this thing is really really fast, typically less than 5ms to encrypt, so I’m not sure it is necessary.  I think we’ll find the limits on SP2014 strings before speed ever becomes an issue.. unless you are doing a large array.
8) Help me think about the architecture for initializing the encryption engine then sticking it in the AppDomain.. is this the best method.  Should I have one instance per engine or should I have one per object to be really really really secure.
9) And the biggy, generally poke some holes in the concept and structure.  As you have already guessed I am not a security expert so I may have some gaping holes that others can identify.  Let’s make this thing better.

For now I’ll just post the aaslib, the aapkg, and the DLL source code.  I’ll chat with some of the other aaOpenSource co-conspirators to figure out how to get this hosted on a nice Git repository or something that real software developers use.

AAPKG File - Example

DLL Code - Code

aaSlib File - aasLib

Take care and …. what does the fox say?

your loyal Archestranaut

The opening session was better than average I’d say.  Senior Management (apologies for not having the exact name) dispatched the elephant in the room pretty quickly by addressing the Schneider acquisition.  Long story short is that they are working through the hard parts but they don’t expect any issue.  No big surprise.. they certainly wouldn’t tip their hand if this thing were about to go off the rails… which I don’t think is going to happen.

Next was a rapid fire set of ideas and demos that would get deeper dives later.

Finally they had the typical cheeseball rah rah technology guy trying to get us all fired out.  I think the only reason this guy could spell Invensys was because it would be on the check he got for his appearance.  So you know what’s coming next right?  Wrong.  I thought he was fantastic.  Instead of talking about the soft mushies of thinking ahead and being brilliant.. and stuff… he brought our attention to the fact that we are really just on the cusp of a massive series of changes in technology that will alter the way we function.  What made him so good was that he used facts… not ideas… to drive the point home.  Example: 6 years ago IOS apps didn’t exist.  Let that sink in for a second.  Today if a vendor doesn’t have an app or at least mobile friendly website we bypass them.  Second, and this was a but of the mushy, was how he very successfully pointed out that most of us completely miss the fact that we are right in the middle of massive changes.  He used a few very good exercises to demonstrate the concept of being so focused on the details that we miss the gorilla walking into the room.. and yes the gorilla did walk into the room, on a video.  All in all he was definitely one of the better rah rah opening session speakers I’ve seen in a while.  BTW the guys name was Jack Uldrich.

On to the sessions.  I can sum up everything in two words… Holy COW (i did have a different word but I’m trying to keep it PG).  These guys have been really really busy.  For the System Platform people, the 2014 release is going to absolutely blow your mind.  I think is easily surpasses the 2.1 –> 3.0 transition.  That was one major change, the graphics.  I’ll try to rattle off the changes they talked about but I know I will miss some:  no more alarm db logger (it’s integrated like history now),  global styling for graphics, locked exports so when the objects are imported the user can’t monkey with the template, InTouch Access Anywhere, huge efforts around situation awareness and high performance HMI (check out the book of the same name), lots of new features for better scripting (try-catch, autocomplete….), huge scalability and speed improvements in Historian (2 million tags), historian data with process context (batch, overlay alarms, operator actions, etc.).  I am sure I’ve missed a bunch but you can tell the entire wonderware software team is incredibly energized.  I wrote a post a few years ago lamenting to snail’s pace of progress with innovation on System Platform.  I wonder now if they were just sitting on the track or were they crouched down ready to explode?  I think the latter is most certainly the case.

On the more personal front I’ve been having a good time meeting old friends and making new ones.  I’ll say that coming to the conference as a customer is a very different feel for me.  As an integrator I was always scratching and clawing to get a little attention and maybe have a nice conversation with a few higher ups.  As a customer I’ve found this to be much simpler.  I don’t know if it’s just a natural confluence of events or something has materially changed.  Either way it’s been fun so far.

Speaking of fun, @everdyners I’m going to have to file a protest over what Elliot Landrum is calling drunk tweeting in your feeble attempt to win the gift card.  I’ll be coming out swinging tomorrow so watch out

Finally I would be remiss if I didn’t mention a group of folks I chatted with quite a bit today and came away duly impressed.  I got to meet with a lot of the Genentech team members from their Vacaville facility.  Long story short is they removed the SCADA and Batch management layer from an old Moore APACS system and replaced with System Platform and InBatch.  The work these people did is nothing short of amazing.  Less than 48 hours after getting their window they were pushing the facility to get up and running to so they could see their masterpiece come to life.  They were aided along the way by the fact that Direktor was the Unix precursor to InBatch so the conversion process for 1100 recipes could be automated.  However, it’s really a story of creativity and lots of hard work and planning before cutover time.  I don’t know if their presentation will be published but if it is you will see a case study in bending InBatch into a pretzel to work with an existing system.  It’s really really neat.

Regards and I’ll try to provide an update form Day 2 tomorrow night.

- Andy