NOTE: The blog of RTSL.eu - Development with SAS®

NOTE: DS2. Data Step Evolved? #sasgf12

2012-05-02T10:00:00.000+01:00

One of the other "futures" sessions I attended at SAS Global Forum was The New SAS Programming Language: DS2 with SAS's Jason Secosky. Jason was at pains to point out that DS2 is not intended as a replacement for the good old DATA step. DS2 is an alternative to DATA step and has more of a focus than the generalistic DATA step.

Generally available in 9.4, PROC DS2 is currently available in SAS V9.3 as an experimental technology. Its focus is on high performance for data manipulation and data analysis. It incorporates threading.

DATA steps are in control of their data; they specify the source of their input data, and they specify the location of their output data. In contrast, DS2 is simply a node in a flow; DS2 uses data streams rather than specific data objects. So, DS2 is not a DATA step replacement, it's new technology.

DS2's syntax is similar in parts to DATA step, with DATA and SET statements, if/then/else statements, expressions and functions. However, DS2 adds structure to code. Some of its syntax will be familiar to SAS/AF SCL coders; it includes methods (including init, term, and run). It has lots more types of variables when compared with DATA step, e.g. integer and varchar. DS2 integrates with other languages (such as R, C, C+ +, IML, and SAS fcmp functions) through the concept of a package. Interestingly, we'll be able to edit our DS2 code in the Eclipse editor, wherein a debugger will be included.

In essence, DS2 is the means of taking code to data (ref: big data) and promises linear scalability.

NOTE: Libnames, Who Needs 'Em?

2012-05-01T11:00:00.000+01:00

My team received what turned out to be an interesting call for help from one of our clients today. We resolved the client's coding error but it also served as a reminder of a little used feature of BASE SAS, namely the ability to specify directory names in code rather than bother with libnames. There are pro's and con's for doing this. I'll discuss these below after I explain the feature.

We're used to specifying data sets on DATA statements in the "libname.dataset" style. However, instead of using a data set name, you can specify the physical pathname to the file, using syntax that your operating system understands. The pathname must be enclosed in single or double quotation marks. Here's an example:

data "c:\mydata\mydataset";

In the foregoing example, the DATA step would create a SAS data set file named mydataset.sas7bdat in the c:\mydata directory.

There's more information in the section titled "Accessing Permanent SAS Files without a Libref" in the SAS 9.3 Language Reference: Concepts. You will see that we can use the same naming technique in almost any situation where a library and data set name are expected, e.g. a SET statement, a MERGE statement, an UPDATE statement, a MODIFY statement, the DATA= option of a SAS procedure, and the OPEN function.

My client's coding error resulted from the fact that they had specified a macro parameter intended as a data set name and they had surrounded it with quotes. The call %demo("name") resulted in a DATA statement like this: data "name". As a result, SAS tried to create a file named name.sas7bdat in the SAS session's current directory. That directory was the root directory of the SASApp server, the user didn't have permission to write to it, and hence the code failed. The intention was to create a data set named "name" in the work directory, the actuality was significantly different. It was all caused by a common misunderstanding/mistake - using quotes around character strings in macros.

So, we understand how we can dispense with LIBNAME statements, but should we take advantage of this capability? Well, I can't see too many advantages, but I can see plenty of disadvantages!

The disadvantages include i) need to accurately specify directory paths throughout the program (rather than eight character libnames), ii) cannot quickly and easily change a directory location (as can be useful when testing), and iii) cannot specify an engine for the library.

Can you think of any advantages? Let us know your suggestions in a comment.

Requirements. Whose Responsibility? #sasgf12

2012-04-30T10:00:00.000+01:00

I was pleased to see some papers on the subject of software development processes at SAS Global Forum this year. The IT industry hasn't yet reached a point where a consensus on the perfect software development process has been reached (will it ever?). So, it's no surprise that opinions differ on some matters.

One paper I attended opined that "requirements are developed by the end user of the software and not by the developer". The paper had a lot to commend it, but on this one point I strongly disagree.

Capturing requirements is a skill. It is not easy to gather all facets of the business requirements, and it is not easy to document them in a fashion that best serves all the needs of the development process (and beyond). Thus, it is unreasonable to expect users (or developers) to possess these skills unless they have been explicitly trained.

If training is required (e.g. in the absence of trained Analysts), does it not make more economic sense to train developers? They can be trained once and then use their skills (and growing experience) multiple times on subsequent projects. If you train a user, they are unlikely to re-use those skills (unless their application is in a constant state of change).

There are a variety of tools and techniques for performing analysis for requirements capture. One of the key skills is the ability to see beyond the current business process and to capture the true needs of the new business process. It is not apparent that users have a proper understanding of all aspects of their current business process; it is far from likely that they can accurately specify their target requirements. If requirements are to be of use, they must be documented in a form that facilitates their subsequent use by a) architects and designers, b) test case authors, and c) maintenance developers.

In my opinion, it is a developer's responsibility to help the user understand their current business process (particularly the processes for dealing with abnormal situations), and to guide them in the art of the possible for their target requirements. Developers need people-skills in addition to knowledge of tools and techniques for requirements capture.

The art of the possible is a key element of the requirements capture phase. We've all had experience of i) users asking for features that seem simple to them but are difficult/expensive for us to implement, and ii) users not asking for features that would be of high value to them but which they thought were too hard for us to deliver. I've seen countless examples of users telling me that they need the ability to:
a) email various reports to groups of people, and
b) write reports as spreadsheets.

Users typically express requirements in terms of things with which they are familiar, i.e. existing technology. We can advise them of the extended capabilities of:
a) portal and publish/subscribe capabilities that avoid the need to clog-up the email system with uncontrolled copies of report, and
b) web report studio and add-in for Microsoft office that give the user the ability to "interact" with the data, without the need for the data to leave the data centre.

If you're a developer, and you don't have professional Analysts to help you, take an interest in requirements capture; appreciate the skills, techniques and tools at your disposal, and (if possible) get some training to enhance your ability.

Delivering a successful project is a result of good teamwork. It is not the users' sole responsibility to produce good requirements; nor am I saying that it is the developers' sole responsibility. It's a question of what each party brings to the table. The users have to be committed and provide their time in addition to their knowledge and experience of the business; the developers must be willing and able to help the users express their requirements. You will succeed as a team.

Garbage in, garbage out. If all of the project's stakeholders are not clear on what is to be delivered, the chances of meeting everybody's expectations are much reduced. The capture of good quality requirements is crucial for ensuring the success of your projects. Play your part!

Papers Without SAS?! #sasgf12

2012-04-30T09:00:00.000+01:00

I was pleased to see a number of papers at this year's SAS Global Forum that dared to focus on topics outside of SAS technology and syntax. Two papers that particularly caught my interest were How to Create a Business Intelligence Strategy by Guy Garrett, and The Systems Development Life Cycle (SDLC) as a Standard: Beyond the Documentation by Dianne Rhodes. These papers were good demonstrations of the fact that you can buy the best software in the world, but you'll not optimise your return on investment if you don't put it to use in a planned, structured manner.

The focus of SAS Global Forum should always be SAS software and solutions. I'm not suggesting the event should be turned into a computer science conference, but there's a balance that can be struck. In my opinion, the balance lies at a point whereby attendees' interest in planning and process can be piqued such that they want to find out more once they return to their office.

NOTE: SAS Global Forum 2012 - Futures #sasgf12

2012-04-25T12:52:00.001+01:00

Due to the failure of my Asus Transformer tablet I'm still reduced to posting from my phone. So please forgive my brevity!

Yesterday was a good day for me at the conference. I attended a number of futures sessions in the morning and gained some insights into a few developments at SAS.

Firstly, collaboration and search with Murali Nori. SAS see an evolutionary path from reporting, to analytics, to collaboration. This can be public or private communities. Collaboration covers many themes, aside from discussion it can include participation in approval processes, integration with Sharepoint, email, or IM. One key element is, perhaps, the creation of the ability to audit and report on the decision making process that surrounds some of your SAS outputs.

Ultimately, the SAS Collaboration Framework should produce better decisions, quicker decisions, with the right people. It'll be like Facebook for the enterprise (secure and auditable)

As a small step in this direction, the new Visual Analytics tool will get the ability to add text comments at the end of this year.

In addition to the collaboration efforts, SAS are also aiming to provide much improved search capabilities; for structured and unstructured data. This means that a search in the portal might return you a list of "related" information items rather just items that precisely match your search (think of "Google for my SAS report"). We might see ranking of the results too.

NOTE: SAS Global Forum 2012 - Workflow Management #sasgf12

2012-04-24T17:31:00.003+01:00

Yesterday, I saw an interesting session with Diane Hatcher and other SAS staff which highlighted the workflow capabilities that ship with many SAS solutions (including eBI) but which most sites are unaware of.

The capabilities are not intended to be a commercial offering that might compete with established business process workflow management; rather, the SAS capabilities are intended to serve SAS solutions' specific requirements. The SAS EGRC solution, for example, encompasses workflow for management of the operational risks that are recorded within the solution; SAS Model Manager encompasses the management of the lifecycle of a model through workflow.

If you have a SAS solution that doesn't explicitly use workflow, you can still surface it and use it. It's a choice you need to make at install time, so making a choice to use it retrospectively can involve some considerable work.

Once you have it installed and working, its Workflow Studio allows you to design workflows using a combination of data objects, swimlanes, roles, decision points,events and notifications. Diane and her colleagues showed an example workfloe process that they had created for demo purposes. The example allowed the creation of a Web Report Studio report in a development area, approval of the finished report, and automated deployment of the report to a production area.

Workflow processes are stored in metadata and can be checked-out and checked-in, allowing release management.

Instances can be driven by 3rd party application, or by event signalling (providing good decoupling between parts of the workflow), or from SAS solutions.

NOTE: SAS Global Forum 2012 - Update #sasgf12

2012-04-24T17:12:00.005+01:00

My Asus Transformer Android tablet refused to switch on yesterday afternoon and has continued to do nothing ever since. This is most annoying, it's less than a year old. I've resorted to typing-up my notes on my Samsung Galaxy Nexus phone. It works, but it's a slower process.

Anyhow, yesterday was an interesting day at SAS Global Forum. I didn't hear anything earth shattering, but I picked-up a lot of useful information.

One of the highlighted outcomes of the Sasware ballot was the addition of read-only global macro variables in 9.4. I can't figure-out how I'd use that feature, but I guess there was some demand for it. Would you use it? I'd love to see a comment telling me how and why.

The morning's Technical Connections session featured five themes: high performance analytics, business visualisation, information management, decision management, and cloud. Some interesting items I picked-up were:

Introduction of a tool for business rule storage and management within SAS. I need to spend time in the demo area to find-out more about this
There have been plenty of demos of the new Visual Analyser at the conference.Its ability to create hierarchies on the fly, for drill-down, will be greatly appreciated by its users
Visual Analyser's integration with mapping also caught my eye as a valuable and nicely implemented feature
Following the introduction of a separate release track for BI tools and clients some while back, there'snow to be a separate release track for Stats and Analytics software too. The next release will be 12.1 (in August)
SAS 9.4 is expected to be released early next year. SAS are interested in feedback on how they can help their customers move through versions more quickly. What version are you currently running? What stops you from upgrading and getting the latest and greatest stuff from SAS?

NOTE: SAS Global Forum 2012 #sasgf12

2012-04-22T17:13:00.001+01:00

It's Monday morning; this is where the conference really starts.

Last night we had the opening session with SAS CEO Dr Jim Goodnight running through the now familiar Big Data themes such as minimising movement of data and increasingly sending code to the data. Products and tools such as the new Visual Analyser and PROC HPLOGISTIC (high performance logistics) were used to demonstrate these themes, with Visual Analyser using the LASR server,and HPLOGISTIC using in-database computations.

Tradition ensures that this morning kicks-off with a technical session wherein SAS staff are able to shows attendees under the hood of some of SAS's latest and greatest technologies. An 8am start means an early rise from bed after last night's lively mixer, but it'll be worth it.

It's been an interesting acclimatisation process since I arrived in The Sunshine State on Saturday afternoon. It was raining! I'm pleased to say that the sun has since made an appearance. I'm doubly pleased to see the rain go away because my hotel room is a 10 minute, outdoor walk from the hotel's restaurant and bus stop for transfer to the conference hotel. Yes, I tried to save money by staying at a local hotel rather than the conference hotel. I only have myself to blame, though I have to say that the walk around my hotel's lake each morning and evening is a very pleasant one.

I was hopeful of watching the Bahrain Grand Prix on Sunday, using the BBC iPlayer. Sadly, the BBC knows I'm out of he UK and tells me that iPlayer is not available. Frustrating, and surprising to see adverts on the BBC news pages - we don't get those when back home in Blighty! Anyway, I got my motorsport fix by watching the NASCAR race from Kansas Speedway. I enjoyed the race, and the high energy commentary team.

The lights are dimmed, here we go...

NOTE: Live-Streaming at SAS Global Forum 2012 #sasgf12

2012-04-21T23:30:00.000+01:00

If you can't make it to Orlando for SAS Global Forum 2012, you can view selected sessions live, online at the SAS Global Forum Live site.

Aside from the opening session, the selected sessions will include the SAS Global Forum Technology Connection on Monday, April 23, 9 a.m. ET.

The Technology Connection session is always a SAS Global Forum highlight. Attendees come in their droves to see senior members of SAS R&D demonstrate the latest analytic technologies and solutions that are transforming big data into useful information and better decisions. The fact that the session includes copious demonstrations (rather than slickly prepared slides) makes the session all the more valuable.

NOTE: Looking Ahead to SAS Global Forum 2012 #sasgf12

2012-04-20T20:30:00.001+01:00

Tomorrow morning (Saturday) I'll be leaving on a jet plane and heading to Orlando for SAS Global Forum 2012.

There's plenty to look forward to but some of the key interests for me include:

1) A new version of the Metacoda Security Plug-in. I look forward to getting a demo on the Metacoda stand in the exhibition area. The plug-in is a great way to manage, review, audit, troubleshoot and document metadata security

2) SAS's new in-memory data analytics tool: Visual Analytics

3) SAS's new proprietary mobile clients with presentation independent back-end. This effectively supercedes the stop-gap Roambi solution that SAS described at last year's SAS Global Forum

There's sure to be more. I can't wait.

Watch out for my regular posts (and tweets) from the conference.

NOTE: Registration for SAS Professionals Convention 2012 Now Open

2012-04-20T20:18:00.000+01:00

Before we've even got SAS Global Forum under our belts, registration for SAS Professionals Convention in Marlow, July 10th to 12th 2012, is open!

At £125 for three days of SAS activities, it's got to be the best value SAS offer you'll see in the UK in 2012. Check out the details by following the link from the SAS Professionals web site.

Day #1 focuses on SAS certification. If you want to demonstrate your SAS skills and knowledge, or provide evidence of the same for others, then certification is your solution.

Day #2 focuses on technical content and includes sessions on analytics, architecture, customer intelligence and the all-new SAS Visual Analytics (in-memory analytics tools). All streams offer high quality speakers but the business intelligence stream is a stand-out, featuring Guy Garrett from Achieve Intelligence and Tricia Aanderud (co-author of the BI Notes blog and the Building Business Intelligence Using SAS book).

Day #3 features SAS Dummy blogster Chris Hemedinger in the morning, and is followed by industry-specific streams in the afternoon.

Combined with social and networking events during the evenings, it's an unmissable event!

NOTE: Explaining Precision

2012-04-02T11:30:00.000+01:00

When teaching the SAS PRG3: Advanced Techniques and Efficiencies course, I always start section 2.2 Controlling Data Set Size with a slight amount of trepidation. My minor apprehension is due to the knowledge that the section includes a discussion on numeric precision. I'm confident that I know and understand the material, but I always find it's like explaining the rules of cricket to an overseas visitor!

There's always one or two learners who look at me like they don't believe it's not a SAS bug that "3 * (1/3)" doesn't precisely equal one.

Well, the next time that I teach PRG3 I shall do so with a tad more confidence because I've found a new ally in my battle to persuade my learners that it's a matter of computer science, not a bug in SAS. In Numbers great and small: the problem of Big Datum, Chris Hemedinger & Bill Brideson make a detailed and lucid explanation.

Accompanied by screenshots and plenty of links for reference, Chris & Bill's explanation knocked me for six when I first read it. And, if you're not on top of the rules of cricket, here's a brief explanation:

The Rules of Cricket as Explained to a foreign visitor
You have two sides, one out in the field and one in.
Each man that's in the side that's in, goes out, and when he's out, he comes in and the next man goes in until he's out.
When they are all out the side that's out comes in and the side that's been in goes out and tries to get those coming in out.
Sometimes you get men still in and not out.
When both sides have been in and out including the not-outs, that's the end of the game.
Howzat?

The humorous piece of seeming nonsense above really does describe the game of cricket perfectly.

NOTE: A Big Analytics Announcement (Visual Analytics)

2012-03-27T12:00:00.001+01:00

Following March 6th's announcement of SAS support for Hadoop (for massively parallel access to massive volumes of data) came March 22nd's announcement of SAS Visual Analytics, a new, in-memory tool with a highly visual interface.

The Visual Analytics solution contains an in-memory analytics engine that will eventually extend across a large amount of SAS's software product portfolio, including publish-to-mobile capability and graphic visualization.

Bringing recent announcements together and creating synergy, a core component of SAS Visual Analytics, the SAS LASR Analytic Server, uses Hadoop (embedded Hadoop Distributed File System) as local storage at the server for fault tolerance.

SAS Visual Analytics includes:

SAS LASR Analytic Server - clients communicate with SAS LASR Analytic Server for calculations on the data resident in-memory
The Hub - a central location to launch the various elements of SAS Visual Analytics
Mobile - a tool for viewing reports, connecting to servers and downloading information on the go
Explorer - an ad hoc data discovery and visualization tool to explore and analyze data
Designer - used to create standard and custom reports and dashboards
Environment Administration - used by administrators to manage users, security and data
Server components run on Red Hat or SUSE Linux, and the mobile client is available for the iPad.
Android support is expected to follow.

SAS's demo video for Visual Analytics hints at some of its capabilities without revealing too much of the detail. I'm looking forward to seeing a lot more detail at next month's SAS Global Forum. Will Visual Analytics knock spots off of existing in-memory solutions such as Qlikview, Spotfire, and Oracle Exalytics? Only time will tell, but it sure looks promising.

Combined with the fact that SAS was recently named as a leader for its data extraction, transformation and loading (ETL) technology in "The Forrester Wave: Enterprise ETL" by Forrester Research, and in the Leaders quadrant for its business intelligence (BI) technology in Gartner's Magic Quadrant for Business Intelligence Platforms, SAS has never looked stronger as an end-to-end solution.

NOTE: Keep Your WORK Library Working

2012-03-27T11:30:00.000+01:00

Steve Overton recently posted a tip on the excellent BI-Notes blog about maintaining space within your WORK library. The WORK library is a key resource and is typically shared by all SAS users, so its smooth operation is crucial for maintaining a reliable service.

Steve offered a unix/linux shell script to winkle-out old files that might otherwise be missed by SAS's cleanwork utility. It's clearly a popular topic and Steve's post received a number of responses, so I thought I'd offer my own small tip.

More than one of my clients applies a quota limit to all users on their use of the /saswork file system. By setting the quota to a suitable proportion of the whole amount of space available to everybody, the quota prevents one user from hogging all of the work space and causing other users' programs to fail. For example, if you have 1TB of WORK space, you might set the quota to 500GB. It's not a silver bullet, but it works well in practice.

The Geek Stuff provides a neat "how to" on the subject for unix/linux. I imagine that a similar technique is possible on Windows too, but I haven't the experience to say so for sure.

NOTE: A Big Data Announcement (Hadoop)

2012-03-26T20:53:00.002+01:00

There was a time when SAS practitioners merely needed to have knowledge of SAS DATA Step and PROC syntax plus a smattering of good data practices and IT practices. Over the years, SAS/ACCESS products gave access to data in other systems without the SAS practitioner needing to know anything but access credentials for those other systems. In more recent years, SAS version 9 has, on the one hand, introduced a range of SAS clients that allow the user to focus on their data knowledge and analytical skills rather than their SAS coding expertise, whilst, on the other hand, SAS version 9 has introduced (necessary) complexity to the architecture incorporating multiple types of SAS servers, comprehension of TCP/IP ports, third-party components such as web servers, plus the Platform suite of tools (from LSF job scheduling to Grid management).

Into this smorgasbord of clients and architecture, SAS added some fascinating new components earlier this month. I was interested in the new features and capabilities, but I also wanted to understand whether this brought more or less complexity for the SAS user and/or those charged with SAS platform support.

So, firstly, what was the announcement? Well, on March 6th, SAS announced the introduction of Hadoop support as part of Enterprise DI Server. Hadoop, in a nutshell, is an Apache, open source, product that provides massively parallel access to massive volumes of data. Significant users and supporters of Hadoop include Amazon, EBay, Facebook, Google, IBM, Macy's, Twitter, and Yahoo. No matter how you look at it, this is a big announcement if you are into big data; and if you're not yet into big data, maybe you soon will be.

There are multiple technologies associated with Hadoop, and SAS seems to have covered them all. For instance,

SAS/ACCESS will provide seamless and transparent data access to Hadoop (via HiveQL). Users can access Hive tables as if they were native SAS data sets.
PROC SQL will provide the ability to execute explicit HiveQL commands in Hadoop

SAS will help execute Hadoop functionality with Base SAS by enabling MapReduce programming, scripting support and the execution of HDFS commands from within the SAS environment. This will complement the capability that SAS/ACCESS provides for Hive by extending support for Pig, MapReduce and HDFS commands. [yes, I did copy the text from the SAS web site; no, I don't (yet) fully understand all of the terms!]
DI Studio will include Hadoop-specific transforms for extracting and transforming data

I'm slowly getting up-to-speed with this stuff myself, and I'll certainly be on the lookout for knowledge at SAS Global Forum next month. In the meantime, I found a couple of blog posts by Mark Troester most informative [1, 2].

The software release is certainly getting large amounts of positive comment from the technology media. The article in Information Week is just one example, singing SAS's praises.

And so, to return to my original question: has this announcement brought more or less complexity for the SAS user and/or those charged with SAS platform support? It seems clear that the SAS platform architect will need to understand Hadoop concepts, and that will require additional skills and knowledge. On the other hand, it sounds like SAS clients will do a great job of allowing the user to focus on their data and their analytical processes rather than learn new Hadoop-specific technical skills. On balance, I'd say that's the right compromise.

NOTE: SGF 2012 Early-Bird Registration Ends March 19th

2012-03-06T12:00:00.000Z

If you haven't already registered for SAS Global Forum 2012 (SGF) you need to hurry-up because the early-bird discounts will end on March 19th.

If you're not convinced of the value of attending, check-out last year's best contributed papers to see the quality and breadth of what's on offer. That's without taking account of the demo area and the ability to network with colleagues and peers.

NOTE: B&D to be Acquired by Sopra

2012-03-05T11:00:00.000Z

One of the UK's largest independant providers of SAS skills, Business & Decision UK, is to be acquired by the Sopra Group. Jointly announced with the same press release (B&D, Sopra) on the 13th February, the transaction is expected to be completed in the next few weeks, subject to regulatory conditions.

You may be able to remember SPS, one of the very first UK-based SAS consultancies, established by Peter Bailey. B&D, headquartered in France, acquired SPS some time ago. Whilst the likes of Amadeus and Base3 continue to provide focused SAS services in the UK, it seems that the trend is towards larger suppliers with a broader base of technology and integration skills. What do you think?

Sopra Group is also headquartered in France. It's results for 2011 show revenue of 1bn euros, 13,000 staff and a focus on IT services and software development.

NOTE: Hot Fix Home

2012-02-28T11:00:00.000Z

A recent upgrade from SAS 9.1 to 9.2 necessitated an upgrade from Enterprise Guide (and Add-In for Microsoft Office) 4.1 to 4.3 for one of my clients. Disappointingly, the EG and Add-In upgrades didn't prove to be the smoothest, and the experience was painful for a significant number of users. Top issues were a) poor migration of EG projects, resulting in users having to re-write some of their projects, and b) Excel freezing when the Add-In was used and the session was left for 15 minutes or so.

As a result of the foregoing, I took more than the usual amount of interest in hot fixes. My usual approach to hot fixes is "if it ain't broke, don't fix it", i.e. I only recommend the application of hot fixes if there's a specific issue to fix.

If you have an interest in hot fixes, the SAS Hot Fix Home is an excellent one-stop-shop for this information in the SAS Support web site. From there, you can choose your version (in the left-hand column), and then select "All SAS 9.x Products" to find a comprehensive list of SAS products with hyper-links to hot fixes for those individual products.

We're now running EG and Add-In with hot fix 20; our users are much happier.

Contrary to the experience with the SAS clients, the deployment of SAS 9.2 Grid has been pain-free and the users have no complaints about performance or reliability/availability. My clients have a robust, scalable SAS service.

NOTE: Enterprise Guide 5.1 - Now Shipping!

2012-02-27T11:00:00.000Z

If you haven't already picked-up the news from elsewhere, I should tell you that Enterprise Guide 5.1 (EG 5.1) started shipping earlier this month. For SAS 9.2 and 9.3 only* at this point in time, EG 5.1 looks like it adds some decent new functionality. I'll be keen to see it running at SAS Global Forum 2012 in a few weeks.

The What's New guide lists many new features. The ones that caught my eye were:

Improved grid support plus support for parallel execution of tasks. I shall be interested to see the degree of WORK library sharing between grid/parallel activity. This has been a weakness in out-of-the-box versions of EG to-date
Ability to export your output as an xlsx file (for use in Microsoft Excel 2007+)
For programmers, EG 5.1 offers additional autocomplete information (for data set names and data set column names)
Sundry enhancements to Query Builder and OLAP Analyser
Available as a 64-bit version, in addition to a 32-bit version

There's also an intriguing new tool called the Data Explorer. This apparently allows users to create views of their data with specific columns plus sorting and filtering. It sounds remarkably like the existing Filter and Sort task, so I'll be interested to understand the distinction between the two.

I hope to get my hands on a copy soon!

*I had understood that EG 5.1 was only available for SAS 9.3, but the EG 5.1 section of the What's New guide explicitly states "You can run SAS Enterprise Guide on either a SAS 9.2 or SAS 9.3 server."

NOTE: Code Kata #1 Follow-Up

2012-02-07T09:00:00.000Z

A few weeks ago I published the first in an irregular series of code katas. In a previous post I had described code katas as a form of training for programmers.

I saw a lot of interest around the post.

Richard Koopman posted an elegant response on his blog, including an additional solution for US coinage. His solution was neat, making good use of arrays and iterative coding, but my attention was also caught by Richard's home-made phone stand for his iPhone. Nice work Richard.

My old friend Dave Booth emailed me a solution that he had written off the top of his head, with no access to SAS at the time. It worked. I was suitably impressed. Dave's solution, like Richard's, used what Dave describes as a greedy algorithm. This is a very descriptive name and the Wikipedia entry offers the "coin change" challenge as an example. Dave also made a good point about handling errors elagantly:

Putting something in the code to try to trap unforeseen outcomes is a good habit to get into.

I agree, and code katas are a good opportunity to practice error trapping strategies.

Finally, Dave drew my attention to an excellent source of puzzles that could form code katas: ProjectEuler.net. The site offers a series of challenges that require a mixture of mathematical and computer science skills to solve. Take a look at the list of challenges and you'll see some good starting problems such as "Add all the natural numbers below one thousand that are multiples of 3 or 5".

In summary, don't wait for me to post more code katas. There are plenty out there on the web, and they all offer great opportunities for you to improve your coding skills.

"The Delegator" - Positive or Negative Epithet?

2012-01-31T12:00:00.001Z

In a recent meeting, an action arose and I asked one of my team (PJ) to catch it and complete it. He (tongue in cheek) commented to all those at the meeting that I was "the best delegator" he'd ever worked with. His humorous implication was that I don't do any work, I just get other people to do it for me. I've known PJ for years and worked on many successful projects with him, so I was happy to be the butt of his joke. However, I see delegation is a positive thing, so it set me thinking about how different people understand the verb "delegate".

The dictionary says it means "to give a particular job, duty, right, etc. to someone else so that they do it for you". The fact that one is giving one's work to somebody else to do doesn't sound altogether positive, but if we accept that a team can do more work than one individual, and if we accept that a team needs a leader to coordinate and dispatch work around the team, then the team leader isn't delegating his/her own work, they are simply spreading the team's work around the members of the team.

However, in my mind, delegation is a tool to help the team learn and grow. And that means that I choose to delegate some of the management work to members of the team also. By doing so, I can coach them to learn new skills; and I inevitably learn new things as we go along too.

Not everyone wants to be a manager, but everybody needs to understand the objectives and challenges of management. Armed with knowledge of management, one can work in harmony with one's leaders and management. Moreover, it is often the case that the perception of the team leader's role is not a match for the actuality. For me, team leadership is about a) achieving more than I can manage on my own, b challenging myself and my team to do new things, c) helping my team members to become the best they can become, d) learning from others, e) enjoying the benefits of being part of a successful team.

So, "The Delegator", is it a positive or a negative epithet? In my mind it is a distinctively positive name. It means I'm helping my team to learn and grow, and I'm helping them with their career and their ability to enjoy their working days (and at the same time they are helping me to do the same).

You Can't Fire Everyone

2012-01-24T12:00:00.002Z

Some years ago I was taught a memorable and valuable lesson by my erstwhile programme manager (we'll call him BJ). I was brought into the project to provide knowledge of i) business intelligence, and ii) software documentation appropriate for the regulated pharmaceutical industry. After a while on the project, BJ asked me to act as BI Delivery Lead. He promised me the ability to recruit a team but initially supplied me with two existing staff members.

After a week I went back to BJ and said he'd have to provide me with two different staff members because the first two a) knew nothing about BI, and b) were at each other's throats all day long. BJ told me that nobody else was available; I'd have to make it work. Then he put his hand on my shoulder and told me he'd especially chosen me to act as BI Delivery Lead because he knew I had good people skills. Flattery is a good management technique! But more importantly, BJ was teaching me an important team leadership lesson: developing people is just as important as developing software.

It's a lesson I'll never forget and, as I worked with my two staff members over the following months and we all achieved increasing degrees of success, I discovered that developing people can be at least as satisfying as developing software.

I was reminded of this specific lesson when I read Roy Osherove's You Can't Fire Everyone post on his 5Whys blog in December. Roy's blog is a treasure trove for those who want to become a better team leader. Roy typifies the process of team development as following three phases: the chaos phase, the learning phase, and the self-leading phase. Roy describes how to recognise which phase you're currently in, and how to progress and develop your team. Recommended reading.

NOTE: Monitoring the SAS 9.2 Metadata Server

2012-01-24T12:00:00.000Z

There's a nice walkthrough of some of the features of the SAS 9.2 metadata server on the SAS Professionals web site in the Social Media / Videos section. You'll have to be a SAS Professionals member to view it from the link I've provided, but if you're not already a member it's a good reason to join now (it's free).

The 7 minute video is particularly useful if you've recently upgraded from 9.1 and would like a reminder of the nice new features you now have available to you.

It particularly caught my eye because I've just recently added the SAS Platform Administration: Fast Track (SPAFT) five-day course to my training repertoire. So, if you plan to attend this SAS UK course this year, you might just find me teaching you (please don't let that put you off booking!).

Code Kata #1

2012-01-18T12:00:00.000Z

In yesterday's post I described Code Katas and how they can be a useful form of programming practice. Dave Watson (the "inventor" of Code Katas) offers 21 on his Code Kata blog. I intend to publish my own from time-to-time, and here is the first. Enjoy...

Imagine you have to write some code for a vending machine. You are given a) the price of the product to be vended, and b) the amount of money tendered by the customer. Your task is to deduce what change to offer, i.e. how many of each type of coin. Optimise your result by minimising the number of coins given in change. Assume your vending machine has unlimited stocks of the following denomination: 1p, 2p, 5p, 10p, 20p, 50p, £1, £2.

Two examples of input and output follow:

i) Product price is 70p; money tendered is £1, i.e. 100p. Optimal output is 1 x 10p and 1 x 20p

ii) Product price is 63p; money tendered is 80p. Optimal output is 1 x 10p, 1 x 5p, 1 x 2p

This challenge is entirely language-independent, so your solution can be produced in the manner that best suits your regular coding activities. Hence, you might write your solution using DATA Steps and/or you might use macros. Your input values might be provided as global macro variables, or parameters to your macro(s), or supplied within a data set.

Once you've cracked the above, you might want to consider the following additions - perhaps on subsequent days.

A) Accept a "list" of coin denominations as additional input rather than hard-coding the list I supplied above. The list might be as a parameter to your macro, or in a data set

B) Rather than assume an unlimited stock of each coin in the machine, accept an input "list" that tells you how many coins of each denomination are in the machine. Consequently, you will need to consider the possibility that your result is a refusal to vend due to "insufficient change"

C) Rather than writing a simple textual result to the SAS log, produce some kind of coloured report or graph depicting the change given

You also might like to consider different solutions. The use of arrays or hash tables would be two alternatives.

Code Katas

2012-01-17T12:00:00.001Z

Last week I made a brief mention of Code Katas and how they can help you as a form code practice. Let's find out some more.

The basic concept surrounding Code Katas encapsulates the ideas that a) we are professional coders, b) practice improves one's skills. Wikipedia tells us "Code Kata is a term coined by Dave Thomas, co-author of the book The Pragmatic Programmer, in a bow to the Japanese concept of kata in the martial arts. A code kata is an exercise in programming which helps a programmer hone their skills through practice and repetition".

Dave has a blog for his books, and a sub-blog specifically about Code Katas. His Code Kata blog tells us all about the background and Dave's reasoning for using Katas.

I greatly enjoy the consultancy and the project management work that I do, but I get at least as much enjoyment from coding too. Not only coding to a specification, but coding for coding's sake. And that, in a nutshell, is what a Code Kata is - coding for coding's sake. If you were into tennis, you might focus on competitions wherein you have to give your best, but you'll be able to achieve better results in those competitions if you practice regularly. There are no prizes for practicing, but it sure helps.

Who benefits if you become a better coder? Your employer or client will certainly benefit, but your own career prospects will improve too. So, don't limit yourself to doing your Code Katas within core hours. Maybe it's something you might consider doing over lunch, or after hours.

Dave lists 21 Katas on his site, most of which I've found applicable to SAS. I'll offer my own contributions from time-to-time here on NOTE:, starting tomorrow. In the meantime, please feel free to send your own Code Katas to me. I'll publish some of them.