Data science teams are multidisciplinary, each with different skills and technologies of choice. Some of them use SAS, others may have analytical assets already built in Python or R. Let's just say each team is unique.

As part of our Continuous Integration/Continuous Delivery with monthly releases, we are always looking to extend SAS Viya integration capabilities to support open-source users and technology. The goal is to enable all types of users to leverage their best skills, ensuring governance of assets, explainable AI and operationalization of models (ModelOps).

With the October 2021 release of SAS Viya, we introduced the Python Code Editor. Data Scientists and Python programmers can now code, execute and schedule Python scripts from within the SAS code editor interface (SAS Studio) or add Python steps to a SAS Flow quickly and intuitively.

Both options offer Data Scientists the flexibility to:

• use Python inside a SAS environment for query, preparation and analysis depending on users' skills, comfort and preferences, as well as the problem they are trying to solve
• efficiently create a flow to integrate SAS and Python code for consistent delivery of analytics-ready data pipelines
• support Information Governance initiatives by displaying tables created by Python, using SAS linage diagrams
• leverage the benefits of using SAS Viya with open source for achieving business value

This post demonstrates the use of this capability with some simple Python code. You will learn:

• how to create a single program or a flow integrating python code in a SAS environment
• what happens from a high-level architecture perspective
• how to use SAS lineage diagrams to present Python table usage

Getting started

A standard SAS Viya installation does not have Python support enabled, as the environment runs in a lockdown mode, for security purposes. Your Kubernetes and/or SAS Administrator can enable Python support in lockdown mode and configure the default location for the SAS environment. Suffice it to mention here, Python 3.x is supported and you can use any Python package, as long as it is available inside the directory structure that contains Python.exe. Details about these configuration options are described in detail in the section Configure SAS to Run External Languages in the SAS Viya Administration documentation.

Intro to Python Code Editor

The Python Code Editor allows you to write, run, and save Python programs. In the example outlined below, we use data in a SAS table called sashelp.class and apply data transformation using Python. By the way, the SAS table could have been a SAS data set or a DBMS table that is supported via SAS/Access software, which includes support for ODBC and JDBC.

Figure 1 - Python Code Editor in SAS Studio

When you run your Python code from the Python Code Editor, SAS Studio runs SAS code invoking the PYTHON Procedure and embeds the Python code in a submit/endsubmit block. This procedure enables running Python statements within SAS code. Considering Figure 1, the Python editor appears similar to the SAS Code Editor; however, notice the Python.py filename, this is Python code! The Code tab is for editing your Python Code and the Log tab displays the executed statements and information from the Python console. When the Python code yields results stored in a SAS table, the table displays in the Output Data tab, where you can interactively browse its content.

The PYTHON procedure

The PYTHON procedure creates a Python subprocess from a SAS Compute Server process and automatically imports the SAS code enabling interaction between SAS and your Python instance. The next section explains this in more detail.

This module provides callback methods enabling variable sharing between Python and SAS (SAS.symget or SAS.symput), move data between SAS data sets and Pandas DataFrames (SAS.sd2df or SAS.df2sd), invoke SAS and FCMP functions (SAS.sasfnc), and submit SAS code within your Python statements (SAS.submit). The documentation for the Python procedure provides examples for each of these.

Using these callback methods, you can create a workflow that mixes SAS and Python programming as needed.

How is the Python code handled at runtime?

What happens when you run your Python program in the SAS coding interface? Let's take an architectural deep dive to explain. The following diagram summarizes what happens when you run a program in the Python Code Editor.

Figure 2 - Running Python code in SAS Studio - what happens?

1. Run your Python code from the Python Code Editor.
2. As SAS Studio works with SAS Compute Server, invoke the Python procedure on the SAS Compute Server, which starts a Python subprocess.
3. The code provided in the code editor is put inside a proc python submit/endsubmit block, which executes inside the Python subprocess.
4. Because the Python code contains a call to the SAS.sd2df() function, the SAS Compute Server reads the specified SAS table, and provides data in the specified Pandas DataFrame.

The sample Python code contains some statements calling a few Pandas functions, but you could use any available Python package in the Python directory configured by your SAS Administrator.

5. The sample code issues a SAS.df2sd() call transfering the results from a Pandas DataFrame back into a SAS table.

Integrating Python steps with SAS steps in a flow

Now that we have developed and tested our Python code, we are going to embed it in a Studio Flow. This allows us to combine Python code and standard data transformation functionality in SAS Studio Flow. Moreover, we can view and define the data transformation using a graphical flow with a mix of SAS and Python transformations. And finally, we can easily schedule the flow.

In the next example, we modify the Python code so it contains no hardcoded table references. Instead, it uses tables connected to the input and output ports of the step.

In the following flow, we have dropped a Python Program step onto the canvas and connected tables representing the input and output tables. The table steps contain information about data location. Note that those table steps could represent a SAS data set but could also be a table that lives in a DBMS system.

Figure 3 - Flow using Python Program node, showing use of input and output ports in Python code

The Python Program step supports input and output ports (not networking ports!) and each has an associated variable representing the name of the table connected to that port. The user can overwrite the names of these ports.

Note: The Python Program step has a single input and output port by default, but you can add and remove ports using the popup menu.

When we run the flow, the code framework generates code for each step in the flow. For the Python Program step, it also creates Python variables, _input1 and _output1, which contain the name of the connected table using SAS table reference syntax libref.tablename.

Just like we observed previously with the Python Code Editor, the Python Program step invokes proc python SAS code and the native Python code provided in the Code tab of the step, and is embedded inside a submit/endsubmit block.

As a result, the generated code for the Python Program step, using input and output ports in Python code from Figure 3, looks like this:

Figure 4 - Generated code for a Python Program step

This code is visible in the Generated Code tab in Studio Flow.

Now that we have embedded our code in a Python Program step in Studio Flow, we can change the properties of the input and output Table steps and point to a different table. The Python code uses the name changes when we run the flow interactively. If your data requires pre- or post-processing, you can add additional steps in the flow.

In the example below, we added a Query step to join data from an additional source using an inner-join and applied a filter. We also change the properties of the Table step by adjusting the name of the table where we store the results of the Python processing.

Figure 5 - Using Python Program and other nodes in same flow

The flow is now complete and the green tick-marks on each of the transformation steps indicate the steps ran successfully.

How to keep an overview of which tables are used where?

Thus far we have created a single Studio Flow with steps that work on input tables and create output tables. When you have multiple flows, perhaps created by other users in the environment, and some of these flows use tables created/updated by our flow, how do you keep an overview of which tables were used where? This is where SAS Lineage Viewer comes into play.

So, let’s assume another user creates a flow that uses our output table to perform some further data transformations creating another table. How would we know about this?

Figure 6 - SAS Lineage Viewer showing where tables are used

The lineage diagram above shows the CLASS and CLASSFIT tables are inputs in Flow 1b and this flow wrote data into table CLASS_PYTHON_NEW. In Flow 2, the table is input, and the flow writes data into table CLASS_PYTHON_FINAL.

This definition of each flow is known to SAS Lineage Viewer and we searched it for our flow, called Flow 1b. The result shows a diagram with all input and output tables. And if a table is used somewhere else, it has a (+) icon. In our example, by expanding the CLASS_PYTHON_NEW table, the diagram shows use by Flow 2.
By using SAS Lineage Viewer, you can quickly see which tables are used where, even when your Studio Flows contain Python Program steps.

Wrapping things up

This Python code editor allows programmers and data scientists to code, execute and schedule Python scripts. The functionality is available from within the code editor interface in SAS Studio or by adding Python steps to or program in a SAS Studio Flow. In either case, it offers flexibility to use Python or SAS for query, preparation, and analysis depending on user's skills, comfort and preference, and problem they are trying to solve. You can now create a single program or flow to integrate code. Additionally, use steps and nodes from SAS and Python code for consistent delivery of analytics and ready data pipelines in a productive, efficient manner. In this post we introduced the module and how to get started, how it works, and the architecture behind it.

Learn more

SAS integrates with open source in every step of the analytics life cycle. Users can learn more about how SAS works with open source by downloading the eBook, “Drive Analytic Innovation Through SAS and Open Source Integration” and by visiting developer.sas.com.

We have also developed specific packages available on Github and supported by SAS R&D: SASPy and SWAT, both aimed at a Python programmer wanting to use functionality available on a SAS server from their Python client environment.

About the co-author

Wilbram Hazejager - Principal Systems Architect

Wilbram Hazejager is a principal systems architect for data management products at SAS. He has been involved for many years in SAS’ data management products in various roles, including product management, working closely with customers, and authoring a number of papers on SAS data management products. He holds a Master of Science degree in Applied Mathematics from the Eindhoven University of Technology in the Netherlands.

The post SAS opens its code editor interface to Python users appeared first on The SAS Data Science Blog.

Scoring with state space models

Scoring is evaluating a previously fitted model at some new predictor setting.  For example, a bank might score a new loan application by applying a rule that is based on a previously fitted logistic regression model.  You might also want to use SSMs for scoring.  Because of the sequential nature of the observation process, it is important to realize that the scoring process for an SSM is inherently different from the scoring process for a model that is based on independent observations. Examples would be simple linear regression or logistic regression.

Think of a sequential data process like a long-running TV series, with a new episode every week. Watching the series for some time and learning about the main characters and their relationships to each other is akin to fitting a model to the initial portion of a sequential data stream.  After you have watched some episodes, you can make educated guesses about the subsequent plotline. You can also imagine different ways the series could play out, while still being consistent with the storyline.

For sequential data, this process of forecasting and what-if analysis, based on the fitted model and ever-increasing history, is called scoring.  Additionally, the scoring process enables you to monitor an ongoing observation process for additive outliers (that is, one-off observations) and structural breaks such as shifts in the mean level or other patterns.

PROC CSSM

The scoring process requires a model that is a good fit for the initial portion of the data stream.   After a suitable model is fitted, a score store is created.  The score store contains sufficient information for you to forecast subsequent observations of the data stream, without having access to its history. If scenario analysis is the only goal, this initially created score store is all that is needed for scoring different future scenarios.  On the other hand, if the goal is to perform stability monitoring of the ongoing data stream, the score store must be updated on a continual basis to incorporate the information from the incoming data.  The CSSM procedure enables you to easily handle all aspects of such a scoring process.  With PROC CSSM you can:

• Fit and diagnose a wide variety of SSMs for a wide variety of sequential data types
• Create the initial score store, after a suitable SSM is found for the initial portion of the data stream
• Without access to the historical data:
  • Perform forecasting and scenario analysis using a previously created score store
  • Create an updated score store based on the newly arrived data
  • Perform stability analysis of an ongoing data stream

Next, let’s illustrate this SSM-based scoring process with a real-life example.   The data used in this illustration are set to a monthly frequency and the computing time for scoring is not a constraint.  However, SSM-based scoring can be effectively used in applications where the scoring must be done at a much higher frequency, such as every minute or every second.

Analyzing monthly traffic accident numbers

The traffic accident data analyzed in this example consist of observations on four variables recorded at monthly intervals from January 1969 to December 1985.  The variables, all in the log scale, are:

• F_KSI: front-seat passengers killed or seriously injured
• R_KSI: rear-seat passengers killed or seriously injured
• logKM: average number of kilometers traveled per car per month
• logPrice: real price of petrol

The time series plots below show the F_KSI and R_KSI numbers over the years:

From the time series plots you can see that:

• F_KSI and R_KSI numbers don’t show a pronounced upward or downward trend but do show monthly seasonal variation
• These two series seem to move together, at least till February 1983. Starting in February 1983, the mean level of F_KSI appears to drop whereas the R_KSI pattern seems to remain unaffected.

The drop in the mean level of F_KSI is attributed to the enactment of the seat-belt law of February 1983. This law required the front seat passengers to wear seat belts.  It didn’t apply to the rear seat passengers.  These data have been analyzed by many researchers in the past.

Stability monitoring of monthly traffic accident numbers

In this section, we will see how you can perform a model-based stability monitoring of the bivariate series (F_KSI, R_KSI).  We will use data from January 1969 to December 1981 as the initial portion of the series. This is used for fitting a suitable model and to create the score store.  Note that the drop in the mean level of F_KSI, which happened in February 1983, is not part of this initial portion.  We hope to detect this drop during the stability monitoring of this series, which begins from January 1982.

Denoting by Y_t the bivariate series (F_KSI, R_KSI), and by X_t $\beta$ the regression effects of logKM and logPrice, the model Y_t = X_t $\beta$ + $\mu$_t + $\gamma$_t + $\epsilon$_t has been shown to be a reasonable model for this initial portion of the series. Here $\mu$_t denotes the mean level, $\gamma$_t denotes the monthly seasonal component, and $\epsilon$_t denotes the noise component (all the terms in the model are bivariate).  The data table mycas.train contains the initial portion of the series.  The following statements fit this model and create the score store, mycas.inscore:

   proc cssm data=mycas.train;
      id date interval=month;
      state wn(2) type=WN cov(g);
      comp wn1 = wn[1];
      comp wn2 = wn[2];
      state meanLevel(2) type=RW cov(rank=1) checkbreak;
      comp f_KSI_Level = meanLevel[1];
      comp r_KSI_Level = meanLevel[2];
      state season(2) type=season(length=12);
      comp f_KSI_season = season[1];
      comp r_KSI_season = season[2];
      model f_KSI = f_KSI_Level logKM logPrice f_KSI_season wn1;
      model r_KSI = r_KSI_Level logKM logPrice r_KSI_season wn2;
      output break(alpha=0.01) ao(alpha=0.01);
      score out(8)=mycas.inscore;
   run;

The score store, mycas.inscore, saves the following information:

• Model specification and parameter estimates
  • This also includes the user-specified outlier and structural break detection options such as the CHECKBREAK option in the specification of the bivariate mean level, and the significance levels of the structural break and outlier detection tests (0.01 for both types of tests in this case).
• Estimate of the latent state vector (and its covariance), which serves as a bridge between the past and the future.
• The last 8 rows, from May to December of 1981, of the input data table mycas.train. This is because we used the OUT(k)= form of the OUT option in the SCORE statement.   This user-supplied, 8-observation window is used during the stability monitoring.

With the saved information in the score store, mycas.inscore, the model specification, or the historical data (mycas.train) are no longer needed to perform the stability monitoring of the future observations.  During the stability monitoring, new observations arrive one at a time.  This new observation gets read into a one-row table, mycas.nextObs, which, in turn, gets scored:

   proc cssm data=mycas.nextObs;
      score in=mycas.inscore out=mycas.outscore;
   run;

This simple scoring step accomplishes two key operations:

• The 9 observations, which are formed by joining the new observation with the 8-observation sliding window of the preceding observations that is stored in mycas.inscore, are scanned for any structural breaks or additive outliers
• An updated score store, mycas.outscore, is created by utilizing the new observation

After each scoring step, the two score stores (mycas.inscore and mycas.outscore)  are swapped so the new observation always gets scored using the latest information. See SAS Help Center: Scoring: Monitoring Streaming Data for a complete code illustration of these steps.   In our example, stability monitoring starts in January 1982.  The main findings of the monitoring process are as follows:

• No breaks in the mean pattern or additive outliers are found between January 1982 and January 1983
• Starting with February 1983, for the next several months, the structural break/outlier detection tests flag February 1983 as an unusual month for two reasons:
  • Drop in the mean level of F_KSI, which is called a level shift (LS)
  • The unusually low value of F_KSI, which is called an additive outlier (AO)

The following table shows a summary of this monitoring process:

• New_Obs column shows the month of the observation that was scored
• AO_Z and LS_Z columns show the Z-statistics for these AO and LS tests, respectively

Clearly, the Z-statistics for both AO and LS are highly significant in all the months between February and July.  However, the significance of LS increases steadily whereas the significance of AO decreases steadily.  This strongly points to February 1983 being a level shift for F_KSI, rather than a one-off value.

Usually when a structural break, such as a shift in the mean level, is detected, the monitoring is suspended.  Before the monitoring can resume, you must update the model to account for the break. Or the cause for the break must be addressed by making appropriate changes to the mechanics of the observation process.  In this traffic monitoring case, the monitoring can resume after the model is updated to account for the effect of the seat-belt law.  In this example, a break in mean level was detected. You can use both PROC CSSM (and PROC SSM) to detect more general types of breaks.

Conclusion

Streaming data and their monitoring, usually without explicit human intervention, are happening everywhere.  SAS Viya provides excellent tools for dealing with such data (for example, SAS Analytics for IoT). The new scoring capability in PROC CSSM (or the underlying CAS actions ssmFit and ssmScore) is designed to neatly fit in such applications.

The post Using State Space Models for the Stability Monitoring of Streaming Data appeared first on The SAS Data Science Blog.

What is a supply chain digital twin?

Envision the following use case. A major consumer packaged goods (CPG) client wanted to:

• predict product shortages
• identify the bottlenecks that cause the supply chain to experience low fill rates
• determine the corrective actions to take to reduce the risk of future shortages

In other words, could we build a digital twin of the supply chain to predict problems? And could we identify solutions to fix future problems before they happened?

A supply chain digital twin is a virtual representation of real-world supply-chain entities and processes. Its digital and physical counterparts are synchronized at a specified frequency and fidelity (Digital Twin Consortium 2021). The expectation of the digital twin is that it will provide enhanced visibility into the future of supply chain operations. As well, it enables playing operational “what-if” games before the implementation of a decision or a policy.

How to build supply chain digital twin

We custom-built a digital twin for this client through the integrated use of several SAS applications to represent, visualize, simulate, analyze, and optimize the supply chain operations:

• SAS DATA and PROC steps retrieved, wrangled, and analyzed supply chain data
• SAS^® Visual Analytics visualized the supply chain
• SAS^® Simulation Studio simulated the supply chain
• SAS^® Visual Data Mining and Machine Learning accelerated the “what-if” games
• SAS^® Optimization helped to identify the corrective actions

As a result, our client is now equipped with the power to continuously optimize their supply chain using a digital twin.

Supply chain network flow

The development of a supply chain starts with the description of the supply chain network flow. Motivated by the CPG industry, Figure 1 shows the flow of a multitiered network composed of suppliers, factories, warehouses, and customers. There is also a transportation channel between each pair of tiers.

At first glance, the illustration might indicate one supplier, one factory, one warehouse, and one customer pool. Nevertheless, at the heart of the supply chain digital twin development lies a flexible supply chain simulation model that is data-driven and scalable with the number of suppliers, factories, warehouses, and customers. Furthermore, the flow might appear to be moving in a single direction. However, underlying information flow is much more complicated. Warehouses replenish finished goods inventory from factories whereas factories replenish component inventory from their suppliers.

Each of these events must happen at the right time for the right quantities. Thus, there is a complex logic to the network flow moving both forward and backward. It requires the implementation of plans to manage suppliers, production, transportation, and inventory throughout the supply chain. Figure 1 also shows fill rate and total cost as two generic key performance indicators (KPIs). The goal of the supply chain digital twin is to mimic this flow for the following reasons:

• predict various KPIs and gain visibility into the future of the supply chain operations
• assess the impact of operational policies and investment decisions in a virtual environment
• stress test the supply chain and identify the best courses of action to take when faced with disruptions

Figure 1 An illustration of supply chain network flow

Once the supply chain network configuration is identified, the next step is to collect all relevant information and data which fully describe the supply chain. A generic list of input data sets includes:

• supply contracts and supplier data
• initial inventory
• production plan
• customer demand
• transportation details
• inventory control policies
• supply chain cost parameters
• characterization of disruptive events.

The disruptive event list could include loss of inventory and/or capacity due to natural disasters or disturbances such as fires, site contaminations, and equipment shutdowns. However, the details of data collection and definition are dependent on the types of decisions that the supply chain digital twin supports. They are further affected by the speed of decision-making.

A production plan is an example of the input data that are used to build a supply chain digital twin. Using SAS Visual Analytics, Figure 2 displays the daily quantity of production to take place on each production line in two factories of the production channel. It further illustrates the division of the production plan into hourly buckets with pre-specified production start and end times.

You might wonder where this production plan is coming from. It comes from a production plan optimization solved using SAS Optimization. The supply chain network simulation takes the optimized production plan as an input. It then uses SAS Simulation Studio to quantify the risk of implementing this plan under uncertainty. SAS DATA and PROC steps provide feedback on how the production plan can be improved with an in-depth analysis of the simulation-generated output data. This is how we bring optimization and simulation into a closed-loop reflecting continuous learning in digital twin development.

Figure 2 Visualization of a production plan over the course of a week

Supply chain KPI prediction

Combining supply chain network flow with all necessary pieces of information and data completes the characterization of the supply chain logic. This logic is represented in SAS Simulation Studio to mimic the flow of thousands of products. These products are starting as raw materials and ending as finished goods through a network of suppliers, factories, warehouses, and customers with transportation fleets operating among different tiers of the supply chain. However, this is done at a level of detail that is needed to support the way the client plans to use the supply chain digital twin. Furthermore, the supply chain digital twin design aligns with the speed of decision-making.

The representation of the supply chain logic in SAS Simulation Studio is followed by the characterization of the uncertainty in the random supply chain inputs. Examples of random inputs include supplier lead-time, production and assembly times, transportation time, and customer demand size. Figure 3 illustrates two example probability density functions - obtained from historical data - to represent the randomness in production-time and transportation-time processes. SAS Visual Analytics plots each of these density functions with the x-axis providing all the possible values production time or transportation time might take. The y-axis shows how likely it is to observe each of these values.

During the simulation of the supply chain network flow, SAS Simulation Studio captures input uncertainty by sampling from these density functions, propagates it throughout the supply chain network, and embeds the effect of uncertainty into the simulation-generated output data.

The next step is to design the “what-if” games. This is done by specifying the levers that the client could be changing in the search for better ways to manage the supply chain. If the purpose of developing the supply chain digital twin is to help improve the management of inventory, then the levers could be chosen as the inventory control parameters. If the goal is to support fleet management, then the levers would likely be chosen as the attributes of the transporters. The lever selection could be further customized to support supply chain reconfiguration decisions.

Next, we execute the supply chain network simulation. Consequently, we generate vast amounts of data representative of how the supply chain might perform in the future. Furthermore, we amass this large amount of data very quickly. By taking advantage of SAS Visual Analytics, Figure 3 shows how box-and-whisker plots illustrate risk profiles for the supply chain fill rate and the total cost. Therefore, not only do we predict the expected values of the two key supply chain KPIs, but we also quantify the risk in their predicted values.

Figure 3 Combining logic, input uncertainty, and game design to predict KPIs

Identification of corrective actions

Our analysis of the supply chain simulation output data is not limited to the prediction of supply chain fill rate and total cost. The supply chain network simulation is designed to generate time-stamped data. This shows the flow of every raw material, work-in-process, and finished goods through the supply chain. This includes the temporal usage of every resource including people, machines, facilities, and transportation fleet across many possible futures that might realize. Using SAS DATA and PROC steps to analyze this supply chain output data set leads to the time-based operational KPIs. This includes flow times, capacity utilization, and inventory fluctuations. Further analysis of these KPIs results in the identification of the supply chain bottlenecks.

Using SAS Visual Analytics, Figure 4 illustrates how the temporal study of supply chain fill rate together with factory inventory and utilization identifies a specific factory as the bottleneck. It causes the supply chain fill rate to fall below 50% between the 6th and 19th months of the time horizon. This simulation-data-driven insight is critical for the identification of a corrective action to help better manage the supply chain under uncertainty.

Visualize the following situation. An action to prescribe for the supply chain shortages, illustrated in Figure 4, is adding capacity to the overutilized factory. This can be done in three different ways:

1. Purchasing a flexible machine that can perform multiple tasks (solution A).
2. Allowing weekend production (solution B).
3. Adding a third shift (solution C).

Figure 4 An Illustrative supply chain digital twin KPI dashboard

The supply chain digital twin can be used to select which one (or a combination) of these actions to take. Figure 5 illustrates an example (operational) risk profile of the supply chain fill rate for each candidate solution. It suggests that the purchase of a flexible machine (solution A) would not only maximize the supply chain fill rate but also reduce the level of risk exposure. Similar information can be readily obtained from a supply chain digital twin for a wider variety of operational and financial KPIs. This enables the client to make decisions with more information.

Figure 5 Optimizing supply chain fill rate under uncertainty

Digital twin of a supply chain summary

This post covers the development of a supply chain digital twin. We helped a SAS client gain enhanced visibility into the future of supply chain network operations. At the heart of this custom-built digital twin lies a flexible, data-driven, and scalable network simulation. Because this simulation can be viewed as a large supply-chain data generation program, KPI prediction and optimization can be accelerated through integration with SAS Visual Data Mining and Machine Learning.

We built digital twins customized to our client's supply chains through the integrated use of multiple SAS applications:

• to represent, visualize (SAS Visual Analytics)
• simulate (SAS Simulation Studio)
• analyze (SAS Visual Data Mining and Machine Learning)
• optimize (SAS Optimization) the supply chain operations.

The resulting capability equips our clients with the power to predict supply chain KPIs. It also quantifies the risk in the KPI predictions. As well, they can identify the corrective actions maximizing profit in addition to improving resiliency. The supply chain digital twin also serves as a virtual laboratory to assess the impact of disruptions, operational policies, and investment decisions.

The post How SAS developed a digital twin of a supply chain appeared first on The SAS Data Science Blog.

Let’s break GAN down further to understand how this algorithm works and dispel the mystery behind it.

• Generative model: A statistical model that can generate new data. This includes the distribution of the data.
• Adversarial training process: There are two networks involved in training. One network generates the data (the generator) while the other network tries to discriminate (the discriminator) if that data is real or fake. If it is deemed to be fake, the generator is notified and tries to improve on the next batch of generated data. Therefore, the two networks are training against each other, hence the adversarial part.
• Deep learning Networks: Deep learning methods use neural network architectures to process data, which is why they are often referred to as deep neural networks.

Why on earth would you want to use a GAN?

Now that you know what a GAN is, what do you do with it? You may have heard of deepfakes and enjoyed seeing videos of political leaders uttering some unbelievable statements. (Somedays, I wonder how we would know the difference!) Other than playing tricks on the world, GANs do have a valuable purpose.

Deep learning models are data-hungry. What if you could just snap your fingers and grow your training data set? Well, GANs can help you create synthetic data for those deep learning models. Synthetic data, or artificial data, serves as proxy data because it maintains the statistical characteristics of the real-world data that it is based off. Synthetic data should generate observations based on existing variable distributions and preserve correlations amongst the variables in the data set.

Deepfakes typically use image data and the type of GAN to create synthetic image data is called a styleGAN. However, other types of data such as tabular data (think rows and columns of integers, text, etc.) can also be created. This is a tabular GAN.

Watch SAS Data Scientist, Brett Wujek, talk about StyleGANs in the SAS Viya Release Highlights (2021.1.2).

I see lots of potential with GANs and synthetic data. Synthetic data allows you to create deep learning models when you may not have previously been able to do so. There simply may not be the volume of data available that is required, especially when you are working with new products or processes. Data may also be expensive and time-consuming to acquire from third-party resources or through data collection methods such as surveys and studies. Synthetic data may also help fulfill the gaps in underrepresented groups such as customer segments, regions, or even the different driving conditions required by computer vision models for self-driving cars. Lastly, because this data is generated, it does not impact human privacy (think GDPR and personal data sharing regulations) and is less risky should the data be breached.

To remind us, that while synthetic data has the potential to help us progress with deep learning, the patterns in the synthetic data must be representative of the real data and should be verified as an initial step in the modeling process.

Learn more

GANs are available in the SAS Data Science Offerings.

The post Who else wants to know the mystery behind GANs? appeared first on The SAS Data Science Blog.

Jan Chvosta

Note from Udo Sglavo: In our previous post, Jan Chvosta, the director of Scientific Computing at SAS, and I discussed the origins of regression analysis and some of the ways it is used today. Now we will further discuss the power of regression framework and choosing the correct regression model.

Udo: In the previous post we discussed the power of the regression framework. In a way, all practitioners are attempting to accomplish a similar task. They want to choose the best regression model and fit it to the data available.  It sounds simple in principle but deciding on what is the best might be tricky. How would you go about this in the case of regression analysis?

Jan: In principle, this sounds simple but there is much to be considered in the process. In the first post, we talked a lot about linear regression but in reality, there are many more types that need to be considered.  It begins with understanding your data and choosing the right regression analysis type. For example, if your data is binary, logistic regression might be a good choice. There are many questions that need to be asked and decisions that need to be made to choose the right regression model. SAS^® Visual Statistics and SAS^® Econometrics documentation can help with these decisions. If you are new to regression analysis, Introduction to Regression Procedures can help to get you started.

Udo: We often hear about regression assumptions that are critical for the framework to work. Could you explain what this means?

Jan: These assumptions are crucial because they play an important role in determining whether the model will be a success or failure. Each regression type has its own set of assumptions that you need to consider and evaluate. If they are too restrictive, you might need to think about a different model. For example, if you are fitting a linear regression you need to:

• confirm that the relation between your response and explanatory variables is linear
• your observations are independent of one another
• variance of the residual is constant for all observations
• residuals are normally distributed

Udo: What happens if the assumptions are violated?

Jan: I think we can illustrate this with a situation we face in our daily lives. We might assume that there isn’t a traffic jam during our morning commute to work thus we make it to work on time. If our first assumption is wrong, however, and there is a traffic jam, then our prediction that we make it to work on time is also wrong. It is similar to regression analysis. If we choose the wrong regression model, our parameter estimates, inference about them, and even predictions can be all wrong.

Udo: This seems like a complex problem. Can you provide us with some guidelines on how to choose the correct model?

Jan: There are many ways to go about this and fortunately SAS is here to help. SAS software can address a variety of models based on different distributional assumptions and data complexities. It can also handle tall and wide data, and overall help you with your modeling efforts. You have a set of very powerful tools available. But the success of a modeling effort is often decided even before statistical or econometric modeling starts.

The idea is that you need to understand what you want prior to any analysis, regression or not. You need to ask yourself many questions before you even start fitting a model. For example, you might ask if a simple but interpretable model is more important to you than a more complex model with higher prediction accuracy.

Many questions will point to your data. Are your data observations discrete, count, rare events, independent in time? Do they follow a pattern? Is this analysis univariate or multivariate? In all of these situations, SAS provides robust tools for users to expand their modeling capabilities and predictive modeling power. However, the users play an essential role because they control the modeling process and make important decisions. The questions asked and tools used will also likely depend on the field specialization of the users.

Udo: Are you suggesting that scientists working in different fields would have a different approach to regression analysis?

Jan: Yes, domain knowledge is important. It comes into play when you need to build a sensible and interpretable scientific model. The domain expert tells you what they need from the model. Regression analysis experts (such as statisticians, econometricians, or data scientists) can provide data modeling inputs that are more appropriate to the choice of regression analysis. The domain knowledge is also necessary for correctly interpreting the results. The two need to work hand in hand to achieve a better fit, a better prediction, more robustness, and better interpretability.

Udo: We touched on many important issues and it is wonderful to hear that we have many regression tools available at SAS. Can you provide a high-level overview of these tools?

Jan: Most of our regression tools are available in SAS Visual Statistics and SAS Econometrics. There is certainly more than one way to group these tools. I am going to choose a grouping that follows a frequently used modeling decision process. It also highlights the procedures we have available for each of the areas.

Figure 1 depicts regression procedures in SAS Visual Statistics. They are grouped according to whether the response variable is categorical or continuous, models for means, quantiles, Generalized Linear Models, nonparametric, semiparametric, or parametric approaches.

Figure 1: Regression Procedures

Figure 2 depicts regression procedures available in SAS Econometrics. The econometric regression tools can be grouped into count data modeling, cross-sectional data regression, spatial data regression, and panel data regression. The time series regression procedures can be grouped into univariate and multivariate analyses.

Figure 2: SAS Econometric and Time Series Regression Procedures

Udo: What are some recent additions to SAS Visual Statistics regression analysis?

Jan:  Regression analyses have been enhanced in SAS Visual Statistics and SAS Econometrics in many areas. I will just mention several recent ones.

Causal inference in SAS Visual Statistics is where regression techniques are used to not only model the data but also help in providing valid causal interpretations.

In addition, Bayesian computation regression techniques are implemented in several procedures. This includes the often-used GENMOD procedure that handles generalized linear regression models. As well, the BGLIMM procedure provides regression modeling capabilities to fixed and clustered data. Combining multiple sources of information in a regression analysis brings unique advantages in numerous applications.

Model selection is an inevitable frontier in regression analysis, as the dimensions of the data get to be both incredibly wide and long. SAS/STAT^® provides a broad spectrum of tools in helping you to select variables in big data. This ranges from traditional but optimized methods such as forward and stepwise, to shrinkage methods, such as LASSO, SCAD, and MCP, to projective methods, such as principal components regression, to model averaging. These techniques are developed in many regression tools, such as linear, GLM, quantile regression, cox proportional-hazard models, as well as nonparametric and semiparametric additive models.

Udo: Are there also recent additions to SAS Econometrics regression analysis?

In econometrics, the Hidden Markov Models (HMM) is an advanced regression tool for time series analysis. In HMMs, there are hidden unobserved states, and at different states, the observations follow different regression models.

State Space Modeling (SSM) is another quickly growing area in time series because almost all kinds of time series models can be written in the state space form. The CSSM procedure was added to SAS Econometrics in the 2020.1.3 release and will soon introduce scenario analysis, forecasting, and monitoring of streamed data.

Spatial data regression available in SAS Econometrics is another relatively new tool that is quickly gaining popularity. I think it is not that hard to imagine that observations are often spatially dependent and omitting that dependence from your model can impact your results.

Udo: Anything else you would like to share with our readers?

Jan: When it comes to regression analysis, there are many procedures and action sets to choose from. There is a lot to learn and explore. Mastering regression analysis and applying them to your research or business problems will certainly have a great impact on meeting your goals and objectives regardless of your area of interest. With SAS you have a very compelling set of tools available. Learning and exploring them can be fun as well as having a great impact on your business.

For more information on spatial econometric modeling:

• Spatial econometric modeling unleashes the geographic potential of your data
• Automate spatial regression model selection using proc cspatialreg

The post Art or science? Choosing the right regression model appeared first on The SAS Data Science Blog.

Jan Chvosta

Note from Udo Sglavo: In many different areas of data science, we often talk about which regression technique to use. There are many other important tools outside of the regression framework. But it seems though that regression is the tool of choice. Interestingly, if you mention the word regression to a group of data scientists, it never fails to create an interesting discussion. They will passionately discuss what the concept means to them. However, the discussion will be heavily influenced by the field of their expertise and experience. We all grasp the general concept, but it seems that regression means different things to different groups of practitioners. In the first of two blog posts, I discuss the origins of regression analysis and how it works with Jan Chvosta, the director of Scientific Computing at SAS.

Udo: Linear regression has been around for a very long time. Can you discuss its origins?

Jan: The idea of regression goes back to the 19^th century and the research work Sir Francis Galton did concerning the hereditary characteristics of sweat pea plants. In his experiments, he discovered that children of plants with very high weight didn’t necessarily possess the same characteristic. On average they were closer to the average population weight than their parents were. In other words, Galton observed that some extreme characteristics weren’t completely passed to the offspring but instead these characteristics tended to regress towards a standard point. He repeated the experiment hundreds of times discovering the regression to the mean principle. In addition to many other areas, today's modern regression analysis is a powerful tool also used in the field of genetics that Galton was pioneering.

Udo: Interesting story but how would you explain a regression today?

Jan: If you were to conduct a quick search for books on regression analysis, you would find many different books written on the topic. Taking a closer look, you would realize that one theme is dominant - linear regression. Simply put, linear regression is an approach that allows us to model a relationship between variables in a linear fashion. Response, also known as the dependent variable, is modeled by one or more explanatory (independent) variables. If there is just one explanatory variable, we get a simple linear regression. If there is more than one explanatory variable, we get multiple linear regression. I would start with linear regression.

Udo: What is the intuition behind linear regression? Can you explain how it works?

Jan: In the case of multiple linear regression, it provides a very intuitive explanation of how a unit change in one independent variable (say age) impacts the dependent variable (say weight), holding all other variables (age, height, gender) in the model constant. The simplicity makes the modeling and its interpretation accessible to a wild range of audiences. It also often works well in applications. The extension from a simple regression to multiple regression leads to the capability of having many covariates jointly predict a response variable. Not only does this frequently work better than simple regressions, but it also provides a better understanding of the association of variables on the outcome. Figure 1 shows The PROC REG output from a multiple linear regression of children's weight on age, height, and gender.

Figure 1: Multiple Linear Regression of Children Weight

If you hold all other variables constant, it is not surprising that age has a positive impact on weight. One month of age increases the weight of a child by 0.43 lbs. There is also a positive impact of height on weight. One inch increase in height increases the weight by 0.49 lbs. Both of these variables are statistically significant from zero at a 0.01% level of significance. The regression results seem to suggest that if the child is a female, she weighs less by 0.73 lbs. However, the female variable is not statistically significant from zero at the 10% level. This small example demonstrates clearly that regression analysis can be a powerful tool.

Udo: A picture is often worth a thousand words. Is there a way to show a linear regression with a picture?

Jan: You can think of a linear regression as a modeling approach where you fit a line to data points in such a way that it minimizes departures (squared) of all points from this line. The REG procedure allows you to fit many different linear and even some nonlinear regression models. Figure 2 shows an example of PROC REG fitted linear regression of weight on age along with information about the data and regression fit.

Figure 2: Fit Plot for Weight

Interestingly, even this fit plot can be tied back to some of the work that Sir Francis Galton did. He was also plotting the mother and daughter pea plant data. He was trying to explain the relationship between them by putting a straight line through the data. Not having the modern software, however, it was a lot of work.

Udo: Thank you for describing the principles of linear regression. Can you elaborate more on its use?

Jan: At a very high level, the relationship between variables matters, and regression analysis can help you to explore it. Imagine you are running a marketing campaign trying to increase sales of your company. You collected the data from similar campaigns in the past and now you are trying to understand how various marketing options impacted the sales. Fitting a regression can help you to determine what marketing options (explanatory variables) had a significant impact on sales (response variable). It can also help you to predict the sales for various combinations of marketing options and find the best combination to maximize your sales.

Udo: Are you suggesting that regression analysis can be used for predictive modeling?

Jan: Yes, regression analysis can definitely help you if you are interested in predictive modeling. For example, if you think about picturing a regression in linear form with estimated coefficients from Figure 2, you just need to plug in values for your inputs (age, height, gender) to predict the dependent variable (weight). You can even get predictions for values of age, height, and gender that weren’t available in the dataset you used to estimate your coefficients.

Udo: Regression analysis has many different aspects. How would you go about learning more?

Jan: Regression analysis is a very powerful tool. You need to understand how to correctly use it. SAS^® Visual Statistics and SAS^® Econometrics documentation is a great resource with many examples.  If you are entirely new to regression analysis in SAS, Introduction to Regression Procedures provides a great overview. In our next post. we further discuss the regression framework and choosing the correct models.

The post Which regression technique is appropriate for my data? appeared first on The SAS Data Science Blog.

Advanced and predictive analytics software includes data mining and statistical software and uses techniques including machine learning, regression, neural networks, rule induction, and clustering to create, test, and execute statistical models.

IDC measures advanced & predictive analytics in its annual Worldwide Business Intelligence and Analytics Software Market Shares* report – and has consistently ranked SAS as the #1 market leader for over two decades!

Advanced and predictive analytics can be used to discover relationships in data and dig into that data to make predictions. SAS helps customers gain these insights through our analytics technology.

“In the advanced and predictive analytics market, SAS represents over a quarter of the market share, with continued focus on statisticians and data scientists and the broader need for advanced analytics.”
—Dan Vesset, IDC Group Vice President, Analytics and Information Management

Let's take a look at how advanced and predictive analytics from SAS helped a customer achieve their business goals:

OTP Bank Romania is part of OTP Bank Group, one of the largest financial institutions in Central and Eastern Europe. They relied on a combination of data mining and predictive analytics to gain insights from an increasing amount of data – and make sure they’re not missing out on ways to increase customer retention. They were able to quickly develop descriptive and predictive models through a streamlined data mining process.

SAS gave OTP Bank Romania the ability to harness algorithms and techniques including decision trees, time series analysis, neural networks, linear and logistic regression, sequence and web path analysis, market basket analysis and link analysis. The bank relied on SAS as a workbench and a set of tools for statisticians or data scientists, which increased collaboration with analysts to improve their business.

They began using predictive analytics to meet the needs of their modeling team. This allowed the bank to ensure control over the quality of loan originations, achieve more accurate prediction of business and risk outcomes, and meet profitability targets required for the bank’s loan portfolios.
OTP Bank Romania gained the ability to:

• More easily work with databases.
• Conduct new analyses based on performance indicators.
• Develop numerous models for a single scope.
• Compare results.
• Choose the best result to meet the bank’s objectives.

SAS brought valuable insights to OTP Bank Romania in addition to 91 of the top 100 companies on the 2020 Fortune Global 500®. A growing part of the business intelligence market, advanced & predictive analytics from SAS can help organizations everywhere achieve their business goals.

Learn more about how SAS helps customers uncover insights using analytics – and try SAS for yourself.

*IDC Business Intelligence and Analytics Software Market Share Reports, 1993-2021

The post SAS Leads Market 28 Years in a Row, IDC Advanced & Predictive Analytics Software appeared first on The SAS Data Science Blog.

Generating and using word embeddings

Word embeddings are the learned representations of words within a set of documents. Each word or term is represented as a real-valued vector within a vector space. Terms or words that reside closer to each other within that vector space are expected to share similar meanings. Thus, embeddings try to capture the meaning of each word or term through its relationships with the other words in the corpus.

Word embeddings are one way to create structured data from text. These embeddings can be used for finding similar terms or in machine translation. Additionally, embeddings can be used in machine learning models to classify sentiment or in document categorization.

There are several word embedding methods available but new techniques seem to move into Sesame Street at a rapid pace. Let's move chronologically through the development of a few of the most popular word embedding techniques.

Early word embedding techniques

SAS has been doing word embeddings since SAS Text Miner, but our approach has changed over the years as new research and techniques have been developed. In SAS Text Miner, word embeddings started with the Term-by-Document matrix. This matrix represents each document in the corpus as a column and each term as a row. The values stored in each cell represent a weighted value of term frequency within a document. Even with a small corpus, this matrix could be large and sparse. A process called Singular Value Decomposition (SVD) can factor this large, sparse matrix into smaller, more information-dense matrices. Performing SVD on the Term-by-Document matrix results in a word embedding matrix as well as a document embedding matrix.

Word2Vec emerged in 2013 through research at Google. Word2Vec takes a prediction-based approach to word embeddings. This approach uses a shallow neural network in one of two configurations. Within both configurations, the context for a word is defined by a predetermined number of terms before and after the given word, also known as a sliding window. In the first configuration, called Continuous Bag of Words (CBOW), the context of a word (i.e. terms around each word) is used as the inputs to a neural network, with the word itself as the output. The second configuration, called Skip-Gram, is like CBOW but reversed. The word itself is input to the neural network and the context is the output.

Source Mikolov et al. (2013) "Efficient Estimation of Word Representations in Vector Space"

In the following year, Global Vectors for Word Representation (GloVe) emerged through research at Stanford. GloVe is an unsupervised approach that uses a co-occurrence matrix. The main idea behind the co-occurrence matrix is that similar words tend to occur together and have a similar context. Instead of using a Term-by-Document matrix, GloVe utilizes a symmetric Term-by-Term matrix, where each row and column represent a term and their matrix value represents their co-occurrence. This resulting matrix is also large and sparse. SVD or PCA (another dimensionality-reduction technique) is utilized to create smaller information-dense matrices, including our word embedding. And like Word2Vec, Glove utilizes a sliding window of terms to calculate co-occurrence.

Word2Vec and GloVe tend to show better results on semantic and syntactic word analogy tasks than the Term-by-Document matrix, but Word2Vec and GloVe don't do the best job on capturing context. Word2Vec and GloVe generate a single embedding for each word, which isn't great for words with the same spellings but different meanings. Thus, starting in 2018 we see several new methods materialize.

A growing number of techniques

In 2018, SAS released the tmCooccur Action in SAS Visual Text Analytics. Expanding on GloVe, this method utilizes a sentence-level context to calculate term co-occurrence and can utilize the results of SAS's text parsing action to better capture context and multi-group phases such as noun-groups and entities.

Embeddings from Language Models (ELMo) is another method that also uses the whole sentence to provide context. Unlike previously mentioned methods, ELMo is a pre-trained bi-directional Long Short-Term Memory (LSTM) model. LSTM is a type of Recurrent Neural Network (RNN). Recurrent Neural Networks are often used in text analytics because they can model sequential data through neuron feedback connections. LSTM adds a few components to RNN models to govern the neuron's output.

Another pre-trained model is the Bidirectional Encoder Representation from Transformers (BERT) released by Google. Like ELMo, BERT can generate a different embedding for each term based on its context. Unlike ELMo, BERT is using a Transformer. Transformers are another type of neural network that also handle sequential data well, but they don't always process data in order. For example, it may process the end of the sentence before the beginning.

Source Devlin et al. (2019) "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

The best word embedding technique

There are certainly a lot of different word embedding techniques. But just like prediction and classification models, there isn't a technique that is best for all uses cases and data sets. In benchmarking tests, one approach may be better for word similarity tasks whereas another may be better for document categorization. With that in mind, experiment and test! If you've generated a word embedding using open source that you want to use on SAS Viya, you can bring it in using the Word Vector Action. Additionally, using the Scripting Wrapper for Analytics Transfer (SWAT) package, you can run this action and other analytics in SAS Viya from Python or R. With this new knowledge embedded in your mind, I hope you can make the most of your text data!

Learn more about word embeddings

• The Wondrous New tmCooccur SAS® Cloud Analytic Services (CAS) Action and Some of Its Many Uses
• Co-occurrence matrix & Singular Value Decomposition (SVD)
• Word embedding
• Word Embeddings in NLP
• Introduction to Word Embeddings and its Application
• Singular Value Decomposition
• Overview of Word Embedding using Embeddings from Language Models (ELMo)
• ELMo: Contextual language embedding

The post Generating word embeddings appeared first on The SAS Data Science Blog.

Overview of causal inference

Cause-and-effect relationships have been puzzling humans for centuries. Which came first – the chicken or the egg? Whenever you ask why or what-if questions, you are indirectly trying to find answers about cause and effect. The causal inference can help you in your effort. It is the new science of providing the methods and tools for identifying the cause and measuring the effect.

In measuring the causal effect, the randomized experiment is always the gold standard. However, in most cases, the randomized experiment is too expensive or even impossible to conduct. For example, if you would like to know the effect smoking has on developing lung cancer, you can’t randomly assign someone to smoke. If you would like to know how having a college degree affects salary, you can’t randomly send someone to college. In real life, most causal analysis is based on observational data!

Big-data challenge and machine learning tools

Before discussing the challenges of estimating causal effects based on observational data, let’s first focus on the data itself. In the era of big data, it’s not surprising to see tens of or even hundreds of thousands of variables that are used to describe an object of interest (customer, patient, area, store). Some of those variables are often categorical (gender, race), count (number of siblings), or continuous (spending amount on a product last year). In this scenario, two variables are our main interests. They are the outcome variable, denoted as $\mathit{Y}$, and the treatment variable, denoted as $\mathit{T}$. For instance, in some personalized pricing cases, $\mathit{Y}$ could represent demand, and $\mathit{T}$ could represent whether a discount was offered or not. The causal model is typically researching the relationship between these two variables.

Later in this post, we will discuss the other variables (denoted as $\mathit{X}$) which might also be critical in the causal effect estimation. However, how to incorporate those high-dimensional mixed-discrete-and-continuous variables ($\mathit{X}$) into the estimation, especially when the relationship between all these variables might be in some unknown nonlinear forms is challenging. The classical linear regressions and the Generalized Linear Models might lead to misspecification. So, we might need some more powerful nonparametric tools. Due to the high dimensionality of the problem, using machine learning or deep learning tools might also be a good idea.

Causation is more than a prediction

Using machine learning and deep learning techniques certainly sounds promising. However, we might not be able to directly apply them in the causal effect estimation. Machine learning is suitable for prediction. Unfortunately, causation is more than a prediction. In the simplest and most common case, the treatment variable is binary. The person is treated or not, and the causal effect is defined as the difference between the potential outcome if the individual were treated,$\mathit{Y(1)}$, and the potential outcome if the individual were not treated, $\mathit{Y(0)}$. However, each individual is either treated or not. You can never observe both $\mathit{Y(1)}$ and $\mathit{Y(0)}$ for the same individual in the data. This effectively means, $\mathit{Y(1)} - \mathit{Y(0)}$, the target of your interest, is always missing for all individuals! Hence, we can’t use machine learning tools to directly predict the causal effect.

Instead of estimating the causal effect directly, we can consider breaking the algorithm into three steps. We then use the machine learning tools for the prediction in steps 1 and 2. In step 3, we estimate the average causal effect using results from machine learning steps 1 and 2 as follows:

• We estimate the relationship $\mathit{f^{(1)}}$(.) between $\mathit{X}$ and $\mathit{Y}$ by using the data of individuals who are treated, $\mathit{E(Y^{(1)}) = f^{(1)}(X^{(1)})}$, and the relationship $\mathit{f^{(0)}}$(.) between $\mathit{X}$ and $\mathit{Y}$ by using the data of individuals who are not treated, $\mathit{E(Y^{(0)}) = f^{(0)}(X^{(0)})}$, where $\mathit{E}$(.) is the expectation and $\mathit{Y^{(T)}}$ and $\mathit{X^{(T)}}$ denote the data when $\mathit{T}$ = 0 (untreated) or 1 (treated);
• We predict the $\mathit{Y}$(0)s for individuals who are treated by using $\mathit{f^{(1)}}$(.) and the $\mathit{Y}$(1)s for individuals who are not treated by using $\mathit{f^{(0)}}$(.);
• Now, we average the difference between $\mathit{Y}$(1)s and $\mathit{Y}$(0)s of all individuals and that is the estimated (average) causal effect.

If the data are not observational but comes from random controlled trials, the three-step method above works well. However, when the data are observational, this plug-in method has a good chance to lead to some highly biased estimation results. This is because it does not take care of certain kinds of important variables, known as confounders! The confounders are the variables that have an impact on both the Outcome $\mathit{Y}$ and Treatment $\mathit{T}$. Simpson’s Paradox is often used to illustrate the importance of confounders.

For example, in a hospital, 2,050 patients treated for an illness are observed. 550 of them are treated and the rest of the 1,500 are not. 445 patients get cured in the treated group and 1,260 patients get cured in the untreated group. Then, the success rate of the untreated group, 84%, is higher than the success rate of the treated group, 81%. So, this seems that the causal effect is at least nonpositive.

However, when a confounder, the severity of illness (SOI), is considered, an interesting fact is revealed. For patients with higher SOI, the success rate of the treated group is higher than the untreated group. For patients with lower SOI, the success rate of the treated group is also higher than the untreated group! That is, without considering the confounder, we might reach the totally wrong conclusion about the causal effect. The details of the example are shown in Table 1.

Table 1:  An example of Simpson’s Paradox

Semiparametric framework comes to the rescue

Now, you know, it’s not easy to estimate the causal effect based on the observational data. This is true even when the machine learning tools can help us to handle the huge number of mixed-discrete-and-continuous variables $\mathit{X}$, $\mathit{Y}$ and $\mathit{T}$, as well as the unknown nonlinear relationships among them. Here, the semiparametric framework comes to the rescue! There are two main steps in the semiparametric framework:

• Nonparametric step: use the machine learning tools (here Deep Neural Networks, DNNs) to estimate the relationship between the treatment variable and the covariates (the so-called propensity score model) and the relationship between the outcome variable and the covariates with the treatment variable (the so-called outcome model).
• Parametric step: for parameters of interest, construct the doubly robust estimator through the influence functions, which takes care of the impact of the confounders.

Among so many machine learning tools, why are DNNs selected? It's because of the DNNs’ convergence rate. One issue that we sometimes experience with machine learning tools in econometrics and other fields is the interpretability of the model. In this particular case, the estimator needs the standard errors to conduct inference. If the machine learning tools can converge to the true unknown functions fast enough in the nonparametric step, the semiparametric framework can provide the valid standard errors of the estimators in the parametric step.

To our knowledge, thanks to the proof in Max H. Farrell, Tengyuan Liang, and Sanjog Misra (2021), DNN is the only machine learning method that can directly fulfill this converge fast requirement. So, by using DNNs in the nonparametric step, we can obtain not only the unbiased estimator but also its standard errors for inference.

The powerful and easy-to-use DEEPCAUSAL procedure

Successfully applying the semiparametric framework including DNN implementation, might not be easy because of the complexity of the estimation. You don’t need to worry, however, because all the technical details are taken care of in the DEEPCAUSAL procedure. For example, the SAS code for estimating the causal effects of interest might look like this:

   proc deepcausal data=mycas.mydata;
         id rowId;
         psmodel t=x1-x20;
         model y=x1-x20;
         infer out=mycas.oest;
   run;

In this simple example, no DNN options are needed, because tens of DNN options can take their default values. Of course, if you are an advanced user, you can even specify different DNNs with different training hyperparameters for different models. See the documentation for details.

The output of PROC DEEPCAUSAL includes the estimates of parameters of interest for both the full population and the subpopulation. For example, the two most famous and useful parameters of interest are in the output: the average treatment effect (ATE), and the average treatment effect for the treated (ATT). An example of the output from the DEEPCAUSAL procedure is shown in Table 2.

Table 2: Example of the estimated parameters of interest

The DEEPCAUSAL procedure and policy evaluation

Besides the parameter estimates, PROC DEEPCAUSAL also supports policy evaluation. which is used to assign the treatment. For example, the policy of sending coupons to customers might be sending them only to customers who spent more than $400 last year. If we know the causal effect on each individual and we can determine the treatment assignment, we have an opportunity to optimize business objectives (maximize revenue, minimize loss, evaluate the impact of a marketing campaign) by setting up the optimal policy.

There are many applications of policy optimization in different fields. Here are just a few: customer targeting, personalized pricing, stratification in clinical trials, learning the click-through rates, and so on. PROC DEEPCAUSAL can evaluate the average effect of a policy or compare the difference between two policies by using the observational data that you have in hand. To do so, you only need to provide the policies in the POLICY= option and the policies to be compared in the POLICYCOMPARISON= option in the INFER statement.

For example, the observational data shows the base policy denoted by t. What if we changed that policy to a new policy denoted by s?  Here are two candidates. Policy s1 is to assign treatment to individuals only if their estimated individual treatment effect is positive. The policy s0 is the opposite. It only assigns treatment to individuals whose estimated individual treatment effects are negative. You’d like to evaluate each policy’s average outcome and compare them with the observed treatment t. The SAS code might look like this:

   proc deepcausal data=mycas.mydata;
         id rowId;
         psmodel t=x1-x20;
         model y=x1-x20;
         infer policy=(s1 s0) policyComparison=(base=(t) compare=(s0 s1))
               out=mycas.oest;
   run;

Table 3 shows the output.

Table 3: Example of policy evaluation and comparison

As you can see from the table, different policies have different average outcomes. Policy s1 leads to an average outcome of 0.65 whereas policy s0 leads to an average outcome close to 0. Compared to the observed treatment assignment t, policy s1 has significant positive gains whereas policy s0 has a significant loss. This is an example of where policy matters and why policy optimization is so important.

Other powerful tools for causal inference

Although we covered several topics in this post, there are many more related to causal modeling. The following list mentions a few that might be of interest to you.

• CAUSALGRAPH procedure in SAS/STAT enables you to analyze graphical causal models and to construct sound statistical strategies for causal effect estimation.
• CAUSALMED procedure in SAS/STAT enables you to decompose a (total) causal effect into direct and indirect effects.
• CAUSALTRT procedure in SAS/STAT enables you to perform an estimation of a causal effect.
• MODEL, PANEL, QLIM, and TMODEL procedures in SAS/ETS and the CPANEL and CQLIM procedures in SAS Econometrics support instrumental variables when there are unmeasured confounders.
• PSMATCH procedure in SAS/STAT enables you to perform propensity score analyses and to assess covariate balance.
• VARMAX procedure supports Granger causality tests for time series data.

Causal inference conclusion

Correctly estimating the causal effects is critical for decision-making in our daily lives and business applications, especially in this big data era. In the 2021.1.4 release, SAS provided a new DEEPCAUSAL procedure that takes advantage of both deep learning and econometrics methods and makes your causal inference modeling much easier.  We hope you find it useful in your modeling efforts.

The post Causal inference and policy evaluation with deep neural networks appeared first on The SAS Data Science Blog.

This year's IEEE Visual Analytics Science and Technology (VAST) Challenge covered such a scenario and a team of SAS volunteers decided to put SAS® Viya® to the test and submit a solution for both Mini Challenges 2 and 3. The mission was to analyze movement and tracking data for several employees of a fictitious company named GAStech. Two weeks' worth of data was provided before a disappearance (kidnapping) of some of the company's employees. Mini Challenge 2 involved using provided credit card and loyalty card usage data to identify any anomalies and suspicious behavior leading up to the disappearance. Mini Challenge 3 required further analysis of social media and text information to determine any changes in risk levels during the night of the disappearance.

The process

We used SAS® Visual Analytics to track the area's vehicle traffic and to analyze financial and loyalty data. We detected popular locations and points of interest by analyzing credit card transaction records. The data also revealed employees' classification given their movement profile at different times during the day. We utilized SAS Viya's support for machine learning (ML) and decision tree scoring to determine actual credit card owners based on location visits and frequency.

We also utilized SAS Visual Text Analytics to tackle Mini Challenge 3 by extracting useful information from provided social media posts to detect risk levels using natural language processing (NLP).

The tools

Map plotting geo-referenced locations

The majority of work was done using SAS Studio and SAS Visual Analytics. We leveraged some core capabilities in SAS data steps to prepare and adjust the provided data for easier analysis. In particular, we calculated aspects like vehicle speeds and credit card owner scores in SAS Studio with greater flexibility. We also used Esri's ArcGIS mapping tools to geo-reference a provided image of the city streets and tourist map. Some shop and business locations were determined by inspecting both the map and provided Esri shape files.

The Solution - Mini Challenge 2

The first step in discovering meaningful patterns was to analyze provided credit and loyalty card data. Each employee was issued a company card and the challenge was to determine the actual card owner based on anonymized transaction data. Knowing who made which purchases at what time of the day allowed not only for the classification of shops and businesses, but also revealed some anomalies in the data (e.g., wrong/missing transaction timestamps, purchases at exact noon times, etc.).

Purchase prices by location

Further cluster analysis using SAS Visual Analytics allowed us to categorize locations into three natural groupings based on the purchase frequency, price and busiest time.

We also added vehicle data into our analysis of the credit and loyalty card data. Using Esri ArcGIS mapping tools, we georeferenced both the city streets and tourist map and created a list of all known locations including their exact GPS location.

Taking the actual vehicle tracking data into account, we could then determine where vehicles were at a given point in time. Clustering GPS data also allowed the creation of travel hotspots grouped by an employee's department. This showed that some employee groups visit some areas more frequently than others.

Travel hotspots

From the general movement patterns GAStech employees can be classified into two broad types:

1. 1. 1. 1. General staff who visit all stores in Abila except Abila Airport, Carlyle Chemical Inc. and Nationwide Refinery (which are all locations involved in industrial interactions with GASTech).
2. Truck drivers that typically drive between industrial locations and GASTech headquarters with only a few exceptions.

We also identified 25 credit card transaction locations geo-referenced from the Abila tourist map to plot on the provided shape files. The other 9 locations were not geo-referenced as there was no associated place on the provided map. Additional non-credit card locations were included in the analysis such as parks, shops and other popular points of interest.
 
GPS data was used to determine a vehicle’s speed (mph) and time spent between waypoints. Knowing when a vehicle was parked is essential to determine potential store visits. We were also able to track a vehicle's motion by animating its travel pattern on a geographical map using SAS Visual Analytics:

Travel Route Animation

A closest known point of interest was assumed if it was less than 0.25 miles away from a parked vehicle (speed less than 5 mph). Taking into account the time of travel revealed that some employees were active all day including night hours. Others show recurring events such as executives playing golf on the weekends.

Utilizing geographic polyline visualization within SAS Visual Analytics also allows the visualization of trajectories and helps when comparing routes taken on different days or cars driven by different employees.

Daily routes taken by CEO

To infer the owners of each credit card, we utilized the previously calculated stop locations of each car. If a given car was near a location at the time of a card transaction, the employee assigned to the car represents a potential match. This method's accuracy decreases if there are multiple candidates for a given transaction.

In order to rank candidates, we used two different approaches:

1. Machine learning / decision tree scoring.
2. Manual modeling and calculation for ranking and scoring.

Leaf nodes to compare candidate scores

With the knowledge about which employee made which purchase, we were able to create purchase profiles. Knowing an individual's credit card transaction history allowed us to validate outlier purchases and highlight anomalies.

Travel Route Network

Analyzing vehicle movement data allowed the identification of travel routes and related stops. Knowing each start and stop location of a given trip allowed us to create a network showing popular travel routes. Comparing visit times (start/stop) and the duration of the stay across employees reveals commonly visited locations like the GAStech headquarters, but also subsets of employees that visit the same location at the same time.

If an employee visits a POI at the same time as someone else we consider this as a meeting event. These meeting events include living in the same household or working together.

Social Company Network

If we consider all possible meetings across all employees we can create a scheduling matrix and identify potential relationships if employees meet regularly. This social network reveals strong relationships between the executives as well as some individuals in the Engineering department.

The final steps in the analysis included the comparison of meeting events. Comparing meeting duration and location not only reveals when individuals arrived at a given location, but also the route they took and other POIs they visited while traveling. This analysis revealed that some employees share a common household, have common social activities (e.g. executives playing golf) or work unusual hours at HQ. A Friday night party just one day before the disappearances is particularly interesting given many employees from both the Engineering and IT department met at the same location.

The Solution - Mini Challenge 3

Risk level through the evening

Solving Mini-Challenge 3 required a deep analysis of social media and documents published during the time of the disappearance. The team used SAS Visual Text analytics to classify text messages into low, medium and high-risk categories depending on whether they relate to emergencies or other generic events. We also uncovered noteworthy events using our automated categorization process (a rally, a fire, an explosion, reckless driving, running a red light, bicycle/pedestrian hit and run and a hostage situation).

We used a risk level model to pinpoint when the risk was rising or falling. The peaks in the visualization represent the fire (first peak), a car being hit (second peak), a hostage situation (all remaining peaks in the latter half of the timeline), and an explosion (the very last peak).

Author Similarity Network

Some messages come from microblogs and contain informative reports (from government, media accounts, and eyewitnesses, for example) as well as chatter/junk/spam. There are re-posts for both types of messages. One way to distinguish between those two types is to identify usernames that may be more trustworthy than others. The term scores from the top 25 authors allowed for the creation of a network that shows pockets of similarity.

As part of our solution, we team also compiled a visualization dashboard to provide first responders a way to monitor potential needs and to prioritize their activities.

Risk Level Dashboard

Results

We compiled our findings in one video highlighting some of the approaches taken when analyzing the VAST challenge data:

The VAST Challenge provides a great opportunity to validate our software against real-world scenarios using complex data sets. Not only do we learn from these projects, but we also send feedback to our development teams to further improve product capabilities for customers.

The team

Spending time on VAST challenges is always fun but also requires a lot of commitment and technical knowledge in various areas of technology. Submitting a solution for these challenges wouldn't have been possible without the help of Riley Benson, Cheryl LeSaint, Rajendra Singh, Don Chapman (MC2) and Biljana Belamaric Wilsey, Russell Albright (MC3). Falko Schulz used SAS Visual Analytics to explore and visualize the data to tell a complete story and focus on the Mini Challenges' questions. Also huge thanks to Rachel Nisbet and Chelsea Mayse for the willingness and thoroughness in producing a beautiful video summary. None of this would have been possible without each of you.

Thanks again to the entire SAS team!

References

• VAST Challenge 2021
• Visual Analytics Benchmark Repository
• YouTube - Submission Video
• SAS Institute Inc. SAS Visual Analytics. (Online.) 2021.

The post Analyzing movement and tracking data using SAS Visual Analytics appeared first on The SAS Data Science Blog.