Simulation Educators

What the Heck is Multivariate Analysis? Part I

noreply@blogger.com (Anonymous) — Mon, 10 Aug 2015 12:56:00 +0000

This article first appeared on bicorner.com

In my next series of articles, I will demonstrate how to use the R statistical software to carry out some simple multivariate analyses, with a focus on principal components analysis (PCA) and linear discriminant analysis (LDA). PCA and LDA, will constitute parts II and III, respectively.

The first thing that you will want to do to analyze your multivariate data will be to load it into R, and to plot the data. You can load dataset into R using the require(rattle), ‘wine’ is a data set in this package. The dataset ‘wine’ contains data on concentrations of 13 different chemicals in wines grown in the same region in Italy that are derived from three different cultivars.

There is one row per wine sample. The first column contains the cultivar of a wine sample (labelled 1, 2 or 3), and the following thirteen columns contain the concentrations of the 13 different chemicals in that sample. The columns are separated by commas. We can load in the file using the require() function as follows:

require(rattle)

wine # a dataset in the rattle package
 V1    V2   V3   V4   V5  V6   V7   V8   V9  V10       V11   V12  V13  V14
 1    1 14.23 1.71 2.43 15.6 127 2.80 3.06 0.28 2.29  5.640000 1.040 3.92 1065
 2    1 13.20 1.78 2.14 11.2 100 2.65 2.76 0.26 1.28  4.380000 1.050 3.40 1050
 3    1 13.16 2.36 2.67 18.6 101 2.80 3.24 0.30 2.81  5.680000 1.030 3.17 1185
 4    1 14.37 1.95 2.50 16.8 113 3.85 3.49 0.24 2.18  7.800000 0.860 3.45 1480
 5    1 13.24 2.59 2.87 21.0 118 2.80 2.69 0.39 1.82  4.320000 1.040 2.93  735
 ...
 176  3 13.27 4.28 2.26 20.0 120 1.59 0.69 0.43 1.35 10.200000 0.590 1.56  835
 177  3 13.17 2.59 2.37 20.0 120 1.65 0.68 0.53 1.46  9.300000 0.600 1.62  840
 178  3 14.13 4.10 2.74 24.5  96 2.05 0.76 0.56 1.35  9.200000 0.610 1.60  560

In this case the data on 178 samples of wine has been read into the variable ‘wine’.

Plotting Multivariate Data

Once you have read a multivariate data set into R, the next step is usually to make a plot of the data.

A Matrix Scatterplot

One common way of plotting multivariate data is to make a “matrix scatterplot”, showing each pair of variables plotted against each other. We can use the “scatterplotMatrix()” function from the “car” R package to do this. To use this function, we first need to install the “car” R package (for instructions on how to install an R package, see How to install an R package).

Once you have installed the “car” R package, you can load the “car” R package by typing:

> library("car")
You can then use the “scatterplotMatrix()” function to plot the multivariate data.

To use the scatterplotMatrix() function, you need to give it as its input the variables that you want included in the plot. Say for example, that we just want to include the variables corresponding to the concentrations of the first five chemicals. These are stored in columns 2-6 of the variable “wine”. We can extract just these columns from the variable “wine” by typing:

> wine[2:6]
 V2   V3   V4   V5 V6
 1   14.23 1.71 2.43 15.6 127
 2   13.20 1.78 2.14 11.2 100
 3   13.16 2.36 2.67 18.6 101
 4   14.37 1.95 2.50 16.8 113
 5   13.24 2.59 2.87 21.0 118
 ...

To make a matrix scatterplot of just these 13 variables using the scatterplotMatrix() function we type:

> scatterplotMatrix(wine[2:6])

In this matrix scatterplot, the diagonal cells show histograms of each of the variables, in this case the concentrations of the first five chemicals (variables V2, V3, V4, V5, V6).

Each of the off-diagonal cells is a scatterplot of two of the five chemicals, for example, the second cell in the first row is a scatterplot of V2 (y-axis) against V3 (x-axis).

A Scatterplot with the Data Points Labelled by their Group

If you see an interesting scatterplot for two variables in the matrix scatterplot, you may want to plot that scatterplot in more detail, with the data points labelled by their group (their cultivar in this case).

For example, in the matrix scatterplot above, the cell in the third column of the fourth row down is a scatterplot of V5 (x-axis) against V4 (y-axis). If you look at this scatterplot, it appears that there may be a positive relationship between V5 and V4.

We may therefore decide to examine the relationship between V5 and V4 more closely, by plotting a scatterplot of these two variable, with the data points labelled by their group (their cultivar). To plot a scatterplot of two variables, we can use the “plot” R function. The V4 and V5 variables are stored in the columns V4 and V5 of the variable “wine”, so can be accessed by typing wine$V4 or wine$V5. Therefore, to plot the scatterplot, we type:

> plot(wine$V4, wine$V5)

If we want to label the data points by their group (the cultivar of wine here), we can use the “text” function in R to plot some text beside every data point. In this case, the cultivar of wine is stored in the column V1 of the variable “wine”, so we type:

> text(wine$V4, wine$V5, wine$V1, cex=0.7, pos=4, col="red")

If you look at the help page for the “text” function, you will see that “pos=4” will plot the text just to the right of the symbol for a data point. The “cex=0.5” option will plot the text at half the default size, and the “col=red” option will plot the text in red. This gives us the following plot:

We can see from the scatterplot of V4 versus V5 that the wines from cultivar 2 seem to have lower values of V4 compared to the wines of cultivar 1.

A Profile Plot

Another type of plot that is useful is a “profile plot”, which shows the variation in each of the variables, by plotting the value of each of the variables for each of the samples.
The function “makeProfilePlot()” below can be used to make a profile plot. This function requires the “RColorBrewer” library. To use this function, we first need to install the “RColorBrewer” R package (for instructions on how to install an R package, see How to install an R package).

> makeProfilePlot {
 require(RColorBrewer)
 # find out how many variables we want to include numvariables # choose 'numvariables' random colours
 colours # find out the minimum and maximum values of the variables: mymin mymax for (i in 1:numvariables)
 {
 vectori mini maxi if (mini < mymin) { mymin mymax) { mymax }
 # plot the variables
 for (i in 1:numvariables)
 {
 vectori namei colouri if (i == 1) { plot(vectori,col=colouri,type="l",ylim=c(mymin,mymax)) }
 else         { points(vectori, col=colouri,type="l")

}
 lastxval lastyval text((lastxval-10),(lastyval),namei,col="black",cex=0.6)
 }
 }

To use this function, you first need to copy and paste it into R. The arguments to the function are a vector containing the names of the varibles that you want to plot, and a list variable containing the variables themselves.

For example, to make a profile plot of the concentrations of the first five chemicals in the wine samples (stored in columns V2, V3, V4, V5, V6 of variable “wine”), we type:

> library(RColorBrewer)
> names mylist makeProfilePlot(mylist,names)

It is clear from the profile plot that the mean and standard deviation for V6 is quite a lot higher than that for the other variables.

Calculating Summary Statistics for Multivariate Data

Another thing that you are likely to want to do is to calculate summary statistics such as the mean and standard deviation for each of the variables in your multivariate data set.

sapply

The “sapply()” function can be used to apply some other function to each column in a data frame, eg. sapply(mydataframe,sd) will calculate the standard deviation of each column in a dataframe “mydataframe”.

This is easy to do, using the “mean()” and “sd()” functions in R. For example, say we want to calculate the mean and standard deviations of each of the 13 chemical concentrations in the wine samples. These are stored in columns 2-14 of the variable “wine”. So we type:

> sapply(wine[2:14],mean)
 V2          V3          V4         V5         V6          V7
 13.0006180   2.3363483   2.3665169 19.4949438 99.7415730   2.2951124
 V8          V9         V10         V11         V12         V13
 2.0292697   0.3618539   1.5908989   5.0580899   0.9574494   2.6116854
 V14
 746.8932584

This tells us that the mean of variable V2 is 13.0006180, the mean of V3 is 2.3363483, and so on.

Similarly, to get the standard deviations of the 13 chemical concentrations, we type:

> sapply(wine[2:14],sd)
 V2          V3          V4          V5         V6          V7
 0.8118265   1.1171461   0.2743440   3.3395638 14.2824835   0.6258510
 V8          V9         V10         V11         V12         V13
 0.9988587   0.1244533   0.5723589   2.3182859   0.2285716   0.7099904
 V14
 314.9074743

We can see here that it would make sense to standardize in order to compare the variables because the variables have very different standard deviations – the standard deviation of V14 is 314.9074743, while the standard deviation of V9 is just 0.1244533. Thus, in order to compare the variables, we need to standardize each variable so that it has a sample variance of 1 and sample mean of 0. We will explain below how to standardize the variables.

Means and Variances Per Group

It is often interesting to calculate the means and standard deviations for just the samples from a particular group, for example, for the wine samples from each cultivar. The cultivar is stored in the column “V1” of the variable “wine”.

To extract out the data for just cultivar 2, we can type:

> cultivar2wine

We can then calculate the mean and standard deviations of the 13 chemicals’ concentrations, for just the cultivar 2 samples:

 sapply(cultivar2wine[2:14],mean)
 V2       V3       V4        V5        V6       V7       V8
 12.278732 1.932676 2.244789 20.238028 94.549296 2.258873 2.080845
 V9      V10      V11      V12      V13        V14
 0.363662 1.630282 3.086620 1.056282 2.785352 519.507042
> sapply(cultivar2wine[2:14])
 V2        V3        V4        V5         V6        V7        V8
 0.5379642 1.0155687 0.3154673 3.3497704 16.7534975 0.5453611 0.7057008
 V9       V10       V11       V12       V13         V14
 0.1239613 0.6020678 0.9249293 0.2029368 0.4965735 157.2112204

You can calculate the mean and standard deviation of the 13 chemicals’ concentrations for just cultivar 1 samples, or for just cultivar 3 samples, in a similar way.

However, for convenience, you might want to use the function “printMeanAndSdByGroup()” below, which prints out the mean and standard deviation of the variables for each group in your data set:

> printMeanAndSdByGroup function(variables,groupvariable)
 {
 # find the names of the variables
 variablenames # within each group, find the mean of each variable
 groupvariable # ensures groupvariable is not a list
 means names(means) print(paste("Means:"))
 print(means)
 # within each group, find the standard deviation of each variable:
 sds names(sds) print(paste("Standard deviations:"))
 print(sds)
 # within each group, find the number of samples:
 samplesizes names(samplesizes) print(paste("Sample sizes:"))
 print(samplesizes)
 }

To use the function “printMeanAndSdByGroup()”, you first need to copy and paste it into R. The arguments of the function are the variables that you want to calculate means and standard deviations for, and the variable containing the group of each sample. For example, to calculate the mean and standard deviation for each of the 13 chemical concentrations, for each of the three different wine cultivars, we type:

> printMeanAndSdByGroup(wine[2:14],wine[1])
 [1] "Means:"
 V1      V2     V3     V4     V5        V6     V7     V8     V9    V10    V11    V12    V13       V14
 1 1 13.7448 2.0107 2.4556 17.0370 106.3390 2.8402 2.9824 0.2900 1.8993 5.5283 1.0620 3.1578 1115.7119
 2 2 12.2787 1.9327 2.2448 20.2380  94.5493 2.2589 2.0808 0.3637 1.6303 3.0866 1.0563 2.7854  519.5070
 3 3 13.1536 3.3338 2.4371 21.4167  99.3125 1.6788 0.7815 0.4475 1.1535 7.3963 0.6827 1.6835  629.8958
 [1] "Standard deviations:"

 V1     V2     V3     V4     V5      V6     V7     V8     V9    V10    V11    V12    V13      V14
 1 1 0.4621 0.6885 0.2272 2.5463 10.4990 0.3390 0.3975 0.0701 0.4121 1.2386 0.1164 0.3571 221.5208
 2 2 0.5380 1.0156 0.3155 3.3498 16.7535 0.5454 0.7057 0.1240 0.6021 0.9250 0.2029 0.4966 157.2112
 3 3 0.5302 1.0879 0.1846 2.2582 10.8905 0.3570 0.2935 0.1241 0.4088 2.3101 0.1144 0.2721 115.0970

[1] "Sample sizes:"
 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14
 1 1  59 59 59 59 59 59 59 59  59  59 59 59 59
 2 2  71 71 71 71 71 71 71 71  71  71 71 71 71
 3 3  48 48 48 48 48 48 48 48  48  48 48 48 48

The function “printMeanAndSdByGroup()” also prints out the number of samples in each group. In this case, we see that there are 59 samples of cultivar 1, 71 of cultivar 2, and 48 of cultivar 3.

Between-groups Variance and Within-groups Variance for a Variable

If we want to calculate the within-groups variance for a particular variable (for example, for a particular chemical’s concentration), we can use the function “calcWithinGroupsVariance()” below:

> calcWithinGroupsVariance function(variable,groupvariable)

{
 # find out how many values the group variable can take
 groupvariable2 levels numlevels # get the mean and standard deviation for each group:
 numtotal denomtotal for (i in 1:numlevels)
 {
 leveli levelidata levelilength # get the standard deviation for group i: sdi numi denomi numtotal denomtotal }
 # calculate the within-groups variance
 Vw return(Vw)
 }

You will need to copy and paste this function into R before you can use it. For example, to calculate the within-groups variance of the variable V2 (the concentration of the first chemical), we type:

> calcWithinGroupsVariance(wine[2],wine[1])
 [1] 0.2620525

Thus, the within-groups variance for V2 is 0.2620525.

We can calculate the between-groups variance for a particular variable (eg. V2) using the function “calcBetweenGroupsVariance()” below:

> calcBetweenGroupsVariance function(variable,groupvariable)

{
 # find out how many values the group variable can take groupvariable2 levels numlevels # calculate the overall grand mean: grandmean # get the mean and standard deviation for each group: numtotal denomtotal for (i in 1:numlevels)
 {
 leveli levelidata levelilength # get the mean and standard deviation for group i:
 meani sdi numi denomi numtotal denomtotal }
 # calculate the between-groups variance Vb Vb return(Vb)
 }

Once you have copied and pasted this function into R, you can use it to calculate the between-groups variance for a variable such as V2:

> calcBetweenGroupsVariance (wine[2],wine[1])
 [1] 35.39742

Thus, the between-groups variance of V2 is 35.39742.

We can calculate the “separation” achieved by a variable as its between-groups variance divided by its within-groups variance. Thus, the separation achieved by V2 is calculated as:

> 35.39742/0.2620525
 [1] 135.0776

If you want to calculate the separations achieved by all of the variables in a multivariate data set, you can use the function “calcSeparations()” below:

> calcSeparations function(variables,groupvariable)

{
 # find out how many variables we have variables numvariables # find the variable names variablenames # calculate the separation for each variable for (i in 1:numvariables)
 {
 variablei variablename Vw Vb sep print(paste("variable",variablename,"Vw=",Vw,"Vb=",Vb,"separation=",sep))
 }
 }

For example, to calculate the separations for each of the 13 chemical concentrations, we type:

> calcSeparations(wine[2:14],wine[1])
 [1] "variable  V2 Vw= 0.2620524692 Vb= 35.39742496027 separation= 135.07762424"
 [1] "variable  V3 Vw= 0.8875467967 Vb= 32.78901848692 separation= 36.943424963"
 [1] "variable  V4 Vw= 0.0660721013 Vb= 0.879611357249 separation= 13.312901199"
 [1] "variable  V5 Vw= 8.0068111812 Vb= 286.4167463631 separation= 35.771637407"
 [1] "variable  V6 Vw= 180.65777316 Vb= 2245.501027890 separation= 12.429584338"
 [1] "variable  V7 Vw= 0.1912704752 Vb= 17.92835729429 separation= 93.733009620"
 [1] "variable  V8 Vw= 0.2747075143 Vb= 64.26119502356 separation= 233.92587268"
 [1] "variable  V9 Vw= 0.0119117022 Vb= 0.328470157461 separation= 27.575417146"
 [1] "variable V10 Vw= 0.2461729438 Vb= 7.451995507778 separation= 30.271383170"
 [1] "variable V11 Vw= 2.2849230813 Vb= 275.7080008223 separation= 120.66401844"
 [1] "variable V12 Vw= 0.0244876469 Vb= 2.481009914938 separation= 101.31679539"
 [1] "variable V13 Vw= 0.1607787296 Vb= 30.54350835443 separation= 189.97232058"
 [1] "variable V14 Vw= 29707.681871 Vb= 6176832.322285 separation= 207.92037390"

Thus, the individual variable which gives the greatest separations between the groups (the wine cultivars) is V8 (separation 233.9). As we will discuss below, the purpose of linear discriminant analysis (LDA) is to find the linear combination of the individual variables that will give the greatest separation between the groups (cultivars here). This hopefully will give a better separation than the best separation achievable by any individual variable (233.9 for V8 here).

Between-groups Covariance and Within-groups Covariance for Two Variables

If you have a multivariate data set with several variables describing sampling units from different groups, such as the wine samples from different cultivars, it is often of interest to calculate the within-groups covariance and between-groups variance for pairs of the variables.

This can be done using the following functions, which you will need to copy and paste into R to use them:

> calcWithinGroupsCovariance function(variable1,variable2,groupvariable)

{
 # find out how many values the group variable can take
 groupvariable2 levels numlevels # get the covariance of variable 1 and variable 2 for each group:
 Covw for (i in 1:numlevels)
 {
 leveli levelidata1 levelidata2 mean1 mean2 levelilength # get the covariance for this group:
 term1 for (j 1:levelilength)
 {
 term1 }
 Cov_groupi # covariance for this group
 Covw }
 totallength Covw return(Covw)
 }

For example, to calculate the within-groups covariance for variables V8 and V11, we type:

> calcWithinGroupsCovariance(wine[8],wine[11],wine[1])
 [1] 0.2866783

> calcBetweenGroupsCovariance function(variable1,variable2,groupvariable)
 {
 # find out how many values the group variable can take
 groupvariable2 levels numlevels # calculate the grand means
 variable1mean variable2mean # calculate the between-groups covariance
 Covb for (i in 1:numlevels)
 {
 leveli levelidata1 levelidata2 mean1 mean2 levelilength term1 Covb }
 Covb Covb return(Covb)
 }

For example, to calculate the between-groups covariance for variables V8 and V11, we type:

> calcBetweenGroupsCovariance(wine[8],wine[11],wine[1])
 [1] -60.41077

Thus, for V8 and V11, the between-groups covariance is -60.41 and the within-groups covariance is 0.29. Since the within-groups covariance is positive (0.29), it means V8 and V11 are positively related within groups: for individuals from the same group, individuals with a high value of V8 tend to have a high value of V11, and vice versa. Since the between-groups covariance is negative (-60.41), V8 and V11 are negatively related between groups: groups with a high mean value of V8 tend to have a low mean value of V11, and vice versa.

Calculating Correlations for Multivariate Data

It is often of interest to investigate whether any of the variables in a multivariate data set are significantly correlated.

To calculate the linear (Pearson) correlation coefficient for a pair of variables, you can use the “cor.test()” function in R. For example, to calculate the correlation coefficient for the first two chemicals’ concentrations, V2 and V3, we type:

> cor.test(wine$V2, wine$V3)

  Pearson's product-moment correlation
 data: wine$V2 and wine$V3
 t = 1.2579, df = 176, p-value = 0.2101
 alternative hypothesis: true correlation is not equal to 0
 95 percent confidence interval:
 -0.05342959 0.23817474
 sample estimates:
 cor
 0.09439694

This tells us that the correlation coefficient is about 0.094, which is a very weak correlation. Furthermore, the P-value for the statistical test of whether the correlation coefficient is significantly different from zero is 0.21. This is much greater than 0.05 (which we can use here as a cutoff for statistical significance), so there is very weak evidence that that the correlation is non-zero.

If you have a lot of variables, you can use “cor.test()” to calculate the correlation coefficient for each pair of variables, but you might be just interested in finding out what are the most highly correlated pairs of variables. For this you can use the function “mosthighlycorrelated()” below.

The function “mosthighlycorrelated()” will print out the linear correlation coefficients for each pair of variables in your data set, in order of the correlation coefficient. This lets you see very easily which pair of variables are most highly correlated.

> mosthighlycorrelated function(mydataframe,numtoreport)
 {
 # find the correlations
 cormatrix # set the correlations on the diagonal or lower triangle to zero,
 # so they will not be reported as the highest ones:
 diag(cormatrix) cormatrix[lower.tri(cormatrix)] # flatten the matrix into a dataframe for easy sorting
 fm # assign human-friendly names
 names(fm) # sort and print the top n correlations
 head(fm[order(abs(fm$Correlation),decreasing=T),],n=numtoreport)
 }

To use this function, you will first have to copy and paste it into R. The arguments of the function are the variables that you want to calculate the correlations for, and the number of top correlation coefficients to print out (for example, you can tell it to print out the largest ten correlation coefficients, or the largest 20).

For example, to calculate correlation coefficients between the concentrations of the 13 chemicals in the wine samples, and to print out the top 10 pairwise correlation coefficients, you can type:

> mosthighlycorrelated(wine[2:14], 10)
 First.Variable Second.Variable Correlation
 84             V7              V8   0.8645635
 150             V8             V13   0.7871939
 149             V7             V13   0.6999494
 111             V8             V10   0.6526918
 157             V2             V14   0.6437200
 110             V7             V10   0.6124131
 154            V12             V13   0.5654683
 132             V3             V12  -0.5612957
 118             V2             V11   0.5463642
 137             V8             V12   0.5434786

This tells us that the pair of variables with the highest linear correlation coefficient are V7 and V8 (correlation = 0.86 approximately).

Standardizing Variables

If you want to compare different variables that have different units, are very different variances, it is a good idea to first standardize the variables.

For example, we found above that the concentrations of the 13 chemicals in the wine samples show a wide range of standard deviations, from 0.1244533 for V9 (variance 0.01548862) to 314.9074743 for V14 (variance 99166.72). This is a range of approximately 6,402,554-fold in the variances.

As a result, it is not a good idea to use the unstandardized chemical concentrations as the input for a principal component analysis (PCA, see below) of the wine samples, as if you did that, the first principal component would be dominated by the variables which show the largest variances, such as V14.

Thus, it would be a better idea to first standardize the variables so that they all have variance 1 and mean 0, and to then carry out the principal component analysis on the standardized data. This would allow us to find the principal components that provide the best low-dimensional representation of the variation in the original data, without being overly biased by those variables that show the most variance in the original data.

You can standardize variables in R using the “scale()” function.

For example, to standardize the concentrations of the 13 chemicals in the wine samples, we type:

> standardizedconcentrations

Note that we use the “as.data.frame()” function to convert the output of “scale()” into a “data frame”, which is the same type of R variable that the “wine” variable.

We can check that each of the standardized variables stored in “standardizedconcentrations” has a mean of 0 and a standard deviation of 1 by typing:

> sapply(standardizedconcentrations,mean)
 V2           V3           V4           V5           V6           V7
 -8.591766e-16 -6.776446e-17 8.045176e-16 -7.720494e-17 -4.073935e-17 -1.395560e-17
 V8           V9         V10           V11           V12           V13
 6.958263e-17 -1.042186e-16 -1.221369e-16 3.649376e-17 2.093741e-16 3.003459e-16
 V14
 -1.034429e-16
> sapply(standardizedconcentrations,sd)
 V2 V3  V4  V5  V6  V7  V8  V9 V10 V11 V12 V13 V14
 1  1   1   1   1   1   1   1   1   1   1   1   1

We see that the means of the standardized variables are all very tiny numbers and so are essentially equal to 0, and the standard deviations of the standardized variables are all equal to 1.

Authored by: Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices
Weird Scientist: the Creators of Quantum Physics
Albert Einstein: No one expected me to lay a golden eggs
The Men of Manhattan: the Creators of the Nuclear Era
Fundamentals of Combat Modeling
LinkedIn Memoirs
Quantum Phaith
Dear Mister President
Handbook of Handguns
Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

A 12 Step Approach to Analytical Modeling

noreply@blogger.com (Anonymous) — Mon, 10 Aug 2015 12:31:00 +0000

This article first appeared on bicorner.com

Define the problem – Defining the problem is necessary. Defining the “right” problem is absolutely critical, otherwise you’re just wasting everyone’s time. However, the customers you are working for may not know how to express the real problem or may not know what the problem really is. The Operations Research Analyst must ask the right questions and draw the right problem out form where it may be hiding.
Define the Business case – Once you identify the real problem, you have to help answer the question, “Is the problem worth solving (or at least worth the OR analyst’s involvement)?”. That may sound odd, but if the problem statement is not translated in to a business case, then the business does not need the problem solved badly enough to warrant the time and effort to solve it. We have to ask, “What is it worth to the business?” Increased profit? Improved production? Savings of marketing dollars?
Define the Model Objective – It takes a well-defined problem and solid business case to ensure that we build the right model or provide the right solution. In my earlier years, I often found myself halfway through a project before realizing I was not building the right model. A model objective that answers the call of the business case is essential. It will keep you on course.
Determine the requirements – We have a well-defined problem, a business case with a supporting model objective, so now let’s go and build the model. Slow down! What are the requirements? Do you have the tools you need, the team you need, and the budget to support it? If data is to be used, do you have access to it, has it been processed, and is it in the right format? We have to determine the requirements or we may fail due to some oversight.
Gather the Data – If data is necessary, it will probably not be handed to you on a silver platter ready for consumption. It may be “dirty”, in the wrong format, incomplete, of just not right. If you have fully identified the requirements, you will already know what has to be accomplished at this stage.
Process the Data – This is where you may spend an abundant amount of time. If you do not have clean, complete, and reliable data, you are doomed. You may have to remove inconsistencies, impute missing values, and so on. Then you have to analyze the data, perform data reduction, and integrate the data so that it is ready for use. Modeling with “bad” data results in a “bad” model!
Build the Model – So, you have done the hard part and it’s time to have fun. Building the model is an iterative process. I doubt that anyone ever uses the first model they build. If they do, they probably should not. There is as much art as there is science in model development. Modeler judgment is as important as p-values, and Chi-square tests. If the model does not make sense to you—is not intuitive—then it is probably not the final model.
Interpret the Model Results – Here is where we often have the most difficulty as analysts. We understand the mathematical and statistical interpretation of results, but we have to go beyond that and translate it to the customer’s language. They are not going to get the significance of the odds ratio of variable X. If you start thinking about that here, it will make life less painful later on.
Validate the Model for Production – This is purely a scientific task. The model must be documented every step of the way and then stand up to the scrutiny of stress testing, peer review, and so on. This is where you find out if you really did build the right model, and you built it right.
Perform an Economic Analysis – Here is where you answer the call of the business case. You must translate the results to dollars, the economic benefit of the solution, or another metric if savings or profit are not objectives. For instance, the military OR may be saving lives instead of dollars.
Present the Results – If you bothered to interpret the results and translate them to “customer speak”, this part is easy. But, you are not just presenting results, you are selling the solution. If your work and your recommendations are not implemented, you just wasted a lot of time, effort and money.
Follow-up – Too often we fix the problem and go back to our lives or onto another problem, but following up with the customer to evaluate the effectiveness of the solution you provided is just as important as the solution that you provided. It is especially important if you want to ever see that customer again.
Conclusion
So there is my view of model building. I have built just about every kind of model: linear programs, nonlinear programs, machine learning models, statistical models, Markov transition models, and so on. I have built models of unmanned aerial vehicles, space launch vehicles, communications networks, software systems, propensity to buy a product, call center operations, combat phenomena, and so on. And I have, more than I would like to admit, built the wrong model. Hopefully, this will help the modeler step out on the right foot, and help the model recipient know what to look for when asking for a modeling solution.

Authored by:Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of Predictive Analytics Using R and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional (ASEP). He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices

Connect with Jeffrey StricklandContact Jeffrey Strickland

Naïve Bayes using R

noreply@blogger.com (Anonymous) — Mon, 10 Aug 2015 12:25:00 +0000

This article was first published on bicorner.com

The following is an excerpt from Predictive Analytics using R, Copyright 2015 by Jeffrey Strickland
In machine learning, Naïve Bayes classifiers are a family of simple probabilistic classifiers based on applying Bayes’ theorem with strong (naïve) independence assumptions between the features.

Naïve Bayes is a popular (baseline) method for text categorization, the problem of judging documents as belonging to one category or the other (such as spam or legitimate, sports or politics, etc.) with word frequencies as the features. With appropriate preprocessing, it is competitive in this domain with more advanced methods including support vector machines (Rennie, Shih, Teevan, & Karger, 2003).

Training Naïve Bayes can be done by evaluating an approximation algorithm in closed form in linear time, rather than by expensive iterative approximation.

Introduction

In simple terms, a Naïve Bayes classifier assumes that the value of a particular feature is unrelated to the presence or absence of any other feature, given the class variable. For example, a fruit may be considered to be an apple if it is red, round, and about 3” in diameter. A Naïve Bayes classifier considers each of these features to contribute independently to the probability that this fruit is an apple, regardless of the presence or absence of the other features.

For some types of probability models, Naïve Bayes classifiers can be trained very efficiently in a supervised learning setting. In many practical applications, parameter estimation for Naïve Bayes models uses the method of maximum likelihood; in other words, one can work with the Naïve Bayes model without accepting Bayesian probability or using any Bayesian methods.

Despite their naïve design and apparently oversimplified assumptions, Naïve Bayes classifiers have worked quite well in many complex real-world situations. In 2004, an analysis of the Bayesian classification problem showed that there are sound theoretical reasons for the apparently implausible efficacy of Naïve Bayes classifiers (Zhang, 2004). Still, a comprehensive comparison with other classification algorithms in 2006 showed that Bayes classification is outperformed by other approaches, such as boosted trees or random forests (Caruana & Niculescu-Mizil, 2006).

An advantage of Naïve Bayes is that it only requires a small amount of training data to estimate the parameters (means and variances of the variables) necessary for classification. Because independent variables are assumed, only the variances of the variables for each class need to be determined and not the entire covariance matrix.

Discussion

Despite the fact that the far-reaching independence assumptions are often inaccurate, the Naïve Bayes classifier has several properties that make it surprisingly useful in practice. In particular, the decoupling of the class conditional feature distributions means that each distribution can be independently estimated as a one-dimensional distribution. This helps alleviate problems stemming from the curse of dimensionality, such as the need for data sets that scale exponentially with the number of features. While Naïve Bayes often fails to produce a good estimate for the correct class probabilities, this may not be a requirement for many applications. For example, the Naïve Bayes classifier will make the correct MAP decision rule classification so long as the correct class is more probable than any other class. This is true regardless of whether the probability estimate is slightly, or even grossly inaccurate. In this manner, the overall classifier can be robust enough to ignore serious deficiencies in its underlying naïve probability model. Other reasons for the observed success of the Naïve Bayes classifier are discussed in the literature cited below.

Example Using R

The Iris dataset is pre-installed in R, since it is in the standard datasets package. To access its documentation, click on ‘Packages’ at the top-level of the R documentation, then on ‘datasets’ and then on ‘iris’. As explained, there are 150 data points and 5 variables. Each data point concerns a particular iris flower and gives 4 measurements of the flower: Sepal.Length, Sepal.Width, Petal.Length and Petal.Width together with the flower’s Species. The goal is to build a classifier that predicts species from the 4 measurements, so species is the class variable.

To get the iris dataset into your R session, do:

> data(iris)

at the R prompt. As always, it makes sense to look at the data. The following R command (from the Wikibook) does a nice job of this.

> pairs(iris[1:4],main=“Iris Data + (red=setosa,green=versicolor,blue=virginica)”, pch=21, + bg=c(“red”,”green3”,”blue”)[unclass(iris$Species)])

The ‘pairs’ command creates a scatterplot. Each dot is a data point and its position is determined by the values that data point has for a pair of variables. The class determines the color of the data point. From the plot note that Setosa irises have smaller petals than the other two species.

Typing:

> summary(iris)

provides a summary of the data.

Sepal.Length     Sepal.Width    Petal.Length     Petal.Width        Species
Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100   setosa   :50
1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  versicolor:50
Median :5.800   Median :3.000   Median :4.350   Median :1.300  virginica :50
Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199
3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800
Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500

Typing:

> iris

Prints out the entire dataset to the screen.

Sepal.Length Sepal.Width Petal.Length Petal.Width Species 1 5.1 3.5 1.4 0.2 setosa 2 4.9 3 1.4 0.2 setosa 3 4.7 3.2 1.3 0.2 setosa 4 4.6 3.1 1.5 0.2 setosa 5 5 3.6 1.4 0.2 setosa -----------------------------Data omitted------------------------- 149 6.2 3.4 5.4 2.3 virginica 150 5.9 3 5.1 1.8 virginica

Constructing a Naïve Bayes classifier

We will use the e1071 R package to build a Naïve Bayes classifier. Firstly you need to download the package (since it is not pre-installed here). Do:

> install.packages(“e1071”)

Choose a mirror in US from the menu that will appear. You will be prompted to create a personal R library (say yes) since you don’t have permission to put e1071 in the standard directory for R packages.

To (1) load e1071 into your workspace (2) build a Naïve Bayes classifier and (3) make some predictions on the training data, do:

> library(e1071) > classifier<-naivebayes br="" iris="">> table(predict(classifier, iris[,-5]), iris[,5], + dnn=list(‘predicted’,’actual’))

As you should see the classifier does a pretty good job of classifying. Why is this not surprising?

predicted setosa versicolor virginica setosa 50 0 0 versicolor 0 47 3 virginica 0 3 47

To see what’s going on ‘behind-the-scenes’, first do:

> classifier$apriori

This gives the class distribution in the data: the prior distribution of the classes. (‘A priori’ is Latin for ‘from before’.)

iris[, 5] setosa versicolor virginica 50 50 50

Since the predictor variables here are all continuous, the Naïve Bayes classifier generates three Gaussian (Normal) distributions for each predictor variable: one for each value of the class variable Species. If you type:

> classifier$tables$Petal.Length

You will see the mean (first column) and standard deviation (second column) for the 3 class-dependent Gaussian distributions:

Petal.Length iris[, 5] [,1] [,2] setosa 1.462 0.1736640 versicolor 4.260 0.4699110 virginica 5.552 0.5518947

You can plot these three distributions against each other with the following three R commands:

> plot(function(x) dnorm(x, 1.462, 0.1736640), 0, 8, + col=“red”, main=“Petal length distribution for the 3 + different species”) > curve(dnorm(x, 4.260, 0.4699110), add=TRUE, col=“blue”) > curve(dnorm(x, 5.552, 0.5518947), add=TRUE, col=“green”)

Note that setosa irises (the red curve) tend to have smaller petals (mean value = 1.462) and there is less variation in petal length (standard deviation is only 0.1736640).

Understanding Naïve Bayes

In the previous example you were given a recipe which allowed you to construct a Naïve Bayes classifier. This was for a case where we had continuous predictor variables. In this question you have to work out what the parameters of a Naïve Bayes model should be for some discrete data.

The dataset in question is called HairEyeColor and has three variables: Sex, Eye and Hair, giving values for these 3 variables for each of 592 students from the University of Delaware. First have a look at the numbers:

> HairEyeColor
, , Sex = Male
Eye Hair Brown Blue Hazel Green Black 32 11 10 3 Brown 53 50 25 15 Red 10 10 7 7 Blond 3 30 5 8
, , Sex = Female
Eye Hair Brown Blue Hazel Green Black 36 9 5 2 Brown 66 34 29 14 Red 16 7 7 7 Blond 4 64 5 8

You can also plot it as a ‘mosaic’ plot which uses rectangles to represent the numbers in the data:

> mosaicplot(HairEyeColor)

Mosaic plot

Your job here is to compute the parameters for a Naïve Bayes classifier which attempts to predict Sex from the other two variables. The parameters should be estimated using maximum likelihood. To save you the tedium of manual counting, here’s how to use margin.table to get the counts you need:

> margin.table(HairEyeColor,3)
Sex Male Female 279 313
> margin.table(HairEyeColor,c(1,3))Sex Hair Male Female Black 56 52 Brown 143 143 Red 34 37 Blond 46 81

Note that Sex is variable 3, and Hair is variable 1. Once you think you have the correct parameters speak to me or one of the demonstrators to see if you have it right. (Or if you can manage it, construct the Naïve Bayes model using the naiveBayes function and yank out the parameters from the model. Read the documentation to do this.)

We can also perform Naïve Bayes with the Classification and Visualization (klaR) package which was not covered in this article.

Authored by: Jeffrey Strickland, Ph.D.

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices

Connect with Jeffrey StricklandContact Jeffrey Strickland

What is Time-Series Analysis, Part 2?

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 21:38:00 +0000

This article first appeared on bicorner.com

In my last article I talked about Time Series Analysis using R and left with simple exponential smoothing. Here I want to talk about more advanced smoothing methods.

Holt’s Exponential Smoothing

If you have a time series that can be described using an additive model with increasing or decreasing trend and no seasonality, you can use Holt’s exponential smoothing to make short-term forecasts.

Holt’s exponential smoothing estimates the level and slope at the current time point. Smoothing is controlled by two parameters, alpha, for the estimate of the level at the current time point, and beta for the estimate of the slope b of the trend component at the current time point. As with simple exponential smoothing, the parameters alpha and beta have values between 0 and 1, and values that are close to 0 mean that little weight is placed on the most recent observations when making forecasts of future values.

An example of a time series that can probably be described using an additive model with a trend and no seasonality is the time series of US Treasury bill contracts on the Chicago market for 100 consecutive trading days in 1981.

We can read in and plot the data in R by typing:

> install.packages("fma")
> ustreasseries > plot.ts(ustreasseries)

To make forecasts, we can fit a predictive model using the HoltWinters() function in R. To use HoltWinters() for Holt’s exponential smoothing, we need to set the parameter gamma=FALSE (the gamma parameter is used for Holt-Winters exponential smoothing, as described below).

For example, to use Holt’s exponential smoothing to fit a predictive model for US Treasury bill contracts, we type:

> ustreasseriesforecasts ustreasseriesforecasts
 Holt-Winters exponential smoothing with trend and without seasonal component.
 Call:
 HoltWinters(x = ustreasseries, gamma = FALSE) Smoothing parameters:
 alpha:
 1 beta : 0.01073
 gamma: FALSE
 Coefficients:
 [,1]
 a 85.32000000
 b -0.04692665
> ustreasseriesforecasts$SSE
 [1] 8.707433

The estimated value of alpha is 1.0, and of beta is 0.0107. These are both high, telling us that both the estimate of the current value of the level, and of the slope b of the trend component, are based mostly upon very recent observations in the time series. This makes good intuitive sense, since the level and the slope of the time series both change quite a lot over time. The value of the sum-of-squared-errors for the in-sample forecast errors is 8.707.

We can plot the original time series as a black line, with the forecasted values as a red line on top of that, by typing:

> plot(ustreasseriesforecasts)

We can see from the picture that the in-sample forecasts agree pretty well with the observed values, although they tend to lag behind the observed values a little bit.

If you wish, you can specify the initial values of the level and the slope b of the trend component by using the “l.start” and “b.start” arguments for the HoltWinters() function. It is common to set the initial value of the level to the first value in the time series (92 for the Chicago market data), and the initial value of the slope to the second value minus the first value (0.1 for the Chicago market data). For example, to fit a predictive model to the Chicago market data using Holt’s exponential smoothing, with initial values of 608 for the level and 0.1 for the slope b of the trend component, we type:

> HoltWinters(ustreasseries, gamma=FALSE, l.start=92, b.start=0.1)
 Holt-Winters exponential smoothing with trend and without seasonal component.
 Call:
 HoltWinters(x = ustreasseries, gamma = FALSE, l.start = 92, b.start = 0.1)
 Smoothing parameters:
 alpha: 1
 beta : 0.04529601
 gamma: FALSE
 Coefficients:
 [,1]
 a 85.32000000
 b -0.08157293
> ustreasseriesforecasts$SSE
 [1] 8.707433

As for simple exponential smoothing, we can make forecasts for future times not covered by the original time series by using the forecast.HoltWinters() function in the “forecast” package. For example, our time series data for the Chicago market was for 0 to 100 days, so we can make predictions for 101-120 (20 more data points), and plot them, by typing:

> ustreasseriesforecasts2> plot.forecast(ustreasseriesforecasts2)

The forecasts are shown as a blue line, with the 80% prediction intervals as an gray shaded area, and the 95% prediction intervals as a light gray shaded area.

As for simple exponential smoothing, we can check whether the predictive model could be improved upon by checking whether the in-sample forecast errors show non-zero autocorrelations at lags 1-20. For example, for the Chicago market data, we can make a correlogram, and carry out the Ljung-Box test, by typing:

> acf(ustreasseriesforecasts2$residuals, lag.max=20)

> Box.test(ustreasseriesforecasts2$residuals, lag=30, type="Ljung-Box")
 Box-Ljung test
 data:  ustreasseriesforecasts2$residuals
 X-squared = 30.4227, df = 30, p-value = 0.4442

Here the correlogram shows that the sample autocorrelation for the in-sample forecast errors at lag 15 exceeds the significance bounds. However, we would expect one in 20 of the autocorrelations for the first twenty lags to exceed the 95% significance bounds by chance alone. Indeed, when we carry out the Ljung-Box test, the p-value is 0.4442, indicating that there is little evidence of non-zero autocorrelations in the in-sample forecast errors at lags 1-20.

As for simple exponential smoothing, we should also check that the forecast errors have constant variance over time, and are normally distributed with mean zero. We can do this by making a time plot of forecast errors, and a histogram of the distribution of forecast errors with an overlaid normal curve:

> plot.ts(ustreasseriesforecasts2$residuals)           # make a time plot
> plotForecastErrors(ustreasseriesforecasts2$residuals) # make a histogram

The time plot of forecast errors shows that the forecast errors have roughly constant variance over time. The histogram of forecast errors show that it is plausible that the forecast errors are normally distributed with mean zero and constant variance.

Thus, the Ljung-Box test shows that there is little evidence of autocorrelations in the forecast errors, while the time plot and histogram of forecast errors show that it is plausible that the forecast errors are normally distributed with mean zero and constant variance. Therefore, we can conclude that Holt’s exponential smoothing provides an adequate predictive model for the US Treasury bill contracts on the Chicago market, which probably cannot be improved upon. In addition, it means that the assumptions that the 80% and 95% predictions intervals were based upon are probably valid.

Holt-Winters Exponential Smoothing

If you have a time series that can be described using an additive model with increasing or decreasing trend and seasonality, you can use Holt-Winters exponential smoothing to make short-term forecasts.

Holt-Winters exponential smoothing estimates the level, slope and seasonal component at the current time point. Smoothing is controlled by three parameters: alpha, beta, and gamma, for the estimates of the level, slope b of the trend component, and the seasonal component, respectively, at the current time point. The parameters alpha, beta and gamma all have values between 0 and 1, and values that are close to 0 mean that relatively little weight is placed on the most recent observations when making forecasts of future values.

An example of a time series that can probably be described using an additive model with a trend and seasonality is the time series of the US Birth data we have already used (discussed in previous article: “What is Time Series Analysis?)

> birthtimeseriesforecasts birthtimeseriesforecasts
 Holt-Winters exponential smoothing with trend and additive seasonal component.
 Call:
 HoltWinters(x = birthtimeseries) Smoothing
 parameters:
 alpha: 0.5388189
 beta : 0.005806264
 gamma: 0.1674998
 Coefficients:
 [,1]
 a   279.37339259
 b    -0.07077833
 s1  -24.22000167
 s2   -1.13886091
 s3  -19.75018141
 s4  -10.15418869
 s5  -12.58375567
 s6   11.96501002
 s7   20.74652186
 s8   18.21566037
 s9   12.93338079
 s10  -4.86769013
 s11   4.49894836
 s12 -4.59412722
> birthtimeseriesforecasts$SSE
 [1] 19571.77

The estimated values of alpha, beta and gamma are 0.5388, 0.0058, and 0.1675, respectively. The value of alpha (0.5388) is relatively low, indicating that the estimate of the level at the current time point is based upon both recent observations and some observations in the more distant past. The value of beta is 0.0058, indicating that the estimate of the slope b of the trend component is not updated over the time series, and instead is set equal to its initial value. This makes good intuitive sense, as the level changes quite a bit over the time series, but the slope b of the trend component remains roughly the same. In contrast, the value of gamma (0.1675) is low, indicating that the estimate of the seasonal component at the current time point is based upon less recent observations.

As for simple exponential smoothing and Holt’s exponential smoothing, we can plot the original time series as a black line, with the forecasted values as a red line on top of that:

> plot(birthtimeseriesforecasts)

We see from the plot that the Holt-Winters exponential method is very successful in predicting the seasonal peaks, which occur roughly in November every year.

To make forecasts for future times not included in the original time series, we use the “forecast.HoltWinters()” function in the “forecast” package. For example, the original data for the monthly live births (adjusted) in thousands for the United States, is from January 1946 to January 1977. If we wanted to make forecasts for February 1977 to January 1981 (48 more months), and plot the forecasts, we would type:

> birthtimeseriesforecasts2 plot.forecast(birthtimeseriesforecasts2)

The forecasts are shown as a blue line, and the gray and light gray shaded areas show 80% and 95% prediction intervals, respectively.

We can investigate whether the predictive model can be improved upon by checking whether the in-sample forecast errors show non-zero autocorrelations at lags 1-20, by making a correlogram and carrying out the Ljung-Box test:

> acf(birthtimeseriesforecasts2$residuals, lag.max=30)
> Box.test(birthtimeseriesforecasts2$residuals, lag=30, type="Ljung-Box")
 Box-Ljung test
 data: birthtimeseriesforecasts2$residuals
 X-squared = 81.1214, df = 30, p-value = 1.361e-06

The correlogram shows that the autocorrelations for the in-sample forecast errors do not exceed the significance bounds for lags 1-20. Furthermore, the p-value for Ljung-Box test is 0.6, indicating that there is little evidence of non-zero autocorrelations at lags 1-20.

We can check whether the forecast errors have constant variance over time, and are normally distributed with mean zero, by making a time plot of the forecast errors and a histogram (with overlaid normal curve):

> plot.ts(birthtimeseriesforecasts2$residuals)           # make a time plot
> plotForecastErrors(birthtimeseriesforecasts2$residuals) # make a histogram

From the time plot, it appears plausible that the forecast errors have constant variance over time. From the histogram of forecast errors, it seems plausible that the forecast errors are normally distributed with mean zero.

Thus, there is little evidence of autocorrelation at lags 1-20 for the forecast errors, and the forecast errors appear to be normally distributed with mean zero and constant variance over time. This suggests that Holt-Winters exponential smoothing provides an adequate predictive model of the monthly live births (adjusted) in thousands for the United States, 1946-1979, which probably cannot be improved upon. Furthermore, the assumptions upon which the prediction intervals were based are probably valid.

Authored by: Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices
Weird Scientist: the Creators of Quantum Physics
Albert Einstein: No one expected me to lay a golden eggs
The Men of Manhattan: the Creators of the Nuclear Era
Fundamentals of Combat Modeling
LinkedIn Memoirs
Quantum Phaith
Dear Mister President
Handbook of Handguns
Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

What is Time-Series Analysis, Part 1

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 21:09:00 +0000

This article first appeared on bicorner.com

This article shows you how to use the R statistical software to carry out some simple analyses that are common in analyzing time series data. If the reader has some basic knowledge of time series analysis it will serve them well since the principal focus of the article is not to explain time series analysis, but rather to explain how to carry out these analyses using R.

Sometimes the time series data set that you have may have been collected at regular intervals that were less than one year, for example, monthly or quarterly. In this case, you can specify the number of times that data was collected per year by using the ‘frequency’ parameter in the ts() function. For monthly time series data, you set frequency=12, while for quarterly time series data, you set frequency=4.

You can also specify the first year that the data was collected, and the first interval in that year by using the ‘start’ parameter in the ts() function. For example, if the first data point corresponds to the second quarter of 1986, you would set start=c(1986,2).

An example is a data set of the monthly live births (adjusted) in thousands for the United States, 1946-1979. The set ‘birth’ is part of the astsa package.

> require(astsa)
> birth ## monthly live births (adjusted) in thousands for the United States, 1946-1979.

Once you have read the time series data into R, the next step is to store the data in a time series object in R, so that you can use R’s many functions for analyzing time series data. To store the data in a time series object, we use the ts() function in R. For example, to store the data in the variable ‘birth’ as a time series object in R, we type:

< birthtimeseries < birthtimeseries
     Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
1946 295 286 300 278 272 268 308 321 313 308 291 296
1947 294 273 300 271 282 285 318 323 313 311 291 293
1948 297 273 294 259 276 294 316 325 315 312 292 301
1949 304 282 313 296 313 307 328 334 329 329 304 312
1950 312 300 317 292 300 311 345 350 344 336 315 323
1951 322 296 315 287 307 321 354 356 348 334 320 340
1952 332 302 324 305 318 329 359 363 359 352 335 342
1953 329 306 332 309 326 325 354 367 362 354 337 345
1954 339 325 345 309 315 334 370 383 375 370 344 355
1955 346 317 348 331 345 348 380 381 377 376 348 356
1956 344 320 347 326 343 338 361 368 378 374 347 358
1957 349 323 358 331 338 343 374 380 377 368 346 358
1958 338 329 347 327 335 336 370 399 385 368 351 362
1959 358 333 356 335 348 346 374 386 384 372 343 346
1960 346 318 359 328 333 329 366 373 367 363 337 346
1961 355 314 343 322 336 327 362 366 361 358 327 330
1962 336 326 337 316 331 331 359 350 356 347 328 336
1963 315 292 322 291 302 310 330 335 333 318 305 313
1964 301 281 302 291 297 291 311 319 317 317 296 307
1965 295 265 300 271 291 290 310 318 310 304 285 288
1966 277 260 282 274 288 287 308 312 306 304 282 305
1967 284 273 286 284 294 288 315 322 317 309 295 306
1968 300 275 301 292 298 306 326 332 329 328 308 324
1969 299 284 306 290 292 285 295 306 317 305 294 287
1970 278 261 275 256 270 264 265 284 284 275 269 275
1971 259 244 267 255 260 253 267 277 277 264 255 260
1972 261 238 257 246 254 255 273 276 286 283 261 276
1973 264 243 259 250 262 253 280 288 270 273 241 266
1974 257 242 266 241 252 250 281 278 286 278 260 272
1975 274 256 276 259 273 272 297 296 290 282 262 275
1976 262 251 285 260 272 265 296 312 289 282 274 281
1977 277

Plotting Time Series

Once you have read a time series into R, the next step is usually to make a plot of the time series data, which you can do with the plot.ts() function in R. To plot the time series of the monthly live births (adjusted) in thousands for the United States, we type:

< plot.ts(birthtimeseries)

We can see from this time series that there seems to be seasonal variation in the number of births per month: there is a peak every summer, and a trough every winter. Again, it seems that this time series could probably be described using an additive model, as the seasonal fluctuations are roughly constant in size over time and do not seem to depend on the level of the time series, and the random fluctuations also seem to be roughly constant in size over time.

Decomposing Seasonal Data

A seasonal time series consists of a trend component, a seasonal component and an irregular component. Decomposing the time series means separating the time series into these three components: that is, estimating these three components.

To estimate the trend component and seasonal component of a seasonal time series that can be described using an additive model, we can use the “decompose()” function in R. This function estimates the trend, seasonal, and irregular components of a time series that can be described using an additive model.

The function “decompose()” returns a list object as its result, where the estimates of the seasonal component, trend component and irregular component are stored in named elements of that list objects, called “seasonal”, “trend”, and “random” respectively.

For example, as discussed above, the time series of the monthly live births (adjusted) in thousands for the United States is seasonal with a peak every summer and trough every winter, and can probably be described using an additive model since the seasonal and random fluctuations seem to be roughly constant in size over time:

> birthtimeseriescomponents

The estimated values of the seasonal, trend and irregular components are now stored in variables birthtimeseriescomponents$seasonal, birthtimeseriescomponents$trend and birthtimeseriescomponents$random. For example, we can print out the estimated values of the seasonal component by typing:

> birthtimeseriescomponents$seasonal # get the estimated values of the seasonal component
 Jan   Feb   Mar   Apr   May   Jun   Jul   Aug   Sep   Oct   Nov   Dec
 1946  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1947  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1948  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1949  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1950  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1951  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1952  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1953  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1954  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1955  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1956  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1957  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1958  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1959  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1960  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1961  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1962  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1963  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1964  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1965  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1966  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1967  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1968  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1969  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1970  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1971  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1972  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1973  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1974  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1975  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1976  -4.3 -25.1  -1.1 -21.6  -9.9  -9.2  16.5  23.6  20.2  13.8  -6.2   3.2
 1977  -4.3

The estimated seasonal factors are given for the months January-December, and are the same for each year. The largest seasonal factor is for August (about 23.6), and the lowest is for February (about -25.1), indicating that there seems to be a peak in births in July and a trough in births in February each year.

We can plot the estimated trend, seasonal, and irregular components of the time series by using the “plot()” function, for example:

> plot(birthtimeseriescomponents)

The plot below shows the original time series (top), the estimated trend component (second from top), the estimated seasonal component (third from top), and the estimated irregular component (bottom). We see that the estimated trend component shows a small increase from about 300 in 1947 to about 360 in 1959, followed by a steady decrease from then on to about 280 in 1966, followed by a slight increase to about 320 in 1969, followed by a steady decease to about 260 from about 1971 through 1974 and the a slight increase to about 280 in 1979.

Seasonally Adjusting

If you have a seasonal time series that can be described using an additive model, you can seasonally adjust the time series by estimating the seasonal component, and subtracting the estimated seasonal component from the original time series. We can do this using the estimate of the seasonal component calculated by the “decompose()” function.

For example, to seasonally adjust the time series of the number of births per month in New York city, we can estimate the seasonal component using “decompose()”, and then subtract the seasonal component from the original time series:

> birthtimeseriescomponents > birthtimeseriesseasonallyadjusted

We can then plot the seasonally adjusted time series using the “plot()” function, by typing:

> plot(birthtimeseriesseasonallyadjusted)

You can see that the seasonal variation has been removed from the seasonally adjusted time series. The seasonally adjusted time series now just contains the trend component and an irregular component.

Forecasts using Exponential Smoothing

Exponential smoothing can be used to make short-term forecasts for time series data.

Simple Exponential Smoothing

If you have a time series that can be described using an additive model with constant level and no seasonality, you can use simple exponential smoothing to make short-term forecasts.

The simple exponential smoothing method provides a way of estimating the level at the current time point. Smoothing is controlled by the parameter alpha; for the estimate of the level at the current time point. The value of alpha; lies between 0 and 1. Values of alpha that are close to 0 mean that little weight is placed on the most recent observations when making forecasts of future values.

To make forecasts using simple exponential smoothing in R, we can fit a simple exponential smoothing predictive model using the “HoltWinters()” function in R. To use HoltWinters() for simple exponential smoothing, we need to set the parameters beta=FALSE and gamma=FALSE in the HoltWinters() function (the beta and gamma parameters are used for Holt’s exponential smoothing, or Holt-Winters exponential smoothing, as described below).

The HoltWinters() function returns a list variable, that contains several named elements.

For example, to use simple exponential smoothing to make forecasts for the time series of monthly live births (adjusted) in thousands for the United States, 1946-1979, we type:

## Holt-Winters exponential smoothing without trend and without seasonal component.

> birthtimeseriesforecasts > birthtimeseriesforecasts
Call:
HoltWinters(x = birthtimeseries, beta = FALSE, gamma = FALSE)
Smoothing parameters:
  alpha: 0.71
  beta : FALSE
  gamma: FALSE
Coefficients:
  [,1]
a  278

The output of HoltWinters() tells us that the estimated value of the alpha parameter is about 0.71. This is very close to zero, telling us that the forecasts are based on both recent and less recent observations (although somewhat more weight is placed on recent observations).

By default, HoltWinters() just makes forecasts for the same time period covered by our original time series. In this case, our original time series included monthly live births (adjusted) in thousands for the United States, 1948-1979.

In the example above, we have stored the output of the HoltWinters() function in the list variable “birthstimeseriesforecasts”. The forecasts made by HoltWinters() are stored in a named element of this list variable called “fitted”, so we can get their values by typing:

> birthtimeseriesforecasts $fitted
 xhat level
 Feb 1946  295   295
 Mar 1946  289   289
 Apr 1946  297   297
 May 1946  283   283
 Jun 1946  275   275
 Jul 1946  270   270
 Aug 1946  297   297
 Sep 1946  314   314
 Oct 1946  313   313
 Nov 1946  310   310
 Dec 1946  296   296
 Jan 1947  296   296
 Feb 1947  295   295
 Mar 1947  279   279
 Apr 1947  294   294
 May 1947  278   278
 Jun 1947  281   281
 Jul 1947  284   284
 Aug 1947  308   308
 Sep 1947  319   319
 Oct 1947  315   315
 Nov 1947  312   312
 Dec 1947  297   297
 .
 .
 .
 Jan 1976  273   273
 Feb 1976  265   265
 Mar 1976  255   255
 Apr 1976  276   276
 May 1976  265   265
 Jun 1976  270   270
 Jul 1976  266   266
 Aug 1976  287   287
 Sep 1976  305   305
 Oct 1976  294   294
 Nov 1976  285   285
 Dec 1976  277   277
 Jan 1977  280   280

We can plot the original time series against the forecasts by typing:

> plot(birthtimeseriesforecasts)

The plot shows the original time series in black, and the forecasts as a red line. The time series of forecasts is much smoother than the time series of the original data here.

As a measure of the accuracy of the forecasts, we can calculate the sum of squared errors for the in-sample forecast errors, that is, the forecast errors for the time period covered by our original time series. The sum-of-squared-errors is stored in a named element of the list variable “birthtimeseriesforecasts” called “SSE”, so we can get its value by typing:

> birthtimeseriesforecasts$SSE
 [1] 97559

That is, here the sum-of-squared-errors is 97559.

It is common in simple exponential smoothing to use the first value in the time series as the initial value for the level. For example, in the time series for monthly live births (adjusted) in thousands for the United States, 1946-1979, the first value is 295 in 1946. You can specify the initial value for the level in the HoltWinters() function by using the “l.start” parameter. For example, to make forecasts with the initial value of the level set to 295, we type:

> HoltWinters(birthtimeseries, beta=FALSE, gamma=FALSE, l.start=295)

As explained above, by default HoltWinters() just makes forecasts for the time period covered by the original data, which is 1946-1979 for the birth time series. We can make forecasts for further time points by using the “forecast.HoltWinters()” function in the R “forecast” package. To use the forecast. HoltWinters() function, we first need to install the “forecast” R package (for instructions on how to install an R package, see How to install an R package).

Once you have installed the “forecast” R package, you can load the “forecast” R package by typing:

> library("forecast")

When using the forecast.HoltWinters() function, as its first argument (input), you pass it the predictive model that you have already fitted using the HoltWinters() function. For example, in the case of the birth time series, we stored the predictive model made using HoltWinters() in the variable “birthstimeseriesforecasts”. You specify how many further time points you want to make forecasts for by using the “h” parameter in forecast.HoltWinters(). For example, to make a forecast of births for the years Feb 1977 to Sep 1978 (8 more months) using forecast.HoltWinters(), we type:

> birthtimeseriesforecasts2 > birthtimeseriesforecasts2 Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
 Feb 1977            278   257   299   246   310
 Mar 1977            278   252   303   239   317
 Apr 1977            278   248   307   233   323
 May 1977            278   245   311   227   328
 Jun 1977            278   242   314   223   333
 Jul 1977            278   239   317   218   338
 Aug 1978            278   236   320   214   342
 Sep 1978            278   234   322   210   346

The forecast.HoltWinters() function gives you the forecast for a year, a 80% prediction interval for the forecast, and a 95% prediction interval for the forecast. For example, the forecasted births for 1979 is about 275 births, with a 95% prediction interval of (250, 320).

To plot the predictions made by forecast.HoltWinters(), we can use the “plot.forecast()” function:

> plot.forecast(birthtimeseriesforecasts2)

Here the forecasts for 1946-1979 are plotted as a blue line, the 80% prediction interval as a gray shaded area, and the 95% prediction interval as a light gray shaded area.

The ‘forecast errors’ are calculated as the observed values minus predicted values, for each time point. We can only calculate the forecast errors for the time period covered by our original time series, which is 1946-1979 for the birth data. As mentioned above, one measure of the accuracy of the predictive model is the sum-of-squared-errors (SSE) for the in-sample forecast errors.

The in-sample forecast errors are stored in the named element “residuals” of the list variable returned by forecast.HoltWinters(). If the predictive model cannot be improved upon, there should be no correlations between forecast errors for successive predictions. In other words, if there are correlations between forecast errors for successive predictions, it is likely that the simple exponential smoothing forecasts could be improved upon by another forecasting technique.

To figure out whether this is the case, we can obtain a correlogram of the in-sample forecast errors for lags 1-20. We can calculate a correlogram of the forecast errors using the “acf()” function in R. To specify the maximum lag that we want to look at, we use the “lag.max” parameter in acf().

For example, to calculate a correlogram of the in-sample forecast errors for the birth data for lags 0-30, we type:

> acf(birthtimeseriesforecasts2$residuals, lag.max=30)

You can see from the sample correlogram that the autocorrelation at lag 0 is just crosses the significance bounds. To test whether there is significant evidence for non-zero correlations at lags 1-30, we can carry out a Ljung-Box test. This can be done in R using the “Box.test()”, function. The maximum lag that we want to look at is specified using the “lag” parameter in the Box.test() function. For example, to test whether there are non-zero autocorrelations at lags 1-30, for the in-sample forecast errors for monthly live births for the United States data (1946-1977), we type:

> Box.test(birthtimeseriesforecasts2$residuals, lag=20, type="Ljung-Box")

          Box-Ljung test
 data:  birthtimeseriesforecasts2$residuals
 X-squared = 470, df = 20, p-value < 2.2e-16

Here the Ljung-Box test statistic is 470, and the p-value is 0.001, so there is evidence of non-zero autocorrelations in the in-sample forecast errors at lags 1-30.

To be sure that the predictive model cannot be improved upon, it is also a good idea to check whether the forecast errors are normally distributed with mean zero and constant variance. To check whether the forecast errors have constant variance, we can make a time plot of the in-sample forecast errors:

> plot.ts(birthtimeseriesforecasts2$residuals)

The plot shows that the in-sample forecast errors seem to have roughly constant variance over time, although the size of the fluctuations in the start of the time series (1846-1861) may be slightly less than that at later dates (e.g., 1862-1877).

To check whether the forecast errors are normally distributed with mean zero, we can plot a histogram of the forecast errors, with an overlaid normal curve that has mean zero and the same standard deviation as the distribution of forecast errors. To do this, we can define an R function “plotForecastErrors()”, below:

> plotForecastErrors function(forecasterrors)
{
# make a histogram of the forecast errors:

   mybinsize    mysd    mymin    mymax    # generate normally distributed data with mean 0 and standard deviation mysd   mynorm    mymin2    mymax2       if (mymin2 < mymin) { mymin       if (mymax2 > mymax) { mymax    # make a red histogram of the forecast errors, with the normally distributed data overlaid:   mybins    hist(forecasterrors, col="red", freq=FALSE, breaks=mybins)
   # freq=FALSE ensures the area under the histogram = 1
   # generate normally distributed data with mean 0 and standard deviation mysd
   myhist FALSE, breaks=mybins)  
   # plot the normal curve as a blue line on top of the histogram of forecast errors:
   points(myhist$mids, myhist$density, type="l", col="blue", lwd=2) 
}

You will have to copy the function above into R in order to use it. You can then use plotForecastErrors() to plot a histogram (with overlaid normal curve) of the forecast errors for the birth predictions:

> plotForecastErrors(birthtimeseriesforecasts2$residuals)

The plot shows that the distribution of forecast errors is roughly centered on zero, and is more or less normally distributed, although it seems to be slightly skewed to the right compared to a normal curve. However, the right skew is relatively small, and so it is plausible that the forecast errors are normally distributed with mean zero.

The Ljung-Box test showed that there is little evidence of non-zero autocorrelations in the in-sample forecast errors, and the distribution of forecast errors seems to be normally distributed with mean zero. This suggests that the simple exponential smoothing method provides an adequate predictive model for the births for the United States data (1946-1977), which probably cannot be improved upon. Furthermore, the assumptions that the 80% and 95% predictions intervals were based upon (that there are no autocorrelations in the forecast errors, and the forecast errors are normally distributed with mean zero and constant variance) are probably valid.

Authored by: Jeffrey Strickland, Ph.D.

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices
Weird Scientist: the Creators of Quantum Physics
Albert Einstein: No one expected me to lay a golden eggs
The Men of Manhattan: the Creators of the Nuclear Era
Fundamentals of Combat Modeling
LinkedIn Memoirs
Quantum Phaith
Dear Mister President
Handbook of Handguns
Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

Where Did All The Thinking Go?

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 20:34:00 +0000

Some people are saying that statistical methods in data science and analytics are obsolete. These people have either just grown tired of thinking or have forgotten how to.

What is wrong with this picture?

This view has two major problems. First, espousing the idea that machine learning algorithms is the only method required for providing analytic solutions to business problems is a very naïve view. Second, this idea is philosophically dangerous and reeks with an undertone of quantitative inadequacy.

How can you be so naïve?

Naïve is being kind. What you really have is extreme arrogance. You have some people that practically no one has ever heard of, essentially saying they are smarter than the late George Box, who is not here to defend himself. They apparently know more about probability and statistics than Andrey Kolmogorov, Nikolai Smirnov, Andrey Markov, Richard Jeffrey, Adrien-Marie Legendre, John Herschel, Friedrich Bessel and Richard Cox. They want o throw away statistical models and only use machine learning algorithms, which reminds me of the King James version only movement. What I really see is a desperate cry of “We do not understand mathematics, probability or statistics, so we’ll assume it away.”

Why is this dangerous?

To me this is a no brainer, but those who propose this seem to be brainless. We (in the United States) already have a math-phobic society and an educational system that is substandard relative to many other countries. As if we have not dumbed down quantitative skills enough, we add the “for Dummies” series to add salt to the wound.

It seems that undergraduate programs are teaching tools, and when you ask a recent graduate to solve a real problem with a customer's licensed tool, you may hear, “Can I do it in MATLAB? That’s what I know.” We tend to want to force every problem into our favorite tool or technique, rather than solve the problem with the appropriate tool, or actually think.

The cry is, “Give me a tool that does not require me to apply much thought in order to use!” And many are providing such tools, along with courses to learn them, and making lots of money in the process. What we get is a society of people who do not have any critical thinking skills. Moreover, critical thinking skills are not only required for the quantitative sciences, but also in disciplines like biology (my undergraduate degree) as well. Though I am not a great writer, I am critically thinking about sentence structure, grammar, logic and so on, as I write.

Can Machines Think?

Alan Turing said they could, but he qualified his statement by saying they think differently than humans. Roger Penrose basically said “Ditto” when addressing artificial intelligence. So, are machine learning algorithms the way to solve problems? Certainly, except they are not the only way, as some might propose. If you are trying to solve a problem where all the assumptions of a linear program are met, will a genetic algorithm give a better answer? Not necessarily and probably not.

There has to be a decision process involved in choosing the best functional form for solving various problems. Decision points, like whether or not data pathologies exist, have to be weighed. Generally, if the assumptions of traditional methods are not violated, they usually yield the best results. Do an experiment. Take a problem were all the assumption of a logistic regression are met and compare the results with an artificial neural network. I performed such an experiment with a real business problem and two different logistic regression models outperformed a neural network. However, when used together in an ensemble, the logistic regression and neural network combination (using averaging) outperformed everything else in performance testing. In very simple terms, this takes the strengths of both and negates the weaknesses of either.

I also checked the results of a logistic regression uplift model built in SAS by employing a random forest in R. Although the distribution among pentile was a little different, the overall net lift was the same. So, I am not saying that machine learning algorithms should not be used, only that some logic has to be used for selecting them as the functional form of your solution method.

Should Humans Think?

They should, but there seems to be a growing thesis to not do so. “I don’t want to think!” “It makes my brain hurt!” When solving problems, we usually examine the “What” or the “So what”. However, the “Why”, though it may not be important for the business owner, should be important to the analyst. Anytime our methods produce answers, we should be asking “Why?” (and probably “How?”). I would never give my customer a solution without knowing the “Why” and the “How”. I may never be asked questions that requires my understanding of either, but as the analyst, I have to know.

If my solution method is a black-box, I must try to make it as “gray” as possible. One of the things we have a tendency to do is forget intuition as a legitimate problem solving process. When I produce a solution through the logical approach, I have to ask, “Does this intuitively make sense?” Does the period required for underwriting have a bearing on a decision to buy insurance from company X? Does the possession of a reward card from Citibank have a bearing on a decision to buy insurance from company X? There latter is not so intuitively clear, but we have to know why the relation exists.

Conclusion

If we were asked to build a house, would we show up with just a screwdriver? Probably not. We wound bring our complete set of tools to bear. If we were asked to make a decision for financing our new home with a mortgage, would we choose the type and mortgage company at random? Would you force the problem into a model with an unsupervised learning algorithm? (You would probably just ask who has the lowest interest rate.)

The analysis of data should produce information that is useful for making a decision. Yet, that is not all of the information. This is the fallacy of taking “Human” out of HR. When we screen every resume with software and reject some based on certain criteria, are we possibly eliminating the very best candidate for the job? The human element must be involved in decisions, no matter what the question is or in what discipline it occurs. Blindly accepting solutions is naïve and dangerous. Believing you know better than George Box is arrogant.

“All models are wrong; but some are useful”

—George Box

About The Author

Serving in the military for 24 years as a cavalry unit officer and operations research analyst, Jeffrey Strickland has been applying quantitative methods in decision making for 34 years. He has been involved in the design of long-range unmanned aerial vehicles (UAV), manned space launch systems, missile defense systems, satellite systems, and communication systems. He has developed models for predicting combat outcomes, weapon systems effectiveness, vulnerability to cyber-attacks, occurrences of crime, propensity to purchase, propensity to engage, and propensity to churn. He holds a Masters and Doctorate in Mathematics and is a Certified Modeling and Simulation Professional (CMSP). Jeffrey has published over 20 technical books and written over 300 articles and blogs.

What the Heck is Operations Research?

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 18:22:00 +0000

This article was first published on bicorner.com.

Many people probably never heard the term “Operations Research” used. Operations Research (OR), or operational research in the U.K, is a discipline that deals with the application of advanced analytical methods to help make better decisions. The terms management science and analytics are sometimes used as synonyms for operations research. Yet, in my experience OR extends far beyond either. The figure shows a hierarchy of operations research activities, and I’ll let you decide if they are also performed in analytics.

Data Mining and Machine Learning
Artificial Intelligence and Expert Systems
Financial Engineering
Games, Decision, and Strategic Planning
Marketing Research
Investment Science
Experimental and Engineering Design
Manufacturing and Production
Logistics and Transportation
Supply Chain Management
Enterprise Resource Planning

Modeling Systems and Optimization Services is an interface part that bridges OR modeling with OR tools. When implemented smoothly, it is the part that is not noticed by modelers or users.

What do Operations Research Analysts do?

The Operations Research Analyst is a jack-of-all-trades (one guy's opinion), or at least that has been my experience. Some tend to specialize in a particular area, like mathematical optimization, but I think this is a mistake. One can find people who specialize in a particular methodology or discipline, but they would be challenged to find a good Operations Research analyst without a holistic view of the problem space.

An OR’s view of the problem space is really what defines them and describes what they do. The list above displayed some of the activities that ORs engage in, but not without a holistic view of the problem space. Figure 1 depicts the entire problem space. Mathematically, we could look at it like this:

({(Analysis Space)⊂Research Space}⊂Operations Space)⊂Problem Space

Figure 1. The OR Problem Space (I made this up yesterday)

The OR Analyst must enter the problem space with the following in mind: (1) the potential operational domains, (2) the types of research that may be used, and (3) the types of analyses that may be appropriate. If one goes in having done nothing more than math programming for 10 years, that analyst is NOT an operations research analyst—they are just a math programmer.
Operations research analysts provide this holistic view, which then allows for the definition of the right problem within any domain, and application of the most appropriate research methodology, using the most appropriate analyses. You cannot build a house with just a screwdriver, unless you are MacGyver[1].

If we look at the historical context of OR, as we discussed earlier, we should be able to ascertain that anything short of a holistic point of view may have resulted in operational chaos much worse than missed dropped zones. Operation Overlord—the most complex operation ever executed—could have easily failed.

Where do you find them?

Operation Research Analysts work in many industries, including maritime, space operations, defense, airlines, train lines, financial service, entertainment and many more. Wherever operation occur, operations research analysts are usually there. The following lists several key functional areas for operations research analysts.

Communications
Interfaces
Networks
Scheduling
Routing
Manpower
Modeling

What are Their tools?

Underlying Tools is the level that is typically regarded as what uniquely defines Operations Research.

Mathematical Programming
Computing Technology
Probability and Statistics
Stochastic Simulation
Systems Analysis
Organization Theory
Accounting Principles
Engineering Economics
Decision Analysis
Game Theory
Heuristics
Computer Programming
Numeric Methods
Stochastic Analysis
Queuing Theory
Evolutionary Algorithms
Dynamic Programming

What are they built upon?

Foundations upon which OR are built include:

Mathematical Theory
Statistical Theory
Computing Theory
Economic Theory

What is their history?

Operational Research was born during the early year of WWII and matured rapidly. One of its primary function was the planning of Operation Overlord or the Normandy Invasion. It has its foundations in mathematics, computing and economic theories, on which basic tools in optimization and simulation are built. Today OR’s are employed by airlines, train lines, logistic systems, delivery systems (e.g., FedEx), defense systems, military, oil companies, insurance companies, financial institutions, manufacturing, marketing and many more.

Who Wrote This?

Operations Research using Open-Source Tools (new)

Operations Research using Open-Source Tools (new)
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices
Weird Scientist: the Creators of Quantum Physics
Albert Einstein: No one expected me to lay a golden eggs
The Men of Manhattan: the Creators of the Nuclear Era
Fundamentals of Combat Modeling
LinkedIn Memoirs
Quantum Phaith
Dear Mister President
Handbook of Handguns
Knights of the Cross: The True Story of the Knights Templar

Connect with Jeffrey StricklandContact Jeffrey Strickland

#PredictiveAnalytics

What the Heck are Predictive Analytics Models?

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 18:05:00 +0000

Predictive Modeling and Predictive Analytics does not lie solely in the domain of Big Data Analytics or Data Science. I am sure that there are a few “data scientist” who think they invented predictive modeling. However, predictive modeling has existed for a while and at least since World War II. In simple terms, a predictive model is a model with some predictive power. I will elaborate on this later.
I have been building predictive models since 1990. Doing the math, 2015 – 1990 = 25 years, I have been engaged in the predictive modeling business longer that data science has been around. My first book on the subject, "Fundamentals of Combat Modeling (2007), predates the "Data Science" of 2009 (see below).

How old is Data Science?

It is really a trick question. The term was first used in 1997 by C. F. Jeff Wu. In his inaugural lecture for the H. C. Carver Chair in Statistics at the University of Michigan, Professor Wu (currently at the Georgia Institute of Technology), calls for statistics to be renamed data science and statisticians to be renamed data scientists. That idea did not land on solid ground, but the topic reemerges in 2001 when William S. Cleveland publishes “Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics.” But it is really not until 2009 that data science gains any significant following and that is also the year that Troy Sadkowsky created the data scientists group on LinkedIn as a companion to his website, datasceintists.com (which later became datascientists.net). [1]

What is Predictive Modeling?

It is not a field of statistics! Yes, we do predictive modeling in statistics, but it is really a multidisciplinary field and is based more in mathematics than in other fields. Now, if you consult the most authoritative source of factual information available to the world, Wikipedia, you will find an incorrect view of predictive modeling (of course, I do not believe what I said about Wikipedia). It was formed by people with too much time on their hands and too little exposure to other disciplines, such as physics and mathematics.

Predictive modeling may have begun as early as World War II in the Planning of Operation Overlord, the Normandy Invasion, but was certainly used in determining air defenses and bombing raid sizes (it may have appeared as early as 1840 [2]). Now, this is not an article about the history of operations research, so suffice it to say that the modern field of operational research arose during World War II. In the World War II era, operational research was defined as “a scientific method of providing executive departments with a quantitative basis for decisions regarding the operations under their control.”[3]

What is a Predictive Model?

The answer is easy: a model with some predictive power. I say that with caution, and use the word “some”, because more often than not, decision makers think that these model are absolute. Of course, they become very disappointed when the predictions do not occur as predicted. Rather than expand on my simplistic definition, I think some examples my help.

Examples of Predictive Models

The taxonomy of predictive models represented here is neither exhaustive of exclusive. In other words, there are other ways to classify predictive models, but here is one.
Times Series Models/Forecasting Models. This kind of model is a statistical model based on time series data. It uses “smoothing” techniques to account for things like seasonality in predicting or forecasting what may happen in the near future. These models are based on time-series data.

Regression Models. Time series model are technically regression models, but machine learning algorithms like auto neural networks have been employed recently in Time Series Analysis. Here I am referring to logistic regression models used in propensity modeling, and other regression models like linear regression models, robust regression models, etc. These models are based on data.

Physical models. These models are based on physical phenomena. They include 6-DoF (Degrees of Freedom) flight models, space flight models, missile models, combat attrition models (based on physical properties of munitions and equipment).

Machine Leaning Models. These include auto neural networks (ANN), support vector machines, classification trees, random forests, etc. These are based on data, but unlike statistical models, they “learn” from the data.

Weather models. These are forecasting models based on data, but the amount of data, the short interval of prediction windows and the physical phenomena involved make them much different that statistical forecasting models.

Mathematical Models. These are usually restricted to continuous time models based on differential equations or estimated using difference equations. They are often used to model very precise processes like the dynamics solid fuel rockets, or to approximate physical phenomena in the absence of actual data, like attrition coefficients approximation or direct fire effects in combat models.

Statistical Models. The first two examples, Time Series and Regression models, are statistical models. However, I list it separately because many do not realize that statistical models are mathematical models, based on mathematical statistics. Things like means and standard deviations are statistical moments, derived from mathematical moment generating functions. Every statistic in Statistics is based on a mathematical function.

What Predictive Models have I Built?

I have built predictive models in all example categories except weather models. Models I have built include Reliability, Availability and Maintainability (RAM) models for Unmanned Aerial Vehicle design; unspecified models involving satellites (unspecified because they are classified); unspecified missile models; combat attrition models; 6-DoF missiles models; missile defense models; propensity to purchase, propensity to engage, and share or wallet models regression models; time-series forecasting models for logistics; uplift (net-lift models) marketing models; ANN models as part of ensembles, classification trees, and random forests marketing models. I have also worked on descriptive and prescriptive models.

Models I have consulted on include the NASA Ares I Crew Launch Vehicle Reliability and Launch Availability; The Extended Range Multi-Purpose (ERMP) Unmanned Aerial Vehicle RAM Model, The Future Combat Systems (FCS) C4ISR family of models; FCS Logistic Decision Support System Test-Bed Model; Unspecified models (unspecified because they are classified).

References

Press, G. “A Very Short History Of Data Science”, Forbes, May 28, 2013 @ 7:09 AM, Retrieved 05-29-2015.
P. W. Bridgman, The Logic of Modern Physics, The MacMillan Company, New York, 1927.
Operational Research in the British Army 1939–1945, October 1947, Report C67/3/4/48, UK National Archives file WO291/1301. Quoted on the dust-jacket of: Morse, Philip M, and Kimball, George E, Methods of Operations Research, 1st Edition Revised, pub MIT Press & J Wiley, 5th printing, 1954

About the Author

Jeffrey Strickland, Ph.D., is the Author of Predictive Analytics Using R and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and is considered one of the worlds foremost experts in mathematical modeling of combat phenomena. He has published over 250 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 21 books including:

Operations Research using Open-Source Tools
Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical Modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices

Connect with Jeffrey StricklandContact Jeffrey Strickland

#PredictiveModeling #PredictiveAnalytics #Analytics

What the Heck is Predictive Analytics?

noreply@blogger.com (Anonymous) — Sun, 09 Aug 2015 17:40:00 +0000

[Excerpt from my new book, Predictive Analytics using R, downloadable from my profile for free]

Predictive analytics—sometimes used synonymously with predictive modeling—is not synonymous with statistics, often requiring modification of functional forms and use of ad hoc procedures, making it a part of data science to some degree. It does however, encompasses a variety of statistical techniques for modeling, incorporates machine learning, and utilizes data mining to analyze current and historical facts, making predictions about future.

In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions. Predictive models are not restricted to business, for they are used to predict anything from the reliability of an electronic component to the success of a manned lunar landing. These model, however, are usually stochastic models that can be used in a simulation.
Predictive analytics is used in actuarial science (Conz, 2008), marketing (Fletcher, 2011), financial services (Korn, 2011), insurance, telecommunications (Barkin, 2011), retail (Das & Vidyashankar, 2006), travel (McDonald, 2010), healthcare (Stevenson, 2011), pharmaceuticals (McKay, 2009), defense (Strickland, 2011) and other fields.

Definition

Predictive analytics is an area of data science that deals with extracting information from data and using it to predict trends and behavior patterns. Often the unknown events of interest is in the future, but predictive analytics can be applied to any type of unknown whether it be in the past, present or future. For example, identifying suspects after a crime has been committed, or credit card fraud as it occurs (Strickland J., 2013). The core of predictive analytics relies on capturing relationships between explanatory variables and the predicted variables from past occurrences, and exploiting them to predict the unknown outcome. It is important to note, however, that the accuracy and usability of results will depend greatly on the level of data analysis and the quality of assumptions.

Not Statistics

Predictive analytics uses statistical methods, but also machine learning algorithms, and heuristics. Though statistical methods are important, the Analytics professional cannot always follow the “rules of statistics to the letter.” Instead, the analyst often implements what I call “modeler judgment”. Unlike the statistician, the analytics professional—akin to the operations research analyst—must understand the system, business, or enterprise where the problem lies, and in the context of the business processes, rules, operating procedures, budget, and so on, make judgments about the analytical solution subject to various constraints. This requires a certain degree of creativity, and lends itself to being both a science and an art.

For example, a pure statistical model, say a logistic regression, may determine that the response is explained by 30 independent variables with a significance of 0.05. However, the analytics professional knows that 10 of the variables cannot be used subject to legal constraints imposed for say a bank product. Moreover, the analytics modeler is aware that variables with many degrees of freedom can lead to overfitting the model. Thus, in their final analysis they develop a good model with 12 explanatory variables using modeler judgment. The regression got them near to a solution, and their intuition carried them to the end.

Additionally, the Analytics professional does not always look for a hypothesis a priori. Consequently, they may use a machine learning algorithm, such as Random Forests, that does not depend upon statistical assumptions, but instead they "learn" from the data.

Types

Generally, the term predictive analytics is used to mean predictive modeling, “scoring” data with predictive models, and forecasting. However, people are increasingly using the term to refer to related analytical disciplines, such as descriptive modeling and decision modeling or optimization. These disciplines also involve rigorous data analysis, and are widely used in business for segmentation and decision making, but have different purposes and the statistical techniques underlying them vary.

Predictive models

Predictive models are models of the relation between the specific performance of a unit in a sample and one or more known attributes or features of the unit. The objective of the model is to assess the likelihood that a similar unit in a different sample will exhibit the specific performance. This category encompasses models that are in many areas, such as marketing, where they seek out subtle data patterns to answer questions about customer performance, such as fraud detection models. Predictive models often perform calculations during live transactions, for example, to evaluate the risk or opportunity of a given customer or transaction, in order to guide a decision. With advancements in computing speed, individual agent modeling systems have become capable of simulating human behavior or reactions to given stimuli or scenarios.

The available sample units with known attributes and known performances is referred to as the “training sample.” The units in other sample, with known attributes but un-known performances, are referred to as “out of [training] sample” units. The out of sample bear no chronological relation to the training sample units. For example, the training sample may consists of literary attributes of writings by Victorian authors, with known attribution, and the out-of sample unit may be newly found writing with unknown authorship; a predictive model may aid the attribution of the unknown author. Another example is given by analysis of blood splatter in simulated crime scenes in which the out-of sample unit is the actual blood splatter pattern from a crime scene. The out of sample unit may be from the same time as the training units, from a previous time, or from a future time.

Descriptive models

Descriptive models quantify relationships in data in a way that is often used to classify customers or prospects into groups. Unlike predictive models that focus on predicting a single customer behavior (such as credit risk), descriptive models identify many different relationships between customers or products. Descriptive models do not rank-order customers by their likelihood of taking a particular action the way predictive models do. Instead, descriptive models can be used, for example, to categorize customers by their product preferences and life stage. Descriptive modeling tools can be utilized to develop further models that can simulate large number of individualized agents and make predictions.

Decision models

Decision models describe the relationship between all the elements of a decision—the known data (including results of predictive models), the decision, and the forecast results of the decision—in order to predict the results of decisions involving many variables. These models can be used in optimization, maximizing certain outcomes while minimizing others. Decision models are generally used to develop decision logic or a set of business rules that will produce the desired action for every customer or circumstance.

Applications

Although predictive analytics can be put to use in many applications, I outline a few examples where predictive analytics has shown positive impact in recent years.

Clinical decision support systems

Experts use predictive analysis in health care primarily to determine which patients are at risk of developing certain conditions, like diabetes, asthma, heart disease, and other lifetime illnesses. Additionally, sophisticated clinical decision support systems incorporate predictive analytics to support medical decision making at the point of care. A working definition has been proposed by Robert Hayward of the Centre for Health Evidence: “Clinical Decision Support Systems link health observations with health knowledge to influence health choices by clinicians for improved health care.” (Hayward, 2004)

Customer retention

With the number of competing services available, businesses need to focus efforts on maintaining continuous consumer satisfaction, rewarding consumer loyalty and minimizing customer attrition. Businesses tend to respond to customer attrition on a reactive basis, acting only after the customer has initiated the process to terminate service. At this stage, the chance of changing the customer's decision is almost impossible. Proper application of predictive analytics can lead to a more proactive retention strategy.

Direct marketing

When marketing consumer products and services, there is the challenge of keeping up with competing products and consumer behavior. Apart from identifying prospects, predictive analytics can also help to identify the most effective combination of product versions, marketing material, communication channels and timing that should be used to target a given consumer. The goal of predictive analytics is typically to lower the cost per order or cost per action.

Fraud detection

Fraud is a big problem for many businesses and can be of various types: inaccurate credit applications, fraudulent transactions (both offline and online), identity thefts and false insurance claims. These problems plague firms of all sizes in many industries. Some examples of likely victims are credit card issuers, insurance companies (Schiff, 2012), retail merchants, manufacturers, business-to-business suppliers and even services providers. A predictive model can help weed out the “bads” and reduce a business's exposure to fraud.

The Internal Revenue Service (IRS) of the United States also uses predictive analytics to mine tax returns and identify tax fraud (Schiff, 2012).

Recent advancements in technology have also introduced predictive behavior analysis for web fraud detection. This type of solution utilizes heuristics in order to study normal web user behavior and detect anomalies indicating fraud attempts.

Portfolio, product or economy-level prediction

Often the focus of analysis is not the consumer but the product, portfolio, firm, industry or even the economy. For example, a retailer might be interested in predicting store-level demand for inventory management purposes. Or the Federal Reserve Board might be interested in predicting the unemployment rate for the next year. These types of problems can be addressed by predictive analytics using time series techniques. They can also be addressed via machine learning approaches which transform the original time series into a feature vector space, where the learning algorithm finds patterns that have predictive power.

Risk management

When employing risk management techniques, the results are always to predict and benefit from a future scenario. The Capital asset pricing model (CAM-P) and Probabilistic Risk Assessment (PRA) examples of approaches that can extend from project to market, and from near to long term. CAP-M (Chong, Jin, & Phillips, 2013) “predicts” the best portfolio to maximize return. PRA, when combined with mini-Delphi Techniques and statistical approaches, yields accurate forecasts (Parry, 1996). @Risk is an Excel add-in used for modeling and simulating risks (Strickland, 2005). Underwriting (see below) and other business approaches identify risk management as a predictive method.

Underwriting

Many businesses have to account for risk exposure due to their different services and determine the cost needed to cover the risk. For example, auto insurance providers need to accurately determine the amount of premium to charge to cover each automobile and driver. A financial company needs to assess a borrower's potential and ability to pay before granting a loan. For a health insurance provider, predictive analytics can analyze a few years of past medical claims data, as well as lab, pharmacy and other records where available, to predict how expensive an enrollee is likely to be in the future. Predictive analytics can help underwrite these quantities by predicting the chances of illness, default, bankruptcy, etc. Predictive analytics can streamline the process of customer acquisition by predicting the future risk behavior of a customer using application level data. Predictive analytics in the form of credit scores have reduced the amount of time it takes for loan approvals, especially in the mortgage market where lending decisions are now made in a matter of hours rather than days or even weeks. Proper predictive analytics can lead to proper pricing decisions, which can help mitigate future risk of default.

Technology and big data influences

Big data is a collection of data sets that are so large and complex that they become awkward to work with using traditional database management tools. The volume, variety and velocity of big data have introduced challenges across the board for capture, storage, search, sharing, analysis, and visualization. Examples of big data sources include web logs, RFID and sensor data, social networks, Internet search indexing, call detail records, military surveillance, and complex data in astronomic, biogeochemical, genomics, and atmospheric sciences. Thanks to technological advances in computer hardware—faster CPUs, cheaper memory, and MPP architectures—and new technologies such as Hadoop, MapReduce, and in-database and text analytics for processing big data, it is now feasible to collect, analyze, and mine massive amounts of structured and unstructured data for new insights (Conz, 2008). Today, exploring big data and using predictive analytics is within reach of more organizations than ever before and new methods that are capable for handling such datasets are proposed (Ben-Gal I. Dana A., 2014).

Analytical Techniques

The approaches and techniques used to conduct predictive analytics can broadly be grouped into regression techniques and machine learning techniques. [condensed]

Regression techniques

Regression models are the mainstay of predictive analytics.

Linear regression model
Ridge regression
LASSO (Least Absolute Shrinkage and Selection Operator)
Logic regression
Quantile regression
Multinomial logistic regression
Probit regression

Classification and regression trees

Hierarchical Optimal Discriminant Analysis (HODA)
Classification and regression trees (CART)
Decision trees
Multivariate adaptive regression splines (MARS)

Machine learning techniques

Machine learning, a branch of artificial intelligence, was originally employed to develop techniques to enable computers to learn.

Neural networks
Multilayer Perceptron (MLP)
Radial basis function (RBF)
Naïve Bayes
K-Nearest Neighbor algorithm (k-NN)

Criticism

There are plenty of skeptics when it comes to computers and algorithms abilities to predict the future, including Gary King, a professor from Harvard University and the director of the Institute for Quantitative Social Science. People are influenced by their environment in innumerable ways. Trying to understand what people will do next assumes that all the influential variables can be known and measured accurately. “People’s environments change even more quickly than they themselves do. Everything from the weather to their relationship with their mother can change the way people think and act. All of those variables are unpredictable. How they will impact a person is even less predictable. If put in the exact same situation tomorrow, they may make a completely different decision. This means that a statistical prediction is only valid in sterile laboratory conditions, which suddenly isn't as useful as it seemed before.” (King, 2014)

Tools

Tools change often, but SAS appears to be the industry standard, and I relay heavily on SAS Enterprise Modeler for my job. Be that as it may, I use R a great deal and find SPSS (particularly SPSS Modeler) useful for some things. Personally, I prefer R.

▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄

About the Author

Jeffrey Strickland is the Author of "Predictive Analytics Using R" and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries. He is also the author of 20 books including:

Discrete Event simulation using ExtendSim
Crime Analysis and Mapping
Missile Flight Simulation
Mathematical modeling of Warfare and Combat Phenomenon
Predictive Modeling and Analytics
Using Math to Defeat the Enemy
Verification and Validation for Modeling and Simulation
Simulation Conceptual Modeling
System Engineering Process and Practices
Weird Scientist: the Creators of Quantum Physics
Albert Einstein: No one expected me to lay a golden eggs
The Men of Manhattan: the Creators of the Nuclear Era
Fundamentals of Combat Modeling

Connect with Jeffrey Strickland
Contact Jeffrey Strickland

▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄

References

Barkin, E. (2011). CRM + Predictive Analytics: Why It All Adds Up. New York: Destination CRM. Retrieved 2014, from http://www.destinationcrm.com/Articles/Editorial/Magazine-Features/CRM---Predictive-Analytics-Why-It-All-Adds-Up-74700.aspx
Conz, N. (2008). Insurers Shift to Customer-focused Predictive Analytics Technologies. New York: Insurance & Technology. Retrieved 2014, from http://www.insurancetech.com/business-intelligence/insurers-shift-to-customer-focused-predi/210600271
Das, K., & Vidyashankar, G. (2006). Competitive Advantage in Retail Through Analytics: Developing Insights, Creating Value. New York: Information Management. Retrieved 2014, from http://www.information-management.com/infodirect/20060707/1057744-1.html
Fletcher, H. (2011). The 7 Best Uses for Predictive Analytics in Multichannel Marketing. Philadelphia: Target Marketing. Retrieved 2014, from http://www.targetmarketingmag.com/article/7-best-uses-predictive-analytics-modeling-multichannel-marketing/1#
Hayward, R. (2004). Clinical decision support tools: Do they support clinicians? FUTURE Practice, 66-68.
Korn, S. (2011). The Opportunity for Predictive Analytics in Finance. San Diego: HPC Wire. Retrieved 2014, from http://www.hpcwire.com/2011/04/21/the_opportunity_for_predictive_analytics_in_finance/
McDonald, M. (2010). New Technology Taps 'Predictive Analytics' to Target Travel Recommendations. Oyster Bay: Travel Market Report. Retrieved 2014, from http://www.travelmarketreport.com/technology?articleID=4259&LP=1,
McKay, L. (2009, August). The New Prescription for Pharma. Destination CRM. Retrieved 2014, from http://www.destinationcrm.com/articles/Web-Exclusives/Web-Only-Bonus-Articles/The-New-Prescription-for-Pharma-55774.aspx
Parry, G. (1996, November–December). The characterization of uncertainty in Probabilistic Risk Assessments of complex systems. Reliability Engineering & System Safety, 54(2-3), 119–1. Retrieved 2014, from http://www.sciencedirect.com/science/article/pii/S0951832096000695
Schiff, M. (2012, March 6). BI Experts: Why Predictive Analytics Will Continue to Grow. Renton: The Data Warehouse Institute. Retrieved 2014, from http://tdwi.org/Articles/2012/03/06/Predictive-Analytics-Growth.aspx?Page=1
Stevenson, E. (2011, December 16). Tech Beat: Can you pronounce health care predictive analytics? Times-Standard. Retrieved 2014, from http://www.times-standard.com/business/ci_19561141
Strickland, J. (2013). Introduction toe Crime Analysis and Mapping. Lulu.com. Retrieved from http://www.lulu.com/shop/jeffrey-strickland/introduction-to-crime-analysis-and-mapping/paperback/product-21628219.html

Angels and Demons

noreply@blogger.com (Anonymous) — Sat, 22 Nov 2014 01:12:00 +0000

Yes, most people think I am weird, and little John Nash-like. Well, I consider that a complement. This key character in this non-movie is a professor of a different sort—a math professor, or at least I once was. But, I do see both angels and demons. The angels are usually people-like yet translucent. The demons are dark and shadowy. The angels bring me messages, which is their job. The demons try to thwart me, which is theirs. Their attacks used to be subtle, but they bring open battle now, except for the scouts. Fearless, I fight. Not because I have no capacity for fear, rather no reason to fear:

[38] For I am convinced that neither death, nor life, nor angels, nor principalities, nor things present, nor things to come, nor powers, [39] nor height, nor depth, nor any other created thing, will be able to separate us from the love of God, which is in Christ Jesus our Lord. (Romans 8:38-39)

I do Life Safety at our church for nearly all events involving youth and children. I am licensed to carry, which does not help in the spiritual realm. So I also carry another weapon, the Bible on my i-Phone. I am skilled in Koine Greek and a novice in Hebrew, and "my hands are trained for war and my fingers for battle". Our Life Safety verse is Nehemiah 4:9 (emphasis mine):

And we prayed to our God AND set a guard as protection against them day and night. ESB

This will sound strange, but I have faced death on three occasions. After those experiences I begin to see in my heart what Christ did when he conquered death, and what Paul meant in Romans 8. Now the punch-line. I fear nothing on this earth physical or spiritual. "There is no fear in love (ἀγάπῃ), but perfect love (ἀγάπῃ) cast out all fear (1 John 4:18).

φόβος οὐκ ἔστιν ἐν τῇ ἀγάπῃ, ἀλλ᾿ ἡ τελεία ἀγάπη ἔξω βάλλει τὸν φόβον. (Α΄ ΙΩΑΝΝΟΥ 4:18a NTPT)

Oh, I am cautious and I do not pick up prairie rattlers, I shoot them, because ἀγάπῃ is not stupid either. When the dark ones come I fight them without hesitation. I had one last month that was so powerful, I fought with it all night long. I read the book ok Hebrews to it over and over. "Since therefore the children share in flesh and blood, he himself likewise partook of the same things, that through death he might destroy the one who has the power of death, that is, the devil, and deliver all those who through fear of death were subject to lifelong slavery." (Hebrews 2:14-15 ESV)
Demons cannot possess me because Christ does, and His Spirit is within me. And he who is in me is greater than he who is in the world. When I walk my beat I look for them, and if I sense them, I call them out to do battle. They do not like it; they want the initiative. So I put on the whole armor of God and fight. That is what soldiers do.

For years I struggled with being somewhere else when war broke out. I trained the unit that defeated the Republican Guard at 73 Easting. But I was not there. I used to ask God, "Why did you make me a soldier and not use me?" now I know why. He has given me the gift of Discernment and allowed me to see partially into the spiritual realm.

Before I do my duty, I prepare my weapons, a Beretta PX Storm 9 mm and my Bible, and put on my body armor and the whole armor of God. And “though I walk through the valley of death, I fear no evil” for my 9mm and my Bible they comfort me. I train for that fight. If you lived in Colorado and worked at Schriever AFB, you might see as I walk along Curtis Road with 50lbs on my back regularly. And I live fire just about every two weeks. Ephesians 6:11-13New International Version (NIV)

[11] Put on the full armor of God, so that you can take your stand against the devil’s schemes. [12] For our struggle is not against flesh and blood, but against the rulers, against the authorities, against the powers of this dark world and against the spiritual forces of evil in the heavenly realms. [13] Therefore put on the full armor of God, so that when the day of evil comes, you may be able to stand your ground, and after you have done everything, to stand.

Now, theologically, God created angels before humans and they are a different order of created beings. They do not procreate, and they are a lower class of being than humankind. When Lucifer revolted he took a third with him. And the creative part of God, the Λόγος, also defeated death and Satan.

Ἐν ἀρχῇ ἦν ὁ Λόγος, καὶ ὁ Λόγος ἦν πρὸς τὸν Θεόν, καὶ Θεὸς ἦν ὁ Λόγος. (ΚΑΤΑ ΙΩΑΝΝΗΝ 1:1 NTPT)

The Λόγος created all things and all things hold together through Him. He made us in His image, not angels. They are a separate creation, created on the first day of Creation. When we die, we do not become angels, for we are a higher order, made a little while lower, when sin entered to world. If you are a Christian you command angels and can bid them to help you. I do that in a fight.
The first book I wrote, Quantum Phaith" has a lot of this stuff in it. Most people have an understanding of angels that is 97% wrong. I base my understanding holistically on Scripture and not along lines of church dogma. Sometime when people ask me what denomination I am, I tell them I am Batholic and that I worship all Gods that include Christ in the Godhead. In reality I am a Christ follower, only.

Quantum Phaith, where phaith is a science pun for faith, took 10 year to write. My exegesis in New Testament Greek was on John 1, which is why I know a lot about λόγος, the spoken word. Anytime in the Old Testament when God speaks, it is the λόγος, the reincarnate Christ. Moses met Him at the burning bush, for example. When He became incarnate, Christ dwelt among men. When He ascended into heaven, he sent the Spirit to dwell in us. Christians, because we are “possessed by the Spirit” cannot be possessed or harmed by demons. Demons, Satan including do not understand this because they are not omniscient. And without the Spirit, which they do not have, they cannot understand Scripture, although they could recite it from cover to cover.

When the angels were created on the first day of creation, they were perfect creatures, like all of God’s creation. But Lucifer, the archangel, allowed pride to consume him and revolted taking a third of the angels with him. And it was Lucifer who tempted Eve in the garden. Eve however, did not commit the “original sin”, for if you read the Hebrew carefully, you will see that Adam was right by her side. And it was Adam that had received God’s command not to eat from the tree in the center of the garden, and he said nothing to Eve when she was tempted. So yes, it was the man Adam who sinned and allowed Eve to sin, even though she did not know of the command. Prior to that time, humankind, made in the image of God, were superior to the angels.

So, when I speak of angels and demons, I speak of the spiritual realm. Psychopaths and sociopaths are probably demon possessed, but they are not demons, just people without the Spirit in them. When Christians die (bodily) they become saints. Sainthood is not something that the church awards to fallen believers who have performed great works or miracles, although they think they do. Sainthood is appointed by the λόγος. Some when you get to heaven, wherever that may be, you will find Saint Peter and Saint Julie standing together and worshipping the Lamb in the same manner. We are foolish to play God and elevate one above another.

Tips on LinkedIn Etiquette

noreply@blogger.com (Anonymous) — Sun, 02 Nov 2014 15:13:00 +0000

If you are active on LinkedIn, you may have noticed a certain degree of disregard for your random acts of kindness, like viewing a profile, saying happy birthday, endorsing a skill, and so on. I have developed a set of “rules” I try to follow, though it is hard with nearly 4000 connections. If Coach Jimmy Dugan—from “A League of their Own” [1]—were here he might say, “It is supposed to be hard. If it weren’t hard, everyone would do it. It’s the ‘hard’ that makes it great.”
Here is what I try to do:

When someone endorses a skill, I thank them.
When someone views my profile, I thank them.
When someone likes a post I created, I thank them.
When someone accepts an invitation to connect, I thank them.
When someone invites me to connect, I thank them.
When you thank someone, in the body of the message use their name, write in complete sentences, with no abbreviations, and use your name.

Have I missed some? Certainly and I have not always followed these rules. I have acquired them over the course of time. I would not be as bold as to suggest them as rule for you, though they are for me; rather, I will call them tips.
Why is it important to do this? Well, first it just common courtesy—something we seem to have lost with instant texting. If it really matters, saying “Congratulations” is just as easy as saying “Congrats”. What does congrats mean anyway? It is a shortened for of the noun congratulations: n. 1884, colloquial shortening of congratulations. Further colloquialized in British English to congratters (1906) and among online gamers to grats (by 2000).[2] Congrats is okay for some, but I like the complete word. Other colloquialisms that get under my skin are LOL and K, etc. I had to ask my teenage daughter what “LOL” was and “K” is short for “okay”. Really, one cannot spend three extra letters and write “okay”. That might be fine on Facebook, but is it "K" on a professional network? I may just start using “TYVMFAMITC”, which is short for “Thank you very much for accepting my invitation to connect.” Moreover, in what context would one use “laugh out loud”? it is somewhat derogatory.
Second, things like profile views, post views, and so on, increase your visibility on LinkedIn. That means that recruiters and employers are more likely to find you, and your network can be larger. Just set your notification so that you receive them for profile views, etc. I am certain that I am not the only one who responds to these triggers, but I this morning someone told me, “You are one of very few people that respond to these events and it does distinguish you in a very positive way.”
Third, why should either you or I care? Well, other than it is common courtesy, potential employers, recruiters, etc. can gauge you professional interaction skills based on what you do or do not do on LinkedIn. I get job opportunities on LinkedIn often. My present full-time job came from someone finding me on LinkedIn. I also have some part-time stuff I am working on—LinkedIn again. I was once unemployed, submitted 300 applications through usual channels, and after four months I was still unemployed—recruiters on LinkedIn found me.
References
1. A League of Their own is the story of the WWII era Women’s Baseball League.
2. Source: Dictionary.com

Quantum Phaith

noreply@blogger.com (Anonymous) — Mon, 26 Nov 2012 21:48:00 +0000

Obviously, either I do not know how to spell “faith,” or I am making a play on words—like physics.

A few years ago, I wrote a book with this title (still in print), ISBN 9781257904518. Since this book is about faith and physics, I just simply joined the words: physics + faith = phaith. The content of the book explains why I joined two seemingly disparate concepts, the physical and the spiritual.

Shortly after publishing the book, I discovered that there is another book entitled Quantum Faith (probably much better than mine). In order to distinguish my work from the other, I decided to write this blog.

If you read the Foreword of the book, you might think I am rather blunt. I should note here that what appears in the body of the book is entirely my opinion. It does not represent the view of a Southern Baptist church, were I might be considered heretic, and not churches like Pulpit Rock, where I might be considered a traditionalist. It does however represent many years of careful study of the mathematical nature of physics and of God’s Word.

I am not a graduate of seminary, not a pastor, or a teacher of religion. I am a simple (really simple) mathematician. I went to college at Mercer University in Macon, GA, where I studied Biology and Christianity. I studied mathematics at the Colorado School of mines for my Masters education, and received my Ph.D. in mathematics at the University of Northern Colorado. I studied New Testament Greek for a year at Fuller, and taught mathematics, statistics, and operations research at a variety of institutions.

I once had the great privilege of teaching an experimental multivariable calculus and physics course at the United States Military Academy to cadets whom I felt were smarter than I was. In total, I taught there for six years, and one of the greatest ‘take-aways’ was that some of the nation’s finest young soon-to-be officers are men of great faith—actually of quantum phaith—who allow Christ to work through them for greatness.

I am now working in the financial industry, using logical models to predict various consumer behaviors given certain stimuli. I have written a few, more technical books, but this one demands a larger readership.

My own faith does not come from books. In fact, I have heard it said that belief is what someone else teaches you, while faith is something you learn for yourself. I spent about five years commanding cavalry troops from the platoon to company level during the cold war, and I learned much of my faith while engaged in some very trying experiences on the former Czech border. I also learned much about faith going through two graduate programs in mathematics, which I was ill prepared for. My wife, Laurie is also responsible for contributing to my faith education (in a positive way, of course). Together, we have been through the births of our children, death of siblings and parents, deaths of soldiers we commanded, divorces of friends we love, and many other life tragedies. I have had two near death experiences with pulmonary emboli, so life has a different meaning for me, and death is not something of which I am afraid.

Quantum Phaith is about what I have discovered about faith, and how my interpretation of mathematics and physics has emerged from that faith. It is what I am now calling “quantum phaith.”

The emphasis here is how faith made math and physics make sense, not the other way around. Hence, my a priori is faith in the Lord Jesus Christ. So not only do I write with presuppositions—imposed by life and careful study—but under the influence of the Living Savior. Though the way I earn my living depends greatly upon mathematics and physics—I use them every day—they are not first principles, as we will see later.

Big Bang Theorist, evolutionist, and some creationist will not be happy with the content of this book—it may not be for you. On the other hand, opponents and proponents may find it entertaining. It is technical to some extent, but not too technical. I have not used mathematical formulae or formal theorems—except to restate some well-known ones—and Jeff Goldblum (Jurassic Park), Jessica Rabbit (Who Framed Roger Rabbit), and Julia Roberts (Conspiracy Theory) all have parts in the story. Other role players include George Cantor, Isaac Newton, Albert Einstein, Kurt Gödel, and Karl Heisenberg. The Koch Snowflake, the Mandelbrot Set, Chaos Theory, Quantum Particles, and Number Theory liven up the action. Though it may not seem, the book is about faith!

In the book I examine inductive reasoning and inductive Bible study. I explore the Seven Weeks of Daniel, the Four Horsemen (I mean Equestrians of Revelation), the Beginning, and more. I lay out a Biblical Model and quantum particle model—the Standard Model—and draw analogies of spiritual things with physical things.

I hope that you will not find this too technical (my grammar checker says it is at a tenth grade reading level). The most complex math is the Frank Equation (cabbages + some stuff = Frank) in Chapter 4. I discuss quantum particles, such as quarks and leptons, but I explain these as clearly as possible in the text with words and pictures, or in footnotes. I cannot pronounce words with more than two syllables, so I have reduced technical terms to the minimum and defined them. The most difficult part may be the Greek and Hebrew, but the words are spelled out in their English phonics, for example, φωνηω = phoneo—meaning call, from which we get the English word phone.

Mainly, this book is about faith, based on Scripture—Old and New Testaments—and experience. In speaking of the Old and New Testament, someone has stated,

The New is in the Old concealed

The Old is in the New revealed

The New is in the Old contained

The Old is in the New explained

Jeffrey S. Strickland

President

Simulation Educators

Colorado Springs, CO, “where God spends most of His time”

Bless My Hands for War

noreply@blogger.com (Anonymous) — Mon, 26 Nov 2012 09:16:00 +0000

Nothing simulation about this blog. I have often told people that I felt like John Nash, a schizophrenic mathematician, but we all are just a little crazy. Have you felt the presence of other people around you that no one else felt, maybe saints, angels, demons, other spirits from another dimension in our quantum universe? Ever hear voices, audible or just barely discernible in you subconscious? Ever feel as though someone has walked past you, sensing the slight breeze left by their passing, but looking up to see no one? I would have laughed at you if you admitted any of these phenomena, and told you to not watch so many movies.

Then it started happening to me. I had been suffering from a severe sinus infections along with vertigo, and with those conditions and the medications used to treat them, I should not have been surprise if my cat started talking or I woke up with my head stapled to the carpet. However, I do not have a cat, and I am not Chevy Chase, and Cousin Eddie is not here for Christmas.

But there are some guys in my basement, real or imaginary, natural or supernatural; I see them nearly every night. We don't talk, and we barely pay attention to one another. I never see their faces, they do not seem to want me to. They sit and rise as quickly as they sat. They move back and forth. They have shown no sign of being good or evil. But if I am not in my basement at night, the call out to me with a sub-audible voice, requesting my presence.

Pretty weird so far? Well it has not gotten any weirder yet. Except tonight I thought they wanted me to bring my ammunition and consecrate it. I was not sure what that entailed and I don't know how to do it, and while I was sitting there trying to figure it out, I began to think "Bless my hands for war." Strange except I have said that many time to God between 1981 and 2005 when I was a soldier warrior. But why would God or anyone for that matter want me to have blessed war hands. War is for young people, who can do forty push-ups and not wind up in traction. War is for the young at heart and the quick at mind. But David prayed to God in Psalm 144 saying this:

1 Blessed be the Lord, my rock,

Who trains my hands for war,

And my fingers for battle;

2 My lovingkindness and my fortress,

My stronghold and my deliverer,

My shield and He in whom I take refuge,

Who subdues my people under me.

3 O Lord, what is man, that You take knowledge of him?

Or the son of man, that You think of him?

4 Man is like a mere breath;

His days are like a passing shadow.

New American Standard Bible : 1995 update. 1995 . The Lockman Foundation: LaHabra, CA

I am certainly no king fighting to gain control of a kingdom from a man whom I swore I would never bring harm to, and thus proved it many times as Saul was often within David's reach. I am not on the run from a king and his soldiers as they seek to take my life, and thus my claim to the thrown. And before you get carried away with analogies, I do not see any resemblance between this account and our current situation in America. This was specifically regarding God's chosen people, his chosen King, and His Chosen nation, Israel. Do not try to take it out of its proper context!

So what is the point? And then it hit me like a .380 hollow-point square in the forehead.

Psalm 144:3-4. "O Lord, what is man, that You take knowledge of him?

Or the son of man, that You think of him? Man is like a mere breath;

His days are like a passing shadow."

Well, it seems that the whole encounter has nothing to do with becoming John Nash-like (I am certain not to win a Nobel prize). It has nothing to do with weapons and ammunition, wars nor rumors of wars. It only has to do with "what is man, that God would even acknowledge his very existence, it being like a breath among an eternity of breathing. Yep, I am here for one tick on the timeline. And so are you, and so are our leaders. If our current administration were to serve five terms, it is just a breath to God.

David had figured out two important things: (1) Saul's reign as king would pass, and (2) David's own reign would pass, and even his own kingdom would pass.

What a dreary thought. Someone has said that we begin to die the moment we are born. Perhaps they had insight into the quantum.

If you look at the remainder of Psalm 144, David cries out to be rescued from his desperate situation. But by the time he reached verse 15, something has changed, and he requites saying: "How blessed is the people whose God is the Lord."

God need not bless my hands to war, my country to greatness, my works to Pulitzer caliber. God has already blessed me, forgiven me, accepted me, and the rest is just a breath...

I don’t know if my “friends will be returning”, it looks like they have left for the night and I feel like I have permission to go to bed. I hope the guys in white jackets will not be here when I awake.

Jeffrey Strickland, Ph.D.

Uplift (Netlift) Modeling

noreply@blogger.com (Anonymous) — Fri, 16 Nov 2012 02:24:00 +0000

This blog describes basic concepts, benefits and challenges of implementation of Net Lift Models in direct marketing campaigns. Net lift models predict which customer segments are likely to make a purchase ONLY if prompted by a marketing undertaking. The modeling work was conducted using stepwise logistic regression in SAS Enterprise Miner ®.

The paper provides examples how net lift probability decomposition models leveraged differences between purchasers in test group and control group to predict which customer segments need a marketing contact and which customers segments are likely to make purchasing decision without a nudge.

TRADITIONAL APPROACH TO DIRECT MARKETING LIST MODELING

Majority of direct marketing campaigns are based on purchase propensity models, selecting customer email, paper mail or other marketing contact lists based on customers’ probability to make a purchase.

Scoring Rank	Response Rate	Lift
1	28.1%	3.41
2	17.3%	2.10
3	9.6%	1.17
4	8.4%	1.02
5	4.8%	0.58
6	3.9%	0.47
7	3.3%	0.40
8	3.4%	0.41
9	3.5%	0.42
10	0.1%	0.01
Total	8.2%

Table 1. Example of standard purchase propensity model output used to generate direct campaign mailing list at 1800Flowers.com

This purchase propensity model had a ‘nice’ lift (rank’s response rate over total response rate) for the top 4 ranks on the validation data set. Consequently, we would contact customers included in top 4 ranks. After the catalog campaign had been completed, we conducted post analysis of mailing list performance vs. control group. The control group consisted of customers who were not contacted, grouped by the same purchase probability scoring ranks.

Sample campaign post analysis results:

	Mailing Group
Scoring Rank	Response Rate
1	27.0%
2	20.3%
3	10.7%
4	8.9%
Total	16.7%

Control Group

Response

Rate

27.9%

20.9%

10.0%

7.5%

16.5%

Incremental Response Rate

-0.91%

-0.56%

0.66%

1.38%

0.15%

Table 2. Campaign Post analysis

As shown the table 2, the top four customer ranks selected by propensity model perform we and control group. However, even though mailing/test group response rate was at decent le incremental response rate (mailing group net of control group) for combined top 4 ranks was low incremental response rate, our undertaking would be likely generating a negative ROI.

What was the reason that our campaign shown such poor incremental results? The purchase propensity model did its job well and we did send an offer to people who were likely to make a purchase. Apparently, modeling based on expected purchase propensity is not always the right solution for a successful direct marking campaign. Since there was no increase in response rate over control group, we could have been contacting customers who would have bought our product without promotional direct mail. Customers in top ranks of purchase propensity model may not need a nudge or they are buying in response to a contact via other channels. If that is the case, the customers in the lower purchase propensity ranks would be more ‘responsive’ to a marketing contact.

We should be predicting incremental impact – additional purchases generated by a campaign, not purchases that would be made without the contact. Our marketing mailing can be substantially more cost efficient if we don’t mail customers who are going to buy anyway.

Since customers very rarely use promo codes from catalogs or click on web display ads, it is difficult to identify undecided, swing customer based on the promotion codes or web display clickthroughs.

Net lift models predict which customer segments are likely to make a purchase ONLY if prompted by a marketing undertaking.

Purchasers from mailing group include customers that needed a nudge, however, all purchasers in the holdout/control group did not need our catalog to made their purchasing decision. All purchasers in the control group can be classified as ‘need no contact’. Since we need a model that would separate ‘need contact’ purchasers from ‘no contact’ purchasers, the net lift models look at differences in purchasers in mailing (contact) group versus purchasers from control group.

In order to classify our customers into these groups we need mailing group and control group purchases results from similar prior campaigns. If there are no comparable historic undertakings, we have to create a small scale trial before the main rollout.

All models described in this project used stepwise logistic regression on data partitioned into test and validation sets. All data prep work was done in base SAS ® and all modeling was done in SAS Enterprise Miner ®.

NET LIFT MODELS

There has been recent mentions of a target selection (i.e., case selection) technique referred to as net lift, uplift, incremental response, differential response, and possible other names. When posed as a return maximization problem, net lift and the usual target selection practice coincide. Net lift applies to target selection in situations with a binary treatment; return maximization provides direction on how to handle problems in situations with more than one treatment.

Definition of Uplift modeling: Analytically modeling to predict the influence on a customer's buying behavior that results from choosing one marketing treatment (customer-facing action) over another. The secondary treatment is often passive – make no contact – as evaluated over a control group. The uplift model answers the question, “How much more likely is this treatment to generate the desired outcome than the alternative treatment?” For each customer, the model's prediction drives the decision of which treatment to apply [3].

Problem statement

Given the following data [2]:

· Cases P = {1,…,n},

· Treatments J = {1,…,U},

· expected return R(i,t) for each case and treatment ,

· non-negative integers n₁,…,n_U such that

n₁ + … + n_U = n

find a treatment assignment

f: P→J

so that the total return

∑_{[i=1 to n]}R_if(i)

is maximized, subject to the constraints that the number of cases assigned to treatment j is not to exceed n_j (j=1,…,U) [2].

Example 1: Mailing campaign

· P: a group of customers,

· two treatments:

1. treatment 1: send a promotional coupon; R_i₁ is the expected return if a coupon is sent to customer i,

2. treatment 2: no coupon is sent; the expected return is zero: R_i₂= 0

Solution to the maximization problem:

• assign treatment 1 to the customers with the n₁ largest values of R_i₁

• assign treatment 2 to the remaining customers

This solution can also be derived from the Neyman-Pearson lemma.

Example 2: Marketing action case

• P: a group of customers,

• two treatments:

• treatment 1: exercise some marketing action; R_i₁ is the expected return if treatment 1 is given to customer i,

• treatment 2: exercise no the marketing action; let R_i₂ be the expected return if treatment 2 is given to customer

Solution to the maximization problem:

The second sum does not involve f, so maximizing total return is equivalent to maximizing the first term

As for to the solution to Example 1, to attain the maximum return:

• assign treatment 1 to the customers with the n₁ largest values of R_i₁ – R_i₂

• assign treatment 2 to the remaining customers

The difference R_i₁ – R_i₂ is called net lift, uplift, incremental response, differential response, etc.

If one considers only the response to treatment 1, bases targeting on a model built out of responses to previous marketing actions, one is proceeding as if the situation were as in Example1. One would mistakenly maximize

Such maximization would not yield the maximum return. One needs to consider the return from cases subjected to no marketing action.

Example 3: A toy example

Consider the following toy example with a population of n = 3 cases, and U = 3 treatments, n₁ = n₂ = n₃ = 1 and returns:

This assignment is one that maximizes total return under the given constraints:

Note that neither case 2 nor case 3 were assigned the treatment that maximize their return.

Although the possibility of a return of 18 exists, this possibility is not realized, since case 2 is not assigned treatment 2.

(In a case like this, one would probably advice that more resources be allocated to treatment 2, so that n₂ > 1.)

Example 4: General case

The problem can be cast as a standard integer linear programming problem. If we let

then the problem can be written as:

subject to the constraints:

Note:

In general, the best assignment that solves the linear programming problem does not vary continuously with the coefficients:

• small changes in the returns R_ij result in only small changes in the best total return,

• but, the assignment that yields the best return may vary considerably.

Example 5: A (n almost real) example and variation

Each week, a call center is responsible for contacting a group of customers. The length n of the list is not fixed, but it does not vary much from week to week.

Based on what is known of the customers, and on historical observations, it is possible to estimate the expected probability of successfully contacting each customer at different combinations of time of the day and call type (“home” or “other”).

Un-adjusted probabilities of successful contact are not constant in time…

Problem: make a (calling time, weekday) assignment so that expected total number of contacts is maximized, subject to the constraint that the call centre capacity is limited.

Remarks:

• in general, we will only know an estimate of R_ij:

which suggests that insisting on solving the full maximization problem is an over-kill

• in practice, proper call optimization is carried dynamically

A solution sketch:

• segment customers, including the probabilities of successful contact at different times as segmentation variables, so that the probability of contact is approximately constant for the segment

• solve the optimization problem for the fraction of each segment that has to be contacted at each time

NET LIFT MODELING APPROACH – PROBABILITY DECOMPOSITION MODELS

Segments used in probability decomposition models:

	Contacted Group	Control Group
Purchasers prompted by contact	A	D
Purchasers not needing contact	B	E
NonPurchasers	C	F

Figure 2. Segments in probability decomposition models

Standard purchase propensity models are only capable of predicting all purchasers (combined segments A and B). The probability decomposition model predicts purchasers segments that need to be contacted (segment A) by leveraging two logistic regression models, as shown in the formula below [1].

P(A I AUBUC) =	P(AUB I AUBUC) x	(2 - 1/P(AUB I AUBUE))
Probability of purchase prompted by contact	Probability of purchase out of contact group	Probability of purchaser being in contact group out of all purchasers

Summary of probability decomposition modeling process:

1. Build stepwise logistic regression purchase propensity model (M1) and record model score for every customer in a modeled population.

2. Use past campaign results or small scale trial campaign results to create a dataset with two equal size sections of purchasers from contact group and control group. Build a stepwise regression logistic model predicting which purchasers are from the contact group. The main task of this model will be to penalize the score of model built in the step 1 when purchaser is not likely to need contact.

3. Calculate net purchasers score based on probability decomposition formula

Results of the probability decomposition modeling process for marketing offer mailing.

S co ring R a nk	Co nta ct Gro up R e sp o nse %	Co ntro l Gro up R e sp o nse %	Incre me nta l R e sp o nse R a te
1	18.8%	12.9%	5.9%
2	7.8%	5.4%	2.4%
3	6.9%	4.5%	2.5%
4	4.3%	3.6%	0.7%
5	3.9%	3.5%	0.4%
6	4.1%	4.1%	0.0%
7	3.7%	4.0%	-0.2%
8	4.7%	4.1%	0.6%
9	5.0%	6.7%	-1.7%
10	11.0%	15.7%	-4.7%

Table 3. Post analysis of campaign leveraging probability decomposition model

Scoring Ranks 1 thru 6 show positive incremental response rates. The scoring ranks are ordered based on the incremental response rates.

CONCLUSION

The probability decomposition model is just one in a group of methods known as net lift models. The net lift models help maximize ROI of marketing campaigns as they let us avoid contacting customers or prospects who are highly likely to buy a product or service anyway. The traditional purchase propensity model may do a good job ranking customers based on their probability to make a purchase but it does not have the ability to select the true responders, the customers who will only make a purchase if contacted. The probability decomposition model has its challenges; it is relatively difficult to interpret as it combines scores of two separate model scores. Following is a list of conditions required for net lift model:

• presence of randomized control group

• analyzed marketing contact is not the only communication leading to purchase

• purchase rate is not correlated to lift, purchase propensity model is not sufficient

• presence of similar/repetitive marketing campaigns or small scale tests

• variation in average lift across scoring ranks

References

1. Jun Zhong, VP Targeting and Analytics, Card Services Customer Marketing, Wells Fargo in the presentation: “Predictive Modeling & Today’s Growing Data Challnges” at Predictive Analytics World in San Francisco, CA in 2009.

2. Lo, Victor S.Y. “The True Lift Model - A Novel Data Mining Approach to Response Modeling” in Database Marketing, SIGKDD Explorations. Volume 4 (2002), Issue 2, pg 78-86

3. Siegel, Eric, “Uplift Modeling: Predictive Analytics Can’t Optimize Marketing Decisions Without It”, Predictive Impact, Inc., 2011.

Net Lift Modeling with SAS

noreply@blogger.com (Anonymous) — Wed, 05 Sep 2012 13:37:00 +0000

Net Lift Modeling is best explained by Victor Lo in "The True Lift Model - A Novel Data Mining Approach to Response Modeling in Database Marketing", SIGKDD Explorations. Volume 4 (2002), Issue 2, pg 78-86. I will not add an explanation here. Rather I want to pass on some modeling notes from my experience with neft lift modeling with SAS Enterprise Guide and Enterprise Miner.

1. A customer developed a net lift model template in SAS Enterprise Miner (EM) that I have been using. The model is well developed and much generalized. However, the modeler/analyst must fully understand the developers customized code as well as SAS EM built-in functions. Also, each model must have startup code. A template is available for this, but it must be customized for the modeler’s environment. Correct library names and directory paths are essential.

2. Input data analysis consumes about 80% of the modeling time. Analysts have an “ethical modeling requirement” to fully understand and develop “intelligent” data sets before embarking on modeling the data. The customer derives its data from several internal and external sources and a typical marketing model may utilize a dozen or more SAS data sets. Model variables can number in the thousands, and observations in the millions. Data sets vary as much in their completeness as they do in their sources. Moreover, the data set developers do not fully understand how the data will be used.

3. Variable names are not always chosen for ease of modeling. For instance, one of the most important variables, from an information value perspective, is numerical with a 12-character name. By the time this variable has gone through variable selection, imputation, transformation, and regression, the name may increase to 21+ characters. When the interactions are formed, the variable name increases to 38 characters. The Lo regression in EM cannot handle the length of “long” variables names.

4. In order for the Lo Regression path to perform properly, numerical variables must be included for interaction terms. In the presence of the data cleansing macros, reformatted binary and categorical variables in a model with binary treatment and response, numerical values do not stand a chance of being included, especially if the intervals are wide. You can transform such a variable in EM but this lengthens the variable name even more. If the regression model does not include at least one numerical interval variable, the path will fail to execute. The EM net lift model template also appears to be sensitive to too many variable imputations and transformations occurring in EM itself.

5. Macros are a wonderful EG tools. An additional needed macro is one that analyzes the variable distribution for skewness, particularly that of an exponential random variate. For logistics regression, when such a variable appears, it is appropriate to perform a log transformation, and it is best to do this before importing the data to EM. I write query programs that perform this function.

6. Information value macro produces a lot of information that is useful in evaluating the possible impact of variables on a given treatment and response. In working with the data, some variables may provide redundant information, yet rank as important. For instance D_XXXXXX (D for binary dummy variable) provides the same information as XXXXXX_P_C (converted to binary and collapsed) and both are be binary. The analyst cannot just rank order variables 1 through 200 and blindly use them in the model with these redundancies. Other variables may be more informative.

7. The data that goes into the sample (and is compiled by macro that samples from different data bases, sorts, and renames variables) is in some cases ambiguous and not labeled to explain the variables. Data from the AXIOM company is a noteworthy exception, but the Getsample macro renaming loses this information. Analyst must invest the time necessary to understand the data they are including in the model. Failure to do so may result in a poor model.

8. I have found and corrected minor errors that have huge impacts in macros and lift model template code. Some of these were caused by a code syntax change between versions and is not the fault of the developer. Another part of the problem is that analyst open a macro or code node, make changes and save them, writing over the original files. Care must be taken to use SAVE AS… with a different file name when using SAS Enterprise products. Versioning is also recommended. Another problem occurs when variables are dropped from data sets or there are variable name changes.

9. As mentioned, numerical variables have little chance of being included in a logistics regression in the presence of numerous binary and nominal variables produced from macros. I have had to use techniques to force an important numerical variable into the logistics regression. Techniques include log transformations, manual selection, and reduced R-square or Chi-square constraints. However, this may allow variables that are not very explanatory into the model as well. Each variable must be analyzed for its Wald chi-square value and dealt with on a case by case basis.

10. Net lift models for rare events seem to be extremely sensitive to oversampling. If oversampling is required to fit a regression model, prior probabilities may need to be addressed in the population. Analysts should perform sensitivity analyses for these models, which requires additional time in the data.

11. Whenever the data changes or additional variables are added, the EM model must be adjusted. This is another time consuming process.

12. Modeling is as much of an art as it is a science. Particularly with data that does not follow the laws of physical dynamics, a marketing model may vary in appearance between different “artists”, much like an artist’s interpretation of painting of a landscape.

13. It is more important to understand the data analysis and modeling processes than it is to understand a single tool. Tools change and the come and go. There have also been coding changes between versions.

14. Food for thought: “All models are wrong; some are useful” (George Box). We are not modeling reality, just a “sub-reality” filled in with assumptions and sketchy data (in some cases). A good model is better than trying to achieve the non-existent perfect model.

15. Operations research analysts are adept at documenting models and processes and could be relied upon for doing this task. Writing is as much a part of their tool box as is analysis.

Back in the Saddle Again

noreply@blogger.com (Anonymous) — Wed, 05 Sep 2012 02:39:00 +0000

I took a break from blogging while getting established in a new job. My next series of blogs will be on predictive modeling using statitical modeling and analysis tools. I am currently working with SAS Enterprise Guide and Enterprise Miner performing net lift modeling. I would like to share some lessons learned.

Weird Scientists

noreply@blogger.com (Anonymous) — Tue, 13 Sep 2011 02:34:00 +0000

I just published the book "Weird Scientists". To see the rest of the characters in this series of blogs, go to Lulu.com and order the hardcover or download. My next series (and next book) is called "Divine Designers - the Creators of Geometry". The first blog in the series will follow in a few days.

Weird Scientist Number 3: Erwin Schrödinger

noreply@blogger.com (Anonymous) — Mon, 22 Aug 2011 00:49:00 +0000

Schrödinger and his cat

“God knows I am no friend of probability theory, I have hated it from the first moment when our dear friend Max Born gave it birth. For it could be seen how easy and simple it made everything, in principle, everything ironed and the true problems concealed. Everybody must jump on the bandwagon [Ausweg]. And actually not a year passed before it became an official credo, and it still is[i].” [1]

- Erwin Schrödinger

Maybe Erwin Rudolf Josef Alexander Schrödinger did not care much for probability theory, but he could not avoid it when trying to formulate his wave theory—and the Schrödinger equation—for subatomic particles. Schrödinger (12 August 1887 – 4 January 1961) was a physicist and theoretical biologist who was one of the fathers of quantum mechanics, and is famed for a number of important contributions to physics, especially the Schrödinger equation, for which he received the Nobel Prize in Physics in 1933. In 1935, after extensive correspondence with friend Albert Einstein, he proposed the Schrödinger’s cat thought experiment[ii].

Life and Career

Early years

In 1887, Schrödinger was born in Vienna, Austria to Rudolf Schrödinger (cerecloth producer, botanist) and Georgine Emilia Brenda (daughter of Alexander Bauer, Professor of Chemistry, k.u.k. Technische Hochschule Vienna).

His mother was half-Austrian and half English; the English side of her family came from Leamington Spa. Schrödinger learned English and German almost at the same time because both were spoken in the family household. His father was a Catholic and his mother was a Lutheran.

In 1898, he attended the Akademisches Gymnasium. Between 1906 and 1910, Schrödinger studied in Vienna under Franz Serafin Exner[iii] (1849–1926) and Friedrich Hasenöhrl[iv] (1874–1915). He also conducted experimental work with Karl Wilhelm Friedrich (“Fritz”) Kohlrausch (1884–1953) [2]. In 1911, Schrödinger became an assistant to Exner. At an early age, Schrödinger was strongly influenced by Schopenhauer[v] [3]. As a result of his extensive reading of Schopenhauer’s works, he became deeply interested throughout his life in color theory, philosophy[vi], perception, and eastern religion, especially Hindu Vedānta [vii].
Middle years

In 1914, Erwin Schrödinger achieved Habilitation[viii] (venia legendi). Between 1914 and 1918, he participated in war work as a commissioned officer in the Austrian fortress artillery (Gorizia, Duino, Sistiana, Prosecco, and Vienna). On 6 April 1920, Schrödinger married Annemarie Bertel. The same year, he became the assistant to Max Wien, in Jena, and in September 1920, he attained the position of ao. Prof. (Ausserordentlicher Professor), roughly equivalent to Reader (UK) or associate professor (US), in Stuttgart. In 1921, he became o. Prof. (Ordentlicher Professor, i.e. full professor), in Breslau (now Wrocław, Poland).

In 1921, he moved to the University of Zürich. In January 1926, Schrödinger published in Annalen der Physik the paper “Quantisierung als Eigenwertproblem” [translation: “Quantization as an Eigenvalue Problem[ix]“] on wave mechanics and what is now known as the Schrödinger equation[x]. In this paper, he gave a “derivation” of the wave equation for time independent systems, and showed that it gave the correct energy eigenvalues for the hydrogen-like atom. This paper has been universally celebrated as one of the most important achievements of the twentieth century, and created a revolution in quantum mechanics, and indeed of all physics and chemistry. A second paper was submitted just four weeks later that solved the quantum harmonic oscillator[xi], the rigid rotor[xii] and the diatomic molecule[xiii], and gives a new derivation of the Schrödinger equation. A third paper in May showed the equivalence of his approach to that of Heisenberg and gave the treatment of the Stark effect[xiv]. A fourth paper in this most remarkable series showed how to treat problems in which the system changes with time, as in scattering problems. These papers were the central achievement of his career and were at once recognized as having great significance by the physics community.

In 1927, he succeeded Max Planck at the Friedrich Wilhelm University in Berlin. In 1933, however, Schrödinger decided to leave Germany; he disliked the Nazis‘ anti-Semitism. He became a Fellow of Magdalen College at the University of Oxford. Soon after he arrived, he received the Nobel Prize together with Paul Dirac. His position at Oxford did not work out; his unconventional personal life (Schrödinger lived with two women) [4] was not met with acceptance. In 1934, Schrödinger lectured at Princeton University; he was offered a permanent position there, but did not accept it. Again, his wish to set up house with his wife and his mistress may have posed a problem. He had the prospect of a position at the University of Edinburgh but visa delays occurred, and in the end, he took up a position at the University of Graz in Austria in 1936.

In the midst of these tenure issues in 1935, after extensive correspondence with friend Albert Einstein, he proposed the Schrödinger’s cat thought experiment.
Later years

In 1939, after the Anschluss[xv], Schrödinger had problems because of his flight from Germany in 1933 and his known opposition to Nazism. He issued a statement recanting this opposition (he later regretted doing so, and he personally apologized to Einstein). However, this did not fully appease the new dispensation and the university dismissed him from his job for political unreliability. He suffered harassment and received instructions not to leave the country, but he and his wife fled to Italy. From there he went to visiting positions in Oxford and Ghent Universities.

In 1940, he received a personal invitation from Ireland’s Taoiseach (prime minister) Éamon de Valera[xvi] to reside in Ireland and agree to help establish an Institute for Advanced Studies in Dublin. He moved to Clontarf, Dublin, became the Director of the School for Theoretical Physics, and remained there for 17 years, during which time he became a naturalized Irish citizen. He wrote about 50 further publications on various topics, including his explorations of unified field theory[xvii].

In 1944, he wrote, “What is Life?” In this book, Schrödinger discusses the limitations of science (physics) in explaining life: “We must therefore not be discouraged by the difficulty of interpreting life by the ordinary laws of physics. For that is just what is to be expected from the knowledge we have gained of the structure of living matter. We must also be prepared to find a new type of physical law prevailing in it. Or are we to term it a non-physical, not to say a super-physical, law?” [5] Though this limitation exists, the book contains a discussion of negentropy[xviii] and the concept of a complex molecule with the genetic code for living organisms. According to James D. Watson’s memoir, DNA, the Secret of Life, Schrödinger’s book gave Watson the inspiration to research the gene, which led to the discovery of the DNA double helix structure. Similarly, Francis Crick, in his autobiographical book What Mad Pursuit, described how he was influenced by Schrödinger’s speculations about how genetic information might be stored in molecules. However, the geneticist and 1946 Nobel-prize winner H. J. Muller had in his 1922 article “Variation due to Change in the Individual Gene” [6] already laid out all the basic properties of the heredity molecule that Schrödinger derives from first principles in What is Life?, properties which Muller refined in his 1929 article “The Gene As The Basis of Life” [7] and further clarified during the 1930s, long before the publication of What is Life? [225].

In Nature and the Greeks (1954), Schrödinger discusses the limitations of the physical world for supplying meaning to every aspect of existence or reality. He writes [8]:

“I am very astonished that the scientific picture of the real world around me is deficient. It gives a lot of factual information, puts all our experience in a magnificently consistent order, but it is ghastly silent about all and sundry that is really near to our heart, that really matters to us. It cannot tell us a word about red and blue, bitter and sweet, physical pain and physical delight; it knows nothing of beautiful and ugly, good or bad, God and eternity. Science sometimes pretends to answer questions in these domains, but the answers are very often so silly that we are not inclined to take them seriously.”

Schrödinger stayed in Dublin until retiring in 1955. During this time he remained committed to his particular passion; involvements with students occurred and he fathered two children by two different Irish women [1] [4]. He had a life-long interest in the Vedānta philosophy of Hinduism, which influenced his speculations at the close of “What is Life?” about the possibility that individual consciousness is only a manifestation of a unitary consciousness pervading the universe [9].

In 1956, he returned to Vienna (chair ad personam). At an important lecture during the World Energy Conference, he refused to speak on nuclear energy because of his skepticism about it and gave a philosophical lecture instead. During this period, Schrödinger turned from mainstream quantum mechanics‘ definition of wave-particle duality and promoted the wave idea alone causing much controversy.

Personal life

Schrödinger suffered from tuberculosis and several times in the 1920s stayed at a sanatorium in Arosa. It was there that he discovered his wave equation [10].

Schrödinger decided in 1933 that he could not live in a country in which persecution of Jews had become a national policy. Alexander Frederick Lindemann, the head of physics at Oxford University, visited Germany in the spring of 1933 to try to arrange positions in England for some young Jewish scientists from Germany. He spoke to Schrödinger about posts for one of his assistants and was surprised to discover that Schrödinger himself was interested in leaving Germany. Schrödinger asked for a colleague, Arthur March, to be offered a post as his assistant.
The request for March stemmed from Schrödinger’s unconventional relationships with women: although his relations with his wife Anny were good, he had had many lovers with his wife’s full knowledge (and in fact, Anny had her own lover, Hermann Weyl). Schrödinger asked for March to be his assistant because, at that time, he was in love with March’s wife Hilde.

Many of the scientists who had left Germany spent mid-1933 in the Italian province of South Tyrol. Here Hilde became pregnant with Schrödinger’s child. On 4 November 1933 Schrödinger, his wife and Hilde March arrived in Oxford. Schrödinger had been elected a fellow of Magdalen College. Soon after they arrived in Oxford, Schrödinger heard that, for his work on wave mechanics, he had been awarded the Nobel Prize.

In early 1934, Schrödinger was invited to lecture at Princeton University and while there, he was made an offer of a permanent position. On his return to Oxford, he negotiated about salary and pension conditions at Princeton but in the end, he did not accept. It is thought that the fact that he wished to live at Princeton with Anny and Hilde both sharing the upbringing of his child was not found acceptable. The fact that Schrödinger openly had two wives, even if one of them was married to another man, was not well received in Oxford either. Nevertheless, his daughter Ruth Georgie Erica was born there on 30 May 1934 [1].

On 4 January 1961, Schrödinger died in Vienna at the age of 73 of tuberculosis. He left a widow, Anny (born Annemarie Bertel on 3 December 1896, died 3 October 1965), and was buried in Alpbach, Austria.

Legacy

The philosophical issues raised by Schrödinger’s cat are still debated today and remains his most enduring legacy in popular science, while Schrödinger’s equation is his most enduring legacy at a more technical level. The huge crater Schrödinger, on the far side of the Moon is named after him. The Erwin Schrödinger International Institute for Mathematical Physics was established in Vienna in 1993.

Color

One of Schrödinger’s lesser-known areas of scientific contribution was his work on color, color perception, and colorimetry[xix] (Farbenmetrik). In 1920, he published three papers in this area:

· “Theorie der Pigmente von größter Leuchtkraft,” Annalen der Physik, (4), 62, (1920), 603-622
· “Grundlinien einer Theorie der Farbenmetrik im Tagessehen,” Annalen der Physik, (4), 63, (1920), 397-426; 427-456; 481-520 (Outline of a theory of color measurement for daylight vision)
· “Farbenmetrik,” Zeitschrift für Physik, 1, (1920), 459-466 (Color measurement).
The second of these is available in English as “Outline of a Theory of Color Measurement for Daylight Vision” in Sources of Color Science, Ed. David L. MacAdam, The MIT Press (1970), 134-182.

Works Cited

[1] Moore, Walter J., Schrödinger: Life and Thought. s.l. : Cambridge University Press, 1992. p. 222. ISBN-13: 978-0521437677 .
[2] Nobelprize.org., Erwin Schrödinger - Biography. The Nobel Prize in Physics 1933. [Online] The Nobel Foundation, 1933. [Cited: July 20, 2011.] http://nobelprize.org/nobel_prizes/physics/laureates/1933/schrodinger-bio.html.
[3] Moore, Walter J., A Life of Erwin Schrödinger. Abridged edition. s.l. : Cambridge University Press, 1994. ISBN-13: 978-0521469340.
[4] O'Connor, J. J. and Robertson, E. F., Erwin Rudolf Josef Alexander Schrödinger. School of Mathematics and Statistics, University of St Andrews, Scotland . 2003. http://www-history.mcs.st-andrews.ac.uk/Biographies/Schrodinger.html. Retrieved 20 July 2011..
[5] Schrödinger, Erwin., What is Life?: with "Mind and Matter" and "Autobiographical Sketches". s.l. : Cambridge University Press, 1992. ISBN-13: 978-0521427081.
[6] Meÿenn, Karl., "Schrödinger, Erwin Rudolf Josef Alexander." Neue Deutsche Biographie. online version, 2007, Vol. 23, pp. 578-580. http://www.deutsche-biographie.de/pnd118823574.html.
[7] Muller, H. J., The American Naturalist: V.56 1922. s.l. : University of Michigan Library, 2001 [1929]. ASIN: B002IKLF18 .
[8] Schrödinger, Erwin., 'Nature and the Greeks' and 'Science and Humanism'. s.l. : Cambridge University Press, 1996. ISBN-13: 978-0521575508.
[9] Schwartz, James., In Pursuit of the Gene. From Darwin to DNA. s.l. : Harvard University Press, 2008. ISBN-13: 978-0674026704.
[10] New World Encyclopedia., "Schrödinger, Erwin." New World Encyclopedia. April 24, 2008. http://www.newworldencyclopedia.org/entry/Erwin_Schr%C3%B6dinger.

Notes

[i] 13th of June, 1946, in a letter to Albert Einstein, as quoted by Walter Moore in Schrödinger: Life and Thought (1989) ISBN 0521437679
[ii] Schrödinger’s cat is a thought experiment (see Chapter 37), usually described as a paradox, that Austrian physicist Erwin Schrödinger devised in 1935. It illustrates what he saw as the problem of the Copenhagen interpretation of quantum mechanics applied to everyday objects. The thought experiment presents a cat that might be alive or dead, depending on an earlier random event. In the course of developing this experiment, he coined the term Verschränkung (entanglement).
[iii] Franz Serafin Exner (24 March 1849 - October 15, 1926) was an Austrian physicist.
[iv] Friedrich Hasenöhrl (November 30, 1874 - October 7, 1915) was an Austro-Hungarian physicist.
[v] Arthur Schopenhauer (22 February 1788 – 21 September 1860) was a German philosopher known for his pessimism and philosophical clarity.
[vi] In his lecture "Mind and Matter," Chapter 4, he said that a phrase "that has become familiar to us" is "The world extended in space and time is but our representation (Vorstellung)." This is a repetition of the first words of Schopenhauer’s main work.
[vii] Vedānta was originally a word used in Hindu philosophy as a synonym for that part of the Veda texts known also as the Upanishads.
[viii] Habilitation is the highest academic qualification a scholar can achieve by his or her own pursuit in several European and Asian countries. Earned after obtaining a research doctorate, such as a Ph.D., habilitation requires the candidate to write a professorial thesis (often known as a Habilitationsschrift, or Habilitation thesis) based on independent scholarship, reviewed by and defended before an academic committee in a process similar to that for the doctoral dissertation. However, the level of scholarship has to be considerably higher than that required for a research doctoral (Ph.D.) thesis in terms of quality and quantity, and must be accomplished independently, in contrast with a Ph.D. dissertation typically directed or guided by a faculty supervisor.
[ix] The eigenvectors of a square matrix are the non-zero vectors that, after being multiplied by the matrix, remain proportional to the original vector (i.e., change only in magnitude, not in direction). For each eigenvector, the corresponding eigenvalue is the factor by which the eigenvector changes when multiplied by the matrix. The prefix eigen- is adopted from the German word "eigen" for "own" in the sense of a characteristic description. The eigenvectors are sometimes also called characteristic vectors. Similarly, the eigenvalues are also known as characteristic values.
[x] The Schrödinger equation was formulated in 1926 by Austrian physicist Erwin Schrödinger. Used in physics, specifically quantum mechanics, it is an equation that describes how the quantum state of a physical system changes in time. In the standard interpretation of quantum mechanics, the quantum state, also called a wavefunction or state vector, is the most complete description that can be given to a physical system. Solutions to Schrödinger’s equation describe not only molecular, atomic and subatomic systems, but also macroscopic systems, possibly even the whole universe.
[xi] The quantum harmonic oscillator is the quantum-mechanical analog of the classical harmonic oscillator. Because an arbitrary potential can be approximated as a harmonic potential at the vicinity of a stable equilibrium point, it is one of the most important model systems in quantum mechanics. Furthermore, it is one of the few quantum-mechanical systems for which a simple, exact solution is known.
[xii] The rigid rotor is a mechanical model that is used to explain rotating systems. An arbitrary rigid rotor is a 3-dimensional rigid object, such as a top. To orient such an object in space three angles are required. A special rigid rotor is the linear rotor that is a 2-dimensional object, requiring two angles to describe its orientation. An example of a linear rotor is a diatomic molecule. More general molecules like water (asymmetric rotor), ammonia (symmetric rotor), or methane (spherical rotor) are 3-dimensional, see classification of molecules.
[xiii] Diatomic molecules are molecules composed only of two atoms, of either the same or the different chemical elements. The prefix di- means two in Greek. Common diatomic molecules are hydrogen (H2), nitrogen (N2), oxygen (O2), and carbon monoxide (CO). Seven elements exist in the diatomic state in the liquid and solid forms: H2 , N2, O2, F2, Cl2, Br2, and I2. Most elements (and many chemical compounds) aside from these form diatomic molecules when evaporated, although at very high temperatures, all materials disintegrate into atoms. The noble gases do not form diatomic molecules.
[xiv] The Stark effect (named after Johannes Starck)is the shifting and splitting of spectral lines of atoms and molecules due to the presence of an external static electric field. The amount of splitting and or shifting is called the Stark splitting or Stark shift.
[xv] The Anschluss (spelled Anschluß at the time of the event, and until the German orthography reform of 1996; German for "link-up"), also known as the Anschluss Österreichs, was the occupation and annexation of Austria into Nazi Germany in 1938.
[xvi] Éamon de Valera(14 October 1882 – 29 August 1975) was one of the dominant political figures in twentieth century Ireland, serving as head of government and head of state and introducing the Constitution of Ireland. De Valera was a leader of Ireland’s struggle for independence from Britain in the Irish War of Independence and of the anti-Treaty forces in the ensuing Irish Civil War (1922–23). In 1926, he founded Fianna Fáil and was head of government from 1932–48, 1951–54 and 1957–59 and President of Ireland from 1959–73.
[xvii] Since the 19th century, some physicists have attempted to develop a single theoretical framework that can account for the fundamental forces of nature—a unified field theory. Classical unified field theories are attempts to create a unified field theory based on classical physics. In particular, unification of gravitation and electromagnetism was actively pursued by several physicists and mathematicians in the years between World War I and World War II. This work spurred the purely mathematical development of differential geometry. Albert Einstein is the best known of the many physicists who attempted to develop a classical unified field theory.
[xviii] The negentropy, also negative entropy or syntropy, of a living system is the entropy that it exports to keep its own entropy low; it lies at the intersection of entropy and life. The concept and phrase "negative entropy" were introduced by Erwin Schrödinger in his 1943 popular-science book What is Life?
[xix] Colorimetry is "the science and technology used to quantify and describe physically the human color perception."

Weird Scientist Number 2: Satyendra Nath Bose

noreply@blogger.com (Anonymous) — Tue, 02 Aug 2011 01:35:00 +0000

"Perhaps Satyen Bose’s greatest charm lies in his ability to look at life in a total manner. The minor pleasure of leisure and pleasant company were to him a part of a bigger universe of the pleasures of the mind and the intellect. In a sense this was also his strongest limitation. Bose was a man who tried to see the world around him in its entirety, in its complexity and in which his particular science and himself were small parts."

B. D. Nag Chaudhuri

Satyendra Nath Bose (1 January 1894 – 4 February 1974), FRS, was an Indian mathematician and physicist noted for his collaboration with Albert Einstein in developing a theory regarding the gaslike qualities of electromagnetic radiation. He is best known for his work on quantum mechanics in the early 1920s, providing the foundation for Bose–Einstein statistics and the theory of the Bose–Einstein condensate. He is honoured as the namesake of the boson (bosonparticle.com, 2011). He was awarded India's second highest civilian award, the Padma Vibhushan in 1954 by the Government of India (NIC, 2005).

Satyendra Nath Bose in Paris 1925

Although more than one Nobel Prize was awarded for research related to the concepts of the boson, Bose–Einstein statistics and Bose–Einstein condensate—the latest being the 2001 Nobel Prize in Physics, which was given for advancing the theory of Bose–Einstein condensates—Bose himself was not awarded the Nobel Prize. Among his other talents, Bose spoke several languages and could also play the esraj, a musical instrument similar to a violin.

In his book, The Scientific Edge, the noted physicist Jayant Narlikar observed (Narlikar, 2003):

“S. N. Bose’s work on particle statistics (c. 1922), which clarified the behaviour of photons (the particles of light in an enclosure) and opened the door to new ideas on statistics of Microsystems that obey the rules of quantum theory, was one of the top ten achievements of 20th century Indian science and could be considered in the Nobel Prize class.”

Early life and career

Bose was born in Calcutta, British India, the eldest of seven children. His father, Surendranath Bose, worked in the Engineering Department of the East Indian Railway Company. Bose attended Hindu School in Calcutta, and later attended Presidency College, also in Calcutta, earning the highest marks at each institution. He came in contact with teachers such as Jagadish Chandra Bose and Prafulla Chandra Roy who provided inspiration to aim high in life. From 1916 to 1921 he was a lecturer in the physics department of the University of Calcutta. In 1921, he joined the department of Physics of the then recently founded Dhaka University (now in Bangladesh and called University of Dhaka) (BANC, 2011).

In 1924, while working as a Reader at the Physics Department of the University of Dhaka, Bose wrote a paper deriving Planck's quantum radiation law without any reference to classical physics and using a novel way of counting states with identical particles. This paper was seminal in creating the very important field of quantum statistics. After initial setbacks to his efforts to publish, he sent the article directly to Albert Einstein in Germany. Einstein, recognizing the importance of the paper, translated it into German himself and submitted it on Bose's behalf to the prestigious Zeitschrift für Physik. As a result of this recognition, Bose was able to leave India for the first time and spent two years in Europe, during which he worked with Louis de Broglie, Marie Curie, and Einstein (Grigrious, 2010).

After his stay in Europe, Bose returned to Dhaka in 1926. He became a professor and was made head of the Department of Physics, and continued teaching at Dhaka University until 1945. He was also Dean of the Faculty of Science at Dhaka University for a long period. When the partition of India became imminent, he returned to Calcutta and taught at Calcutta University until 1956, when he retired and was made professor emeritus (Chatterjee & Chatterjee, 2002).

Bose–Einstein statistics

Bose was an ardent follower of Einstein’s ideas and decided to attempt another derivation of Planck’s law of radiation without using any Maxwellian wave theory of radiation. He had read Planck’s work on the distribution of energy from a black body based on this new theory. Satyendra Nath had always been a perfectionist and would not accept any ad hoc assumption while working out a theory. So he was not happy with Planck’s derivation which had such ad hoc assumptions (Chatterjee & Chatterjee, 2002).

While presenting a lecture at the University of Dhaka on the theory of radiation and the ultraviolet catastrophe, Bose attempted a statistical explanation of the interaction of atoms and radiation using the principle, recently developed by Werner Heisenberg, of uncertainty concerning the characteristics of electrons around the atomic nucleus (Walker A. R., 2011). During this lecture, Bose committed an error in applying the theory, which unexpectedly gave a prediction that agreed with the experiment. He adapted this lecture into a short paper called “Planck's Law and the Hypothesis of Light Quanta”.

The error was a simple mistake—similar to arguing that flipping two fair coins will produce two heads one-third of the time—that would appear obviously wrong to anyone with a basic understanding of statistics. However, the results it predicted agreed with experiment, and Bose realized it might not be a mistake at all. He for the first time took the position that the Maxwell–Boltzmann distribution would not be true for microscopic particles where fluctuations due to Heisenberg's uncertainty principle will be significant. Thus he stressed the probability of finding particles in the phase space, each state having volume , and discarding the distinct position and momentum of the particles (Grigrious, 2010).

When Bose sent his paper to Philosophical Magazine (Chatterjee & Chatterjee, 2002) for publication it was rejected because the editors thought he had made a simple mistake in his calculations. So he sent a copy to Einstein, soliciting his opinion. (Walker A. R., 2011). Here is what Bose wrote in his first letter to Einstein (Chatterjee & Chatterjee, 2002):

“I have ventured to send you the accompanying article for your perusal and opinion. I am anxious to know what you think of it. You will see that I have tried to deduce the coefficient in Planck’s laws, independent of the classical electrodynamics, only assuming that the ultimate elementary regions in the phase space had the content . I do not know sufficient German to translate the paper. If you think the paper worth publication, I shall be grateful if you arrange for its publication in Zeitschrift fur Physik. Though a complete stranger to you, I do not feel any hesitation in making such a request. Because we are all your pupils though profiting only by your teachings through your writings. I do not know whether you still remember that somebody from Calcutta asked your permission to translate your papers on relativity in English. You acceded to the request. The book has since been published. I was the one who translated your paper ‘Generalised Relativity’.”

Einstein approved. His theory finally achieved respect when Einstein sent his own paper in support of Bose's to Zeitschrift für Physik, asking that they be published together (Walker A. R., 2011). This was done in 1924. Very soon the paper was published, translated by Einstein, and with the following translator’s remark (Chatterjee & Chatterjee, 2002):

“In my opinion Bose’s derivation of the Planck formula signifies an important development. The method considered here yields also the quantum theory of ideal gases which I shall discuss elsewhere.”

The reason Bose's "mistake" produced accurate results was that since photons are indistinguishable from each other, one cannot treat any two photons having equal energy as being two distinct identifiable photons. By analogy, if in an alternate universe coins were to behave like photons and other bosons, the probability of producing two heads would indeed be one-third ( ). But what Bose actually did was more than derive a formula. He introduced new concepts in physics Bose's "error" (Grigrious, 2010), later named as Bose Statistics. Einstein understood the significance of it and immediately applied it to the case of ideal gas and found a new relation known as Bose-Einstein Statistics (Chatterjee & Chatterjee, 2002).

Einstein adopted the idea and extended it to atoms. This led to the prediction of the existence of phenomena which became known as Bose-Einstein condensate, a dense collection of bosons (which are particles with integer spin, named after Bose), which was demonstrated to exist by experiment in 1995 (Grigrious, 2010).

Later work

Bose's ideas were afterwards well received in the world of physics, and he was granted leave from the University of Dhaka to travel to Europe in 1924 (Grigrious, 2010). He spent a year in France and worked with Marie Curie, and met several other well-known scientists. He then spent another year abroad, working with Einstein in Berlin. Upon his return to Dhaka, he was made a professor in 1926. He did not have a doctorate, and so ordinarily he would not be qualified for the post, but Einstein recommended him. His work ranged from X-ray crystallography to unified field theories. He also published an equation of state for real gases with Megh Nad Saha (Chatterjee & Chatterjee, 2002).

When Satyendra Nath returned to Dacca from Europe in 1926, the post of the professor of physics at the Dacca University was vacant. The selection committee recommended the name of D.M. Bose as its first choice and that of S.N. Bose as its alternative choice. D.M. Bose was then the Ghose professor of physics at Calcutta. He was comfortably settled in his research work and was also in close touch with his uncle, Sir J.C Bose and his research laboratory, the Bose Institute, adjacent to the University College of Science and Technology. Since D.M. Bose did not accept the position offered to him by the Dacca University, Satyendra Nath became the professor of physics at Dacca University, where he continued till 1945 (Chatterjee & Chatterjee, 2002).

Apart from physics he did some research in biochemistry and literature (Bengali, English). He made deep studies in chemistry, geology, zoology, anthropology, engineering and other sciences. Being an Indian of Bengali descent, he devoted a lot of time to promoting Bengali as a teaching language, translating scientific papers into it, and promoting the development of the region (Grigrious, 2010).

In 1944 Bose was elected General President of the Indian Science Congress. In 1958 he became a Fellow of the Royal Society.

Works Cited

BANC. (2011). Satyendra Nath Bose . Retrieved August 1, 2011, from Bengali Association of North Carolina: www.banc-online.org/pdfs/SNB.pdf

bosonparticle.com. (2011). Articles about boson particle. Retrieved August 1, 2011, from Boson Particle: http://www.bosonparticle.com/articles

Chatterjee, S., & Chatterjee, E. (2002). Satyendra Nath Bose. National Book Trust.

Grigrious, V. I. (2010, February). Satyendra Nath Bose. Bhouthika Jyothi, 6(Ruby Jubilee Special), p. 41.

Narlikar, J. V. (2003). The Scientific Edge. Penguin Books.

NIC. (2005). Padma Vibhushan Awardees. (National Informatics Centre, Government of India.) Retrieved August 1, 2011, from india.gov.in: http://india.gov.in/myindia/padmavibhushan_awards_list1.php?start=250

Walker, A. R. (2011). From black bodies to bar codes: lasers. In Magical inventions or the art of discovery. Unpublished.

Weird Scientis Number 1: Clinton Davisson

noreply@blogger.com (Anonymous) — Tue, 26 Jul 2011 02:49:00 +0000

This is the first in a series of blogs featuring the creators of Quantum Mechanics/Theory. These weird scientists (in the most respectable sense) are part of my not-yet-published book "Weird Scientists (not to be confuses with "Weird Science", a 1985 American teen fantasy comedy film).

Clinton Davisson

“We think we understand the regular reflection of light and X rays - and we should understand the reflections of electrons as well if electrons were only waves instead of particles ... It is rather as if one were to see a rabbit climbing a tree, and were to say ‘Well, that is rather a strange thing for a rabbit to be doing, but after all there is really nothing to get excited about. Cats climb trees—so that if the rabbit were only a cat, we would understand its behavior perfectly.’ Of course, the explanation might be that what we took to be a rabbit was not a rabbit at all but was actually a cat. Is it possible that we are mistaken all this time in supposing they are particles, and that actually they are waves?”

- Clinton Davisson

Clinton Joseph Davission (October 22, 1881 – February 1, 1958) studied the properties of subatomic particles, not cats or rabbits. Davisson was an American physicist who won the 1937 Nobel Prize in Physics for his discovery of electron diffraction. Davisson shared the Nobel Prize with George Paget Thomson, who independently discovered electron diffraction at about the same time as Davisson. Their results provided poof for Louis de Broglie's pioneering theory of wave-particle duality in quantum mechanics.

Biography

Early years

Davisson was born in Bloomington, Illinois, on 22 October 1881, the first of two children. His father, Joseph, who had settled in Bloomington after serving in the Civil War, was a contract painter and paperhanger by trade. His mother, Mary, occasionally taught in the Bloomington school system. Their home was, as Davisson's sister, Carrie, characterized it, "a happy congenial one—plenty of love but short on money."

Davisson, slight of frame and frail throughout his life, graduated from high school at age 20, in 1902. For his proficiency in mathematics and physics, he received a one-year scholarship to the University of Chicago; his six-year career there was interrupted several times for lack of funds. He acquired his love and respect for physics from Robert Millikan[i]; Davisson was "delighted to find that physics was the concise, orderly science [he] had imagined it to be, and that a physicist [Millikan] could be so openly and earnestly concerned about such matters as colliding bodies" (Gehrenbeck, 1978).

In 1905, upon the recommendation of Millikan, Davisson was hired by Princeton University as Instructor of Physics. He completed the requirements for his B.S. degree from Chicago in 1908, mainly by working in the summers. While teaching at Princeton, he did doctoral thesis research with Owen Richardson[ii]. He received his Ph.D. in physics from Princeton in 1911; in the same year he married Richardson's sister, Charlotte (Kelly, 1962) (Nobelprize.org, 1937).

Career

Before finishing his undergraduate degree at Chicago, he became a part-time instructor in physics at Princeton University, where he came under the influence of the British physicist Richardson, who was directing electronic research there. Davisson's PhD thesis at Princeton, in 1911, extended Richardson's research on the positive ions emitted from salts of alkaline metals. Davisson later credited his own success to having caught "the physicist's point of view—his habit of mind—his way of looking at things" from such men as Millikan and Richardson (Kelly, 1962).

After completing his degree, Davisson married Richardson's sister, Charlotte, who had come from England to visit her brother. After a honeymoon in Maine, Davisson joined the Carnegie Institute of Technology in Pittsburgh as an instructor in physics. The 18-hour-per-week teaching load left little time for research, and in six years there he published only three short theoretical notes. One notable break during this period was the summer of 1913, when Davisson worked with J. J. Thomson at the Cavendish laboratory in England.

Davisson was then appointed as an assistant professor at the Carnegie Institute of Technology. In April 1917, he was refused enlistment in the United States Army, because of his frailty (MacRae, 1972). In June of the same year he accepted war-time employment in the Engineering Department of the Western Electric Company (later Bell Telephone Laboratories), New York City—at first for summer, then, on leave of absence from Carnegie Tech., for the duration of the World War. His work was to develop and test oxide-coated nickel filaments to serve as substitutes for the oxide-coated platinum filaments then in use. At the end of the war, he resigned an assistant professorship to which he had been appointed at Carnegie Tech. to continue as a Member of the Technical Staff of the Telephone Laboratories (Nobelprize.org, 1937).

At the end of the war, Davisson accepted a permanent position at Western Electric after receiving assurances of his freedom there to do basic research. He had found that his teaching responsibilities at the Carnegie Institute largely precluded him from doing research (Kelly, 1962). The assignment that engaged Davisson and Lester Germer in their first joint effort reflects one of the chief interests of the parent company, AT&T, at this time: to conduct a fundamental investigation into the role of positive-ion bombardment in electron emission from oxide-coated cathodes. They published their results in the Physical Review in 1920, concluding that positive-ion bombardment has a negligible effect on the electron emission from oxide-coated cathodes (C. J. Davisson, 1920).

Davisson remained at Western Electric (and Bell Telephone) until his formal retirement in 1946. He then accepted a research professor appointment at the University of Virginia that continued until his second retirement in 1954 (Kelly, 1962).

Electron Diffraction and the Davisson-Germer Experiment

“Discoveries in physics are made when the time for making them is ripe, and not before.” (Davisson, 1965)

- Clinton Davisson

The Davisson–Germer experiment was a physics experiment conducted by American physicists Clinton Davisson and Lester Germer in 1927, which confirmed the de Broglie hypothesis. The de Broglie hypothesis says that particles of matter (such as electrons) have wave properties. This demonstration of wave–particle duality was important historically in the establishment of quantum mechanics and of the Schrödinger equation.

The experiment consisted of firing an electron beam from an electron gun on a nickel crystal at normal incidence (i.e. perpendicular to the surface of the crystal). The electron gun consisted of a heated filament that released thermally excited electrons, which were then accelerated through a potential difference of 54 V, giving them a kinetic energy of 54 eV. An electron detector was placed at an angle to obtain a maximum reading, and measured the number of electrons that were scattered at that particular angle (Germer, 1964) (Eisberg & Resnick, 1985).

Davisson, Germer and Calbick in 1927, the year they demonstrated electron diffraction. In their New York City laboratory are Clinton Davisson, age 46; Lester Germer, age 31, and their assistant Chester Calbick, age 23. Germer, seated at the observer's desk, appears ready to read and record electron current from the galvanometer[iii] (seen beside his head); the banks of dry cells behind Davisson supplied the current for the experiments.

Diffraction is a characteristic effect when a wave is incident upon an aperture or a grating, and is closely associated with the meaning of wave motion itself. In the 19th Century, diffraction was well-established for light and for ripples on the surfaces of fluids. In 1927, while working for Bell Labs, Davisson and Germer performed their famous experiment showing that electrons were diffracted at the surface of a crystal of nickel. This celebrated Davisson-Germer experiment confirmed the de Broglie hypothesis that particles of matter have a wave-like nature, which is a central tenet of quantum mechanics. In particular, their observation of diffraction allowed the first measurement of a wavelength for electrons. The measured wavelength agreed well with de Broglie's equation , where is Planck's constant and is the electron's momentum (Germer, 1964) (Davisson, 1965).

The sixth of January 1927 might well be regarded as the birthday of electron waves, for it was the day that data directly supporting the de Broglie hypothesis of electron waves were first observed. Note the peak deflection at 65 volts, and the detailed study of the region directly below. Calbick's handwriting is neat and cautious; Germer's is bold and expansive. Davisson made no entries in any of the research notebooks kept in the Bell Labs files.

Davisson and Germer succeeded where others had failed. In fact, the others (Walter Elsasser, E. G. Dymond, Patrick Blackett, James Chadwick and Charles Ellis), who had the idea of electron diffraction considerably ahead of Davisson and Germer, were not able to produce the desired experimental evidence for it. George Paget Thomson, who did find that evidence by a very different method, testified to the magnitude of the technical achievement as follows (Thomson, 1961):

"[Davisson and Germer's work] was indeed a triumph of experimental skill. The relatively slow electrons [they] used are most difficult to handle. If the results are to be of any value the vacuum has to be quite outstandingly good. Even now [1961] ... it would be a very difficult experiment. In those days it was a veritable triumph. It is a tribute to Davisson's experimental skill that only two or three other workers have used slow electrons successfully for this purpose."

From 1930-1937, Davisson devoted himself to the study of the theory of electron optics[iv] and to applications of this theory to engineering problems. He then investigated the scattering and reflection of very slow electrons by metals. During World War II he worked on the theory of electronic devices and on a variety of crystal physics[v] problems.

In 1946 he retired from Bell Telephone Laboratories after 29 years of service. From 1947 to 1949, he was Visiting Professor of Physics at the University of Virginia, Charlottesville, Va.
The National Academy of Sciences awarded Davisson the Comstock Prize in 1928. In 1931 Franklin Institute awarded him the Elliott Cresson Medal, and in 1935 the Royal Society (London) awared him the Hughes Medal. In 1941, the University of Chicago awarded him the Alumni Medal. He held honorary doctorates from Purdue University, Princeton University, the University of Lyon and Colby College.

Personal life

In 1911, Clinton married Charlotte Sara Richardson, a sister of Professor Richardson. Clinton and Charlotte Davisson had four children, including the American physicist Richard Davisson. Clinton died in Charlottesville on February 1, 1958, at the age of 76, and was survived by his wife, three sons and one daughter. The crater Davisson on the Moon is named after him.

Notes

[i] Robert A. Millikan (22 March 1868 – 19 December 1953) was an American experimental physicist, and Nobel laureate in physics for his measurement of the charge on the electron and for his work on the photoelectric effect. He served as president of Caltech from 1921 to 1945. He also served on the board of trustees for Science Service, now known as Society for Science & the Public, from 1921-1953.
[ii] Sir Owen Willans Richardson, FRS (26 April 1879 - 15 February 1959) was a British physicist who won the Nobel Prize in Physics in 1928 for his work on thermionic emission, which led to Richardson's Law—the current from a heated wire seemed to depend exponentially on the temperature of the wire.
[iii] A galvanometer is a type of ammeter: an instrument for detecting and measuring electric current. It is an analog electromechanical transducer that produces a rotary deflection of some type of pointer in response to electric current flowing through its coil. The term has expanded to include uses of the same mechanism in recording, positioning, and servomechanism equipment.
[iv] Electron optics deals with the focusing and deflection of electrons using magnetic and/or electrostatic fields.
[v] Chystal physics (physical crystallography), is the study of the physical properties of crystals and crystalline aggregates and changes in the properties under the influence of various factors.

Works Cited

C. J. Davisson, L. H. (1920). Physics Review, 15, p. 330.

Davisson, C. (1965). The Discovery of Electron Waves. In P. 1.-1. Nobel Lectures. Amsterdam: Elsevier Publishing Company.

Eisberg, R., & Resnick, R. (1985). Quantum Physics of Atoms, Molecules, Solids, Nuclei, and Particles (2nd ed.). New York: John Wiley & Sons.

Gehrenbeck, R. K. (1978, January). Electron diffraction:fifty years ago. Physics Today.Germer, L. H. (1964, July). Low-Energy Electron Diffraction. Physics Today, pp. 19-23.

Kelly, M. J. (1962). Clinton Joseph Davisson. In Biographical Memoirs Vol. XXXVI (pp. 52-79). New York: Columbia University Press.MacRae, A. U. (1972, January).

Lester H. Germer - Obituary. Physics Today, pp. 93-97.Nobelprize.org. (1937). Clinton Davisson - Biography. The Nobel Prize in Physics 1937.

Thomson, G. P. (1961). The Inspiration of Science. London: Oxford U.P.

General Relativity (excerpt from Albert Einstein: "Nobody expected me to lay golden eggs"

noreply@blogger.com (Anonymous) — Sat, 16 Jul 2011 03:36:00 +0000

“The general theory of relativity is as yet incomplete insofar as it has been able to apply the general principle of relativity satisfactorily only to gravitational fields, but not to the total field. We do not yet know with certainty, by what mathematical mechanism the total field in space is to be described and what the general invariant laws are to which this total field is subject. One thing, however, seems certain: namely, that the general principle of relativity will prove a necessary and effective tool for the solution of the problem of the total field.”

- Albert Einstein, “The theory of relativity” 1949 [1]

Introduction to General Relativity

General relativity was Einstein’s theory of gravity, published in 1915 [2], which extended special relativity to take into account non-inertial frames of reference—areas that are accelerating with respect to each other. General relativity takes the form of field equations, describing the curvature of space-time and the distribution of matter throughout space-time. The effects of matter and space-time on each other are what we perceive as gravity.

The theory of the space-time continuum already existed, but under general relativity, Einstein was able to describe gravity as the bending of space-time geometry. Einstein defined a set of field equations, which represented the way that gravity behaved in response to matter in space-time. Physicists could use these field equations to represent the geometry of space-time that was at the heart of the theory of general relativity.

As Einstein developed his general theory of relativity, he had to refine the accepted notion of the space-time continuum into a more precise mathematical framework. He also introduced another principle, the principle of covariance. This principle states that the laws of physics must take the same form in all coordinate systems. In other words, all space-time coordinates are treated the same by the laws of physics—in the form of Einstein’s field equations. This is similar to the relativity principle, which states that the laws of physics are the same for all observers moving at constant speeds. In fact, after general relativity was developed, it was clear that the principles of special relativity were a special case.

Einstein’s basic principle was that no matter where you are—Toledo, Mount Everest, Jupiter, or the Andromeda galaxy—the same laws apply. This time, though, the laws were the field equations, and your motion could very definitely impact what solutions came out of the field equations. Applying the principle of covariance meant that the space-time coordinates in a gravitational field had to work exactly the same way as the space-time coordinates on a spaceship that was accelerating. If you are accelerating through empty space—where the space-time field is flat, as in the left picture of this figure—the geometry of space-time would appear to curve. This meant that if there is an object with mass generating a gravitational field, it had to curve the space-time field as well (as shown in the right picture of the figure).

Without matter, space-time is flat (left), but it curves when matter is present (right).

In other words, Einstein had succeeded in explaining the Newtonian mystery of where gravity came from! Gravity resulted from massive objects bending space-time geometry itself.
Because space-time curved, the objects moving through space would follow the “straightest” path along the curve, which explains the motion of the planets. They follow a curved path around the sun because the sun bends space-time around it.

Again, you can think of this by analogy. If you are flying by plane on Earth, you follow a path that curves around the Earth. In fact, if you take a flat map and draw a straight line between the start and end points of a trip, that would not be the shortest path to follow. The shortest path is actually the one formed by a “great circle” that you’d get if you cut the Earth directly in half, with both points along the outside of the cut. Traveling from New York City to northern Australia involves flying up along southern Canada and Alaska—nowhere close to a straight line on the flat maps we are used to.

Similarly, the planets in the solar system follow the shortest paths—those that require the least amount of energy—and that results in the motion we observe.

In 1911, Einstein had done enough work on general relativity to predict how much the light should curve in this situation, which should be visible to astronomers during an eclipse [3].

When he published his complete theory of general relativity in 1915, Einstein had corrected a couple of errors [249]. In 1919, an expedition set out to observe the deflection of light by the sun during an eclipse, in to the west African island of Principe. The expedition leader was British astronomer Arthur Eddington, a strong supporter of Einstein [3].

Eddington returned to England with the pictures he needed, and his calculations showed that the deflection of light precisely matched Einstein’s predictions. General relativity had made a prediction that matched observation.

Albert Einstein had successfully created a theory that explained the gravitational forces of the universe and had done so by applying a handful of basic principles. To the degree possible, the work had been confirmed, and most of the physics world agreed with it. Almost overnight, Einstein’s name became world famous. In 1921, Einstein traveled through the United States to a media circus that probably was not matched until the Beatlemania of the 1960s [4].

References

[1] Einstein, Albert., The Theory Of Relativity . s.l. : Citadel, 2000 [1949]. ISBN-13: 978-0806517650.

[2] Einstein, Albert., "Die Feldgleichungen der Gravitation." Sitzungsberichte der Preussischen Akademie der Wissenschaften zu Berlin. November 25, 1915, pp. 844–847. http://nausikaa2.mpiwg-berlin.mpg.de/cgi-bin/toc/toc.x. cgi?dir=6E3MAXK4&step=thumb.

[3] Einstein, Albert., "Über den Einfluß der Schwerkraft auf die Ausbreitung des Lichtes (About the influence of gravity on the propagation of light )." Annalen der Physik. 1911, Vol. 340, 10. translated "On the Influence of Gravitation on the Propagation of Light" in The collected papers of Albert Einstein. Vol. 3 : The Swiss years: writings, 1909–1911 (Princeton University Press, Princeton, NJ, 1994), Anna Beck translator.

[4] Pais, Abraham., Subtle is the Lord. The Science and the Life of Albert Einstein. s.l. : Oxford University Press, 1982. pp. 382–386. ISBN 019853907X

Using Math to Defeat the Enemy

noreply@blogger.com (Anonymous) — Thu, 14 Jul 2011 10:47:00 +0000

Preface

Many of the criticisms directed towards military simulations derive from an incorrect application of them as a predictive and analytical tool. The outcome supplied by a model relies to a greater or lesser extent on human interpretation and therefore should not be regarded as providing a ‘gospel’ truth. However, most game theorists and analysts generally understand this, it can be tempting for a layman—for example, a politician who needs to present a 'black and white' situation to his electorate—to settle on an interpretation that supports his preconceived position. Tom Clancy, in his novel Red Storm Rising, illustrated this problem when one of his characters, attempting to persuade the Soviet Politburo that the political risks of war with NATO were acceptable, used as evidence the results of a simulation carried out to model just such an event. It is revealed in the text that there were in fact three sets of results from the simulation; a best-, intermediate- and worst-case outcome. The advocate of war chose to present only the best-case outcome, thus distorting the results to support his case (Clancy, 1988).

There have been many charges over the years of computerized models being unrealistic and slanted towards a particular outcome. Critics point to the case of military contractors, seeking to sell a weapons system. For obvious reasons of cost, weapons systems (such as an air-to-air missile system for use by fighter aircraft) are modeled extensively on computers. Without testing of their own, a potential buyer must rely to a large extent on the manufacturer's own model. This might well indicate a very effective system, with a high kill probability (Pk). However, it may be the model was configured to show the weapons system under ideal conditions, and its actual operational effectiveness will be somewhat less than stated. The US Air Force quoted their AIM-9 Sidewinder missile as having a Pk of 0.98 (it will successfully destroy 98% of targets it is fired at). In operational use during the Falklands War in 1982, the British recorded its actual Pk as 0.78 (Allen T. B., 1987).

Human factors have been a constant thorn in the side of the designers of military simulations. Whereas political-military simulations are often required by their nature to grapple with what modelers refer to as "soft" problems, purely military models often seem to prefer to concentrate on hard numbers. While a warship can be regarded, from the perspective of a model, as a single entity with known parameters (speed, armor, gun power, and the like), land warfare often depends on the actions of small groups or individual soldiers where training, morale, intelligence, and personalities (leadership) come into play. For this reason, it is more taxing to model—there are many difficult-to-formulate variables. One valid criticism of some military simulations is these nebulous human factors are often ignored (partly because they are so hard to model accurately). Other perplexing issues include aggregation-disaggregation, communication networks, attrition, and end-game modeling.

“Using Math to Defeat the Enemy: Combat Modeling for Simulation” is intended to provide a foundation in the underlying combat modeling issues of military simulations. Of course, this is just a background, and a more rigorous treatment can be found in my book, Mathematical Modeling of Warfare and Combat Phenomenon (2011), Lulu.com, ISBN 978-1-4583-9255-8. Ultimately, this is a resource/reference book covering a wide gambit of military modeling issues.
This book is organized in two parts: Simulation (Part I) and Modeling (Part II). There are numerous practical applications and example models used in past and current military simulations.

This book is a result of about 25 years of use, application, research, and teaching military modeling and simulation. Much of the material in this book based on practical experience with modeling and simulation and extraction of my course notes from PowerPoint presentations.

Jeffrey S. Strickland, Ph.D.
CMSP, ASEP
President
Simulation Educators
Colorado Springs, Co
www.simulation-educators.com

Chaos Theory for Beginners (excerpt from Quantum Phaith)

noreply@blogger.com (Anonymous) — Wed, 13 Jul 2011 02:22:00 +0000

It’s a Jungle Out There [1]

"The whole history of science has been the gradual realization that events do not happen in an arbitrary manner, but that they reflect a certain underlying order, which may or may not be divinely inspired."

- Stephen Hawking

Life finds a way

Remember Jurassic Park? Handsome mathematician Doctor Ian Malcolm[2] (pictured here [6]) explaining to pretty Doctor Sattler why he thought it was unwise to have T-rexes and the likes romping around on an island? John Hammond, the annoying owner, promised that nothing could go wrong and that all precautions were taken to ensure the safety of visitors. Dr. Malcolm did not agree. "Life finds a way," he said.

Nature is highly complex, and the only prediction you can make is that she is unpredictable. The amazing unpredictability of nature is what Chaos Theory looks at. Why? Because, nature is marvelous and mysterious instead of being boring and translucent. Chaos Theory has managed to somewhat capture the beauty of the unpredictable and display it in the most awesome patterns. Nature, when looked upon with the right kind of eyes, presents herself as one of the most fabulous works of art ever wrought.

What is Chaos Theory?

Chaos Theory is a mathematical sub-discipline that studies complex systems. Examples of the complex systems that Chaos Theory helped fathom are earth's weather system, the behavior of water boiling on a stove, migratory patterns of birds, or the spread of vegetation across a continent. Chaos is everywhere, from nature's most intimate considerations to art of any kind. Chaos-based graphics show up all the time, wherever flocks of little space ships sweep across the movie screen in highly complex ways, or awesome landscapes adorn the theater of some dramatic Oscar scene. Do you remember Ace Ventura dangling from a rope over that abyss, trying to save the little raccoon? Remember all those beautiful mountains in the background? That was not a location shot. That was computer-generated Chaos art.

Complex systems are systems that contain so much motion (so many elements that move) that computers are required to calculate all the various possibilities. That is why Chaos Theory could not have emerged before the second half of the 20th century.
However, there is another reason that Chaos Theory was born so recently, and that is the ‘Quantum Mechanical Revolution’ and how it ended the ‘deterministic’ era.

Up to the Quantum Mechanical Revolution, people believed that things were directly caused by other things, that what went up had to come down, and that if only we could catch and tag every particle in the universe we could predict events from then on. Entire governments and systems of belief were (and, sadly, are still) founded on these beliefs, and when Sigmund Freud invented psychoanalysis, he headed out from the idea that malfunctions in the mind are the results of traumas suffered in the past. Regression would allow the patient to stroll down memory lane, pinpoint the sore spot and rub it away with Freud's healing techniques that were again based on linear cause and effect.

Chaos Theory however taught us that nature most often works in patterns, which are caused by the sum of many tiny pulses.

How Chaos Theory was born and why

It all started to dawn on people when in 1960 a man named Edward Lorenz [3] created a weather-model on his computer at the Massachusetts Institute of Technology. Lorentz’ weather model consisted of an extensive array of complex formulas that kicked numbers around like an old pigskin. Clouds rose and winds blew, heat scourged or cold came creeping up the breeches.

Colleagues and students marveled over the machine because it never seemed to repeat a sequence; it was really quite like the real weather. Some even hoped that Lorentz had built the ultimate weather-predictor and if one chose the input parameters identical to those of the real weather howling outside the Maclaurin Building, it could mimic earth's atmosphere and would be a precise prophet.

Then one day Lorentz decided to cheat a little bit. A while earlier, he had let the program run on certain parameters to generate a certain weather pattern and he wanted to take a better look at the outcome. Instead of letting the program run from the initial settings and calculate the outcome, Lorentz decided to start half way down the sequence by inputting the values that the computer had come up with during the earlier run.

The computer that Lorentz was working with calculated the various parameters with an accuracy of six decimals; but the printout gave these numbers with a three decimal accuracy. So, instead of inputting certain numbers (like wind, temperature and stuff like that) as accurate as the computer had them, Lorentz settled for approximations; 5.123456 became 5.123 (for instance). And that puny little inaccuracy appeared to amplify and cause the entire system to swing out of whack.

Exactly how important is all this? Well, in the case of weather systems, it is very important. Weather is the total behavior of all the molecules that make up earth's atmosphere; and in the previous chapters, we have established that a tiny particle cannot be accurately pinpointed, due to the Uncertainty Principle! Moreover, this is the sole reason why weather forecasts begin to be bogus around a day or two into the future. We cannot get an accurate fix on the present situation, just a mere approximation, and so our ideas about the weather are doomed to fall into misalignment in a matter of hours, and completely into the nebulas of fantasy within days. Nature will not let herself be predicted.

Principle (7):The Uncertainty Principle prohibits accuracy. Therefore, the initial situation of a complex system cannot be accurately determined, and the evolution of a complex system cannot be accurately predicted.

[1] "It's a Jungle out There" is a song written by Randy Newman, the theme song for the TV series Monk since its second season. In 2004, it won an Emmy Award for best theme song.

[2] Ian Malcolm is played by Jeff Goldblum (please do not tell me you have never heard of him). Jeffrey Lynn "Jeff" Goldblum (born October 22, 1952) is an American actor. His career began in the mid-1970s and since then he has appeared in major box-office successes including The Fly, Jurassic Park (two films), and Independence Day. From 2009 to mid-2010 he starred as Detective Zach Nichols on the USA Network's crime drama series Law & Order: Criminal Intent.

[3] Edward Norton Lorentz (May 23, 1917 - April 16, 2008) was an American mathematician and meteorologist, and a pioneer of chaos theory. He discovered the strange attractor notion and coined the term butterfly effect.

Exerpt from new book: "Quantum Phaith"

noreply@blogger.com (Anonymous) — Mon, 11 Jul 2011 00:14:00 +0000

"Everyone else would climb a peak by looking for a path somewhere in the mountain. [John] Nash would climb another mountain altogether and from that distant peak would shine a searchlight back onto the first peak."

-Donald Newman (A Beautiful Mind by S. Nasar, p. 12)

Introduction

I have struggled with what should come first, a discussion of biblical faith or a discussion of quantum physics (or quantum mechanics). In the end, I had to consider what came from what. Many scientists would claim that their faith—if they have any at all—evolved from their study of the science. For me it was Biblical faith that revealed the mysteries of mathematics and physics.
There is a verse in Scripture that many know or have heard (even people who have never picked up a Bible). The New American Standard translation of Luke 17:6 says:

And the Lord said, "If you had faith like a mustard seed, you would say to this mulberry tree, ‘Be uprooted and be planted in the sea’; and it would obey you”. (NASB)

I have to admit that my faith is much smaller than mustard seed, which by the way is very tiny. I have always used, not by choice, sweat, blood, and a shovel to move trees (or at least very small trees). Although I have never held a mustard seed, there in an abundance of information about them is available through the internet. I suppose Jesus had first-hand experience when he spoke about a mustard seed—I have no reason to doubt otherwise.

Κόκκος σίναπι (What would we do without mustard seeds?)

The Greek renders Jesus’ reference to a mustard seed as κόκκον σινάπεως, transliterated as kosson sinapeōs. The word Κόκκος, kokkos, is translated five times as ‘grain’ and once as ‘corn’ in the King James Version (KJV). The word σίναπι, sinapi, occurs five times in the KJV and is translated as ‘mustard seed’ in all five instances. Strong’s Lexicon says that mustard is “the name of a plant which in oriental countries grows from a very small seed and attains a height of a tree, 10 feet (3 m) and more; hence a very small quantity of a thing is likened to a mustard seed, and also a thing which grows to a remarkable size.” [Strong, J., The exhaustive concordance of the Bible (electronic ed.). Ontario : Woodside Bible Fellowship, 1996]

The Latin Vulgate renders the verse as:

"dixit autem Dominus si haberetis fidem sicut granum sinapis diceretis huic arbori moro eradicare et transplantare in mare et oboediret vobis"

I mention the Latin here since it is from the Latin word quanta that we derive the English word quantity. I also want to make it clear that the Latin did not render the Greek work for grain as quantum, rather as granum. However, Matthew 13:32 describes the mustard seed as “quod minimum quidem est omnibus seminibus”, that is, “the least indeed of all seeds”. Although I lack direct historical literary evidence that Jesus had in mind the concept of a small quantity when he speaks of a mustard seed, in this case, the context of His words speaks of a “small quantity of a thing”.

Later, we will explore the concept or meaning of biblical faith, but first let’s digress and explore the concept of small quantities of things.

Phundamentals of Physical Quantities

If you are close to my age (52), then you probably remember the television series Quantum Leap. If you are younger, you may have heard terms like quantum computing or quantum numbers. At any rate, you have probably heard the word quantum, whether you paid any attention to it or not.

In physics, a quantum (plural: quanta) is the minimum amount of any physical entity involved in an interaction. Behind this, one finds the fundamental notion that a physical property may be ‘quantized’, referred to as ‘the hypothesis of quantization’ [Wiener, N., Differential Space, Quantum Systems, and Prediction. Cambridge : The Massachusetts Institute of Technology Press, 1966]. This means that the magnitude can take on only certain discrete values, leading to the related term called “quantum number”. An example of a quantized entity is the energy transfer of elementary particles of matter (called fermions) and of photons, and other bosons. [Srednicki, Mark., Quantum Field Theory. Cambridge : Cambridge University Press, 2007. ISBN 978-0521864497]

A photon is a single quantum of light, and is referred to as a light quantum. The energy of an electron, bound to an atom (at rest), is said to be quantized, which results in the stability of atoms, and of matter in general. As incorporated into the theory of quantum mechanics, physicists regard this as part of the fundamental framework for understanding and describing nature at the infinitesimal level.

Normally quanta are considered discrete packets with energy stored in them. Max Planck (he won the Nobel Prize for his work in quantum theory) considered these quanta to be particles that can change their form (meaning that they can be absorbed and released). This phenomenon can be observed in the case of blackbody radiation, when it is being heated and cooled.

Phundamentals of Phaith

"Now faith is the assurance of things hoped for, the conviction [evidence] of things not seen."

- Hebrews 11:1 (NASB, brackets added)

"Seeing isn’t believing; believing is seeing."

- The Santa Claus

I have seen the Grand Canyon, walked around the rim, taken photos, and bought something from the myriad of gift shops. It does not take much conviction to assume it is real. Ditto with the Statue of Liberty, Monument Valley, the Saturn V rocket, Disney World, and so on. I did not witness the atomic explosion at Hiroshima (I was not born), the gunfight at the O.K. Corral, the third Crusade, the sack of Jerusalem, or the birth of Christ. However, I have seen historical evidence that all these events occurred, and I do not need conviction to believe they did.

I have not seen (direct) evidence of the eternal life that Christ promised to believers. I’ve heard about it, read about it, watched movies about it, but I have not seen it. I have had parents and in-laws die. They had known Christ in a personal relationship, but I did not see them ascend to heaven. In fact, the last time I saw them was in an urn or in the ground. Yet, I am convicted that they are living eternity with Christ—it is something I have hoped for them and for myself someday. I am assured of this by faith. I stated at the outset (in the preface) that belief is something we get from others, while faith is something we develop for ourselves. That is not entirely accurate. “So faith comes by hearing, and hearing by the word of Christ.” (Romans 10:17, NASB)

So, ‘seeing’ is not involved in developing faith. Rather, it is directly linked to ‘hearing’ the Word of God. Now, I submit to you that the most intellectual people in the world might read the Word of God, and never be moved by it, while a child might hear it—read it with the Spirit speaking to their soul—and have faith.

I am not quite sure I agree precisely with the elf in The Santa Claus, but too often, we look for evidence before we believe. For me, I had to believe before I could see the evidence. It was there right before me for years. I read it, I studied it, and I dissected it. Nevertheless, not until that ‘quantum leap’ of faith came about did I see it. And quantum leap it was—just a tiny little bit, like a mustard seed. I did not have an encounter with death. I did not have any traumatic experience that drove me to faith. In some sense, I just stopped trying to understand on my own. You might call it a Holy Spirit encounter, but I don’t even remember feeling it. One moment it was not there, the next moment it was. I had merely said, “Lord, I give up. I can’t do it on my own.” Voilà, faith arrived. Later we will visit Principle 10c: “Truth can only be noticed when the private perspective is doubted”. There was just too much of me in the way of my understanding. I was looking for conviction in something I could see (the words), but missing the Word that was trying to assure me.

As I recently heard:

“Intimacy with God determines your clarity from God.”

- Pastor Daniel Rolfe [1]

2 Corinthians 4:1-6 says:

"1 Therefore, since through God’s mercy we have this ministry, we do not lose heart. 2 Rather, we have renounced secret and shameful ways; we do not use deception, nor do we distort the word of God. On the contrary, by setting forth the truth plainly we commend ourselves to everyone’s conscience in the sight of God. 3 And even if our gospel is veiled, it is veiled to those who are perishing. 4 The god of this age has blinded the minds of unbelievers, so that they cannot see the light of the gospel that displays the glory of Christ, who is the image of God. 5 For what we preach is not ourselves, but Jesus Christ as Lord, and ourselves as your servants for Jesus’ sake. 6 For God, who said, “Let light shine out of darkness,” made his light shine in our hearts to give us the light of the knowledge of God’s glory displayed in the face of Christ. (NIV)

Well now, if God’s Word—Christ—is not believable to you, then you have no hope in understanding it, no matter how educated you have become. If His word is not clear to you, then perhaps you are not intimate enough with Him!"

[1] Daniel Rolfe is a senior pastor at Mountain Springs Church in Colorado Springs, CO.

http://www.lulu.com/product/hardcover/quantum-phaith/16231128

2nd Annual Modeling & Simulation Summit

noreply@blogger.com (Anonymous) — Fri, 08 Jul 2011 03:47:00 +0000

August 29 - 31, 2011, Venue to be Confirmed, Orlando, FL

IDGA’s 2nd Annual Modeling & Simulation Summit will provide the most up-to-date news on the latest advancements in technologies and the lessons learned from recent efforts. This event will take a closer look at military strategies for M&S including Irregular Warfare and Counter IED training. Due to increasing challenges, acquisition decisions are currently being made, as well as new requirements for industry.

I will be presenting a focus session on Monday, August 29th: 10:00 - 12:00 Latest Models for Complex Combat Simulation

Using Math to Defeat the Enemy: Combat Modeling for Simulation

Combat Modeling for Simulation presents mathematical and heuristic models of combat phenomena, including environmental effects, movement, attrition, detection and communication. Concrete examples will be used to relate the latest techniques and tools of combat modeling. The specific application of missile modeling is described in detail.

What will be covered:

Simulation scenario development: the elements of a scenario and how to develop scenarios

Environmental modeling examples and challenges

Physical modeling including movement models, sensing and detection models, shooter models and communicate

Missile modeling: dynamics of missile flight, challenges of 3-, 5-, and 6-DOF modeling, and example missile model in MATLAB® and Simulink®

How you will benefit:

Receive up-to-date information on models underlying various combat training and analysis simulations

Receive concrete examples of combat modeling situations and uses

Get exposure to the latest tools and techniques for combat modeling