The Stats Geek

Top 10 Commandments of Statistical Inference: #3

noreply@blogger.com (J Buser) — Wed, 19 Mar 2008 18:36:00 +0000

We are currently following the 10 commandments of Statistical Inference. The 3rd commandment, is that:

Thou shalt not make statistical inference with the absence of a model

The 4th commandment was to honor the assumptions of your model…as we discussed why that is important, however some people go even further down the road of insanity and not only “misplace” the assumptions, but misplace the model itself. Situations in which one wishes to infer statistical inference calls for the use of a model. This ensures that you are “following the rules” of the prescribed model. For some reason I am amazed at the backlash this sometimes gets. The “why can’t you just give me an answer” or “why do you need to make it so complicated” complaints. In our society of “quick hit answers” a lot of times, there is this thought that there is no time to set up the proper model. I used to work for this person who said “it is better to ask forgiveness, than permission” and forced the collection and analysis of data beyond the ability to model the data correctly. What happened? Well, sure, we got an answer, and he went on his merry little way…only to have to come back and do the exact same experiment because the results of the first one proved to be invalid. The result was lost time, money, and resources. If the design was set-up correctly in the beginning, it would have taken 3 days to run. He wanted it in 1. He got it in 1, and then spent months trying to backtrack to get to the answer he would have gotten in 3 days…

So, the next time someone tells you, I can model it and it will take x amount of time, you have every right to ask why, but make sure your push for a faster initial result doesn’t cause long term implications.

Top 10 Commandments of Statistical Inference: #4

noreply@blogger.com (J Buser) — Mon, 10 Mar 2008 17:39:00 +0000

Now, we are really cooking with the Ten Commandments of Statistical Inference. A long time ago, I wrote an entry that talked about how one should really look and can vs. should. The 4th commandment really speaks to that point:

Thou Shalt honor the assumptions of thy model!

From the engineer who chooses to calculate a performance index on an unstable process to the marketer who uses a t-test when there is a correlation between their samples, I find this to be the most commonly broken commandment. Unfortunately, I also think it is one of the most dangerous that we have talked about in this series. All models have assumptions, and it is important to make sure that you satisfy these assumptions. Otherwise, your results are suspect at best, down-right worthless most of the time. Usually, at this point I get the argument “but mathematically, I can calculate it.” Sure, mathematically you CAN calculate anything. But theoretically, should you?

So, why is this so common? Because it is a misunderstanding of what the assumptions are in the first place. Compound that with the advent of many software programs out there that make it easier and more user-friendly to calculate results. Now, don’t get me wrong, this is not a bad thing. I myself do not want to go back to hand calculations or Excel formulas. However, if you have never seen the formula, never spent the time understanding the assumptions, then you may not have a grasp what you are doing is correct. Sure, plug in numbers and you will get an answer, but is it right?

How important is this to get right? Let’s put it this way, if a Doctor diagnoses you wrong, but does everything else right according to his diagnosis, is he right? No, no way we would let him get away with it. So, why do we let it pass in statistics?

Top 10 Commandments of Statistical Inference: #5

noreply@blogger.com (J Buser) — Fri, 07 Mar 2008 20:04:00 +0000

Well, I am glad on the last post was well received, so now I think it is safe for me to start counting down the last 5 commandments of Statistical Inference. Number five:

Thou shalt not adulterate they model to obtain statistical significance.

Now, when you first look at this you think…Adulterate? But it does make sense. It comes down to some of our previous discussions, and that is make sure you do not knowingly (or unknowingly) allow extraneous variables or inferior ingredients into model. Make sure you take steps to control for the things you can control for. Sometimes, it is as easy as excluding certain people, or certain parameters. Other times you may have to really think practically about what CAN affect your model and control for that. In the marketing world it might be to control the time of day when your sends go out. In an experimental design in the lab you may want to add test at separate times and add a blocking variable. Use common sense and your own knowledge of the situation. This is not a “math” problem per-se. Of course there are some techniques available to help, but this typically requires you to sit down and map out your process and brainstorm about all the things that can affect your design and control the heck out of them when you can. This is always my favorite part of designing experiments. It’s when you can be creative. Next post we will discuss about how to understand and fit your situation into a needed model (rather than the other way around!).

Statistical Humor Gone Awry!

noreply@blogger.com (J Buser) — Wed, 05 Mar 2008 19:53:00 +0000

So, we have gone through the first 5 commandments of Statistical Inference. I am going to move to the next 5 in the next few posts, but before I do it has come to my attention that by presenting these, I have gotten myself into a little bit of a dilemma. My attempt at statistics humor may have gone awry!

I get the feeling that people wondering when it is ok to do statistical analysis, or is it EVER correct to do statistical analysis. Worst yet, I think statistics can only be done in a laboratory with white coats!

First and foremost let me assure you I do believe whole heartedly in statistical analysis ;-) If I didn’t, well, I wouldn’t have worked for the last decade plus in the arena. Secondly, I am not the anti-layman stats guy in the ivory tower throwing pennies and guessing the probability of it hitting someone. Far be it, I am actually a trained psychologist, not a statistics major so I whole heartedly believe that a statistician can encompass people who are not “classically” trained but apply statistics to solve problems. Lastly, the 10 commandments are a tongue in cheek attempt at humor that some of us stats geeks really enjoy. Sadly, when I first received “the list” from one of my co-workers, all stats geeks I was working with at the time stopped whatever they were doing and ran the Xerox copier out of paper.

Bottom line, I think anyone can perform valid statistical testing, but it must be valid, and must follow the rules of what the models were designed for. If you can do this, you will have a wonderful design and results, if not, you are going to find yourself into a mess and you won’t really know why! If you need help, find someone who does understand all the nuances that’s why they are there!

Top 10 Commandments of Statistical Inference: #6

noreply@blogger.com (J Buser) — Fri, 29 Feb 2008 16:37:00 +0000

So, the 6th commandment of Statistical Inference is:

Thy shalt not covet thy Colleague’s data.

This sounds like a pretty easy one, but I am amazed to see even now people who view other people’s data and “want what they have.” Why is this so bad? This can drive people to reach beyond what they should in an effort to find “statistical significance” to keep up with the Joneses. How do they do this? Maybe when the numbers don’t match-up they use a less stringent model. Perhaps they refuse to use an adjustment when the situation calls for it. This leads them to travel in and out of the grey that is statistics. These tactics are egregious enough, but then there are those that understand statistics even less and wonder why their data doesn’t look like someone else’s. Many times I have been asked, and sometimes almost blamed or considered a bad statistician, if the data doesn’t look like someone else’s. In some limited fashions I have been prodded to make it look more favorable. This was refused much to the persons chagrin as they did not understand that it was much more than just data integrity on the line.

Bottom line, data is data. It can be made into anything that you want, but only those that truly understand it and use it correctly will learn and help improve the situation. Wishing to have the results of others is not an issue in itself; if it helps you drive improvement towards that goal. It’s when it drives you to look the other way in the analysis where it causes problems.

Top 10 Commandments of Statistical Inference: #7

noreply@blogger.com (J Buser) — Wed, 27 Feb 2008 15:42:00 +0000

The seventh commandment of Statistical Inference is:

Though Shalt not bear false witness against they control group.

To understand this commandment, first you must know what a control group. Typically, when doing a scientific study, one has looks at the difference between two groups that are statistically the same. The first group is the experimental group in which they receive the treatment, the second; the control group, is the group which does not receive the treatment (e.g., placebo). So, how can someone bear false witness (lie) about the control group? The first, and most common is not making sure the control group is statistically the same from the experimental group. By not sampling properly, and then saying that they are the “same” as the experimental group when they are not, this could affect your results. Also, you could have introduced confounding variables into the mix by not controlling both groups properly. You know have no idea if what you see in the experiment is happening because of the treatment, or because of variables you did not control for. Finally, all people can be biased, even researchers. They want to prove their theory. If the experiment is testing in anyway the effectiveness of a treatment, they may have a tendency to favor the experimental group unknowingly. This is why researchers do “double-blind’ studies in which neither the test group nor the researcher knows who belongs in which group.

Not all testing is strict scientific testing, but make sure that you are sampling the correct groups, controlling for the correct variables, and not allowing biases to enter into the results.

Top 10 Commandments of Statistical Inference: #8

noreply@blogger.com (J Buser) — Tue, 26 Feb 2008 16:30:00 +0000

In our last few posts, we talked about two of the top 10 commandments of statistical inference. We learned that: Though shalt not infer causal relationships from statistical inference and though shalt not apply large sample approximation in vain. Today, for the number 8 commandment:

Though shalt not worship the 0.05 significance level!

This is one I run into all the time. It goes with my earlier post on “Statistical Inference Is Not A License.’ For some reason, a common mistake is to focus solely on a 0.05 significance level and I think it is because of a couple of reasons. First, it is what is taught in schools for the most part. I remember back to my first few stats classes and it seems like it was best case to just teach “Above or Below 0.05.” It’s also the misunderstanding and overreaching that we tend to do when we have statistical significance. Finally, of course, there can be statistical reasons, but when I ask people why the 0.05 level is so important to them, few understand it this deeply.

The main thing to remember is what it really means. It means that you are 95% confident depending upon your models assumptions and are willing to take that 5% risk that there is no difference (Type I Error). So, the real question is, how important is this to you? If you are in a lab, then a 0.01 significance might be what you shoot for. In a sociological study, perhaps maybe a 0.10-0.15 level will satisfy you. Whatever the case, it will depend on how strict you wish to be or not be and what situation you are in.

Top 10 Commandments of Statistical Inference: #9

noreply@blogger.com (J Buser) — Mon, 25 Feb 2008 17:09:00 +0000

OK, so last post we discussed the first rule (or, number 10) of statistical inference. That was, Thou shalt not infer causal relationships from statistical significance. Number 9 is:

Thou shalt not apply large sample approximation in vain.

This is more of a tricky concept. For the most part, the larger the sample size the closer to the population you are. Right? Well, based off of that, the closer one is to the sample representing the population. Most statistical models are based off of this assumption, and therefore the larger sample size you have the “easier” it becomes to find statistical significance. Even a first year stats student begins to be able to point this out. Just look at the back of any statistical book and look at the t-distribution and watch what happens to the t-value needed to find significance. It goes down….

In fact, I remember one of my interview questions for my first job was:

You have one sample with a correlation value of .30 and no significance and another value of .28 and statistical significance. Why would a smaller value have significance?

Because significance has little to do with strength and a larger sample size can help “find significance.”

In other words, don’t use large sample sizes just to find significance. It is important to take the correct sample size for the statistical model you are using. Each model is different and if you don’t know the assumptions of the model, you should search out and ask of the ramifications of a large sample size. In other words, know thy model!

Top 10 Commandments of Statistical Inference: #10

noreply@blogger.com (J Buser) — Thu, 21 Feb 2008 21:03:00 +0000

All,

I am starting a new series on the Top 10 Commandments of Statistical Inference. In my first job as a statistical consultant I was given this and it’s been hanging on my wall ever since. I wish I could claim it was mine, but I can’t. For today:

Number 10: Thou shalt not infer causal relationships from statistical significance.

This seems like it should be the number 1, but it isn’t. This goes back to my last post. All too many times, we see statistical significance and we use this as a license to do what we want, or saying that x caused y. Bottom line, there are very few, if any, situations in which you can infer a causal relationship from finding significance. Only if you are able to control for every outside variable, and are able to directly manipulate the variables you are testing, could you indicate causality. Of course this is next to impossible. Therefore, as mentioned before, you can only state “Based off of what I know, I can indicate that I am X% confident that what I found in the study, I would find in the general population.”

Statistical Significance Is Not A License

noreply@blogger.com (J Buser) — Wed, 20 Feb 2008 22:18:00 +0000

Please go check out Avinash Kaushik’s blog about statistical significance. I found his blog very helpful and in the entry I like the fact that he begins to discuss how we must use statistics when testing our assumptions. He also points to Brian Teasley’s stats calculator. I pulled this down and tried to find the assumption underneath. I am contacting both, to see if I can get those. I will let you know my thoughts on those.

However, one concern I have, is that it brought up an all too familiar ring to my ear. I am increasingly seeing “Statistical Significance” become a license to do what we want. I want to remind everyone, what statistical significance really means. Simply put, in most cases that we are dealing with, statistical significance indicates that you are x% confident that what you found in your testing or sample, you would find in the general population. In the case of an A/B test, it simply tells you that I am x% confident that there is a difference between A and B and what I found in my testing, I would find in population. That is ALL that it tells you. Furthermore, it is contingent upon you doing the right test the right way in the first place. So, even if you have statistical significance, does not mean what you found was really right. Wrong assumptions, wrong manipulations, and wrong sampling are the issues I find the most often. The sampling piece can be can be the most problematic. You could do a test in one month and find results, and do the same test the next month and find widely different results…both being significant! What went wrong? You probably do not have your assumptions or sampling down pat. Make sure you do that before you test. Otherwise, “statistical significance” can change from the license to do what you want to a pink slip!

Again, I will let you know what I find out about the calculator!

Synergy

noreply@blogger.com (J Buser) — Fri, 08 Feb 2008 20:47:00 +0000

In business, most of the time, it doesn’t happen. People tend to work on their own area and own sides leaving two or more extremely strong wheels to spin on their own.

As an analyst who was initially trained in a “conservative” setting, I find that it is sometimes hard to create synergy with those who do not think in tight theoretical ways. When I first started my career, I was pretty idealistic and very conservative with data. All things had to be “balanced.” This made it extremely difficult to create synergy with those who do not have to have things balanced. As I moved on with my career, I quickly realized that more flexibility was needed on my side. This was accelerated when I began working for a photomask company as the corporate statistician. You see, photomask manufacturing is pretty much N-of-1 manufacturing, and as you know statistics in manufacturing is all about replication. So, I quickly found out the key was not to focus on what was different and how to “fit” a statistical model to non-replicates, but to find out really what was the same, what WAS replicated, and control the heck out of that. It worked well, and led to my first two publications. However, I never really was part of a synergy, since my main focus was to pound out reports and focus on how to analyze the same data differently.

I then moved to my current position, and marketing was a new area of focus for me. It was much more fluid than the “lab” of a research facility. What I did realize, however, is that there was some synergy happening within the company. People were working together, not in silos. Lately, we are really starting to pick-up momentum, and it is very exciting. Things are “coming together” in a way few analysts actually get to see. Most of us typically sit back and pound reports and think of new ways to get and analyze data. What is really exciting though is when the metrics begin to line up with the corporate identity. That is a great feeling for an analyst.

So, what’s my point? To create synergy, everyone needs to change and it takes time. It took me years and different situations, to change from a “theoretical” statistician and come closer to the middle. Yet, it can’t just be one person. It will not work if one person moves all the way to the other person. Others also have to come towards the middle. When it does happen though, it can be a very exciting time for everyone.

More Polls

noreply@blogger.com (J Buser) — Mon, 04 Feb 2008 21:37:00 +0000

OK, this primary on the democratic side is going to be a wild one (at least in the news). We have another “swing” with a poll. The CNN/Opinion Research Poll that was reported today has indicated that Obama has now “erased” a gap between himself and Clinton with one day to go. Compare this to a few months ago when Clinton had a “significant” lead over Obama. Meaning? Not much.

Here’s why. Although I appreciate that they state the current poll has a 4.5 point error rate and do not say he is now in the lead, it means little to nothing as to how things will shake out tomorrow. This is a national survey. So, they are again sampling from a population who may not even live in a state that will vote tomorrow, and even if he or she does, who knows if they will even vote in a primary. So, it means nothing to the amount of delegates that Obama or Clinton could pick up. But it sure makes for great headlines, which is the scary part. By dissecting each poll and showing these “wild swings” the media is creating news, not reporting it. For the casual observer, if they see this, they may decide to hitch themselves on the wagon of the winner and it may have a small effect on the outcome tomorrow.

Maybe more interesting is the vote in California, who is voting tomorrow. There was a large fluctuation between two weeks ago when Clinton had a double-digit lead to a poll on Sunday that shows an insignificant lead for Clinton (within the 4.5 points). How interesting is this? Not as interesting as they want it to seem. Could it be Oprah? Could it be Maria Shriver? Or could it just be bad sampling. I am still on point to say that a poll should not fluctuate this much within a two week period if the sampling is right (whether the premise, delivery or results of the poll is right or not.) Bottom line, anytime you are sampling from the same population there should not be such a fluctuation, even if you are asking the wrong thing. In my work, the first thing I look to when I see something like we see here is, “Did I get my sampling right?” “Am I asking the same questions from the same population?” In the case of these polls, I say probably not.

So, sorry, maybe Oprah isn’t responsible for such a wild swing after all. Who could be? Hmmm, didn’t Edwards just drop out in the last two weeks? A point that is lost on them I suppose…

Primary Polls

noreply@blogger.com (J Buser) — Fri, 01 Feb 2008 19:46:00 +0000

So, why have political polls been so wrong? I get asked this from time to time. Well, it is a complicated answer. First, we must address which polls have the problems. The first type of polls, (the ones that seem so wrong) are the pre-voting polls that are taken weeks or days prior to polling. The second type, the exit polls, is taken directly after the voter has voted. Obviously, this one is much more accurate (although the last two elections, even these are failing much more frequently.)

Let’s focus on the pre-voting polls. These polls are taken months, weeks, and days before the vote. To keep things simple, people from a particular demographic are sampled and polled about who they would vote for in the upcoming primary. In the recent primaries, they have been WAY off. Why? Well, it can be quite complicated, but I think there are several things at play. First, the models are broken. Many people are still living in a world where they think that the old social model is still in existence. This is not true. No longer can we typecast people according to strict demographics. Where you could once count on a particular demographic to react or vote one way, you can no longer do so. Why? People have so much more information at their finger types due to technology. In years past, people would get their information from regional and perhaps one national news source and they could be swayed easier since they only got a couple of views. Now, people are inundated with news 24 hours a day 7 days a week. They also no longer have to count on social networking with people in their vicinity, but rather can converse with people from all across the world who actually hold many of their views, creating micro-groups of people with the same thoughts. One-person on an island no longer exists. In other words, the old models are no longer accurate. This leads to the second issue which is sampling. If the models are broken, surely the sampling is as well. When you rely on asking a few people to predict the whole, you must have the correct samples in place. Because of what was stated above, undoubtedly the samples are wrong. How can you tell? Look and see how fast the same polls are changing from week to week or in some cases day to day. One polling center can have wild swings. This is no fluke. If your sample is not accurate, this can happen anytime you are attempting to predict. Furthermore, when polling a primary, you may be asking people who have no plan on voting in the primary. Another reason for the wild swings? Because of the information explosion, people tend to change their mind much quicker than before. We are a society of instant news and change, which makes it that much harder to predict.

So, what to look for with Super Tuesday coming up? Well, certainly do not look too far into the polls to tell you what is going to happen. Only way to be for sure on who will come out ahead is by watching the actual results come in.

Why Statisticians Shouldn't Watch Sports

noreply@blogger.com (J Buser) — Wed, 14 Nov 2007 20:05:00 +0000

Check out this link. I got a kick out of this. First of, I really liked his reasoning. For the most part, he controlled for all the variables he needed to control for and made things simple yet elegant (I assume when he controlled for defensive points, he also controlled for only yards allowed by Defense.). Secondly, I understand this man's pain. Can't even watch a game without trying to analyze some mundane fact that only other people like him would like, which in turn causes my wife soem pain as well, having to hear it.

So, for all of you people out there that want to go check out if their team is a "bend but don't break" defense, one bit of warning. He focused on the Pac-10. So, what would be interesting to know if his ratio would stay the same in different conferences. I would assume possibly not. If not, then to have an accurate ratio, you may need to focus on each conference and then within division I football.

Demming

noreply@blogger.com (J Buser) — Fri, 02 Nov 2007 21:22:00 +0000

Earlier today I sat through a meeting in which the presenter mentioned Deming. Wow, this was the first time I have heard Deming in about 2 years, or the time period that I have spent in Marketing. Of course my interest was peaked. The presenter went on to explain how important it was to experiment every day. I totally agree with this. However, I was a little disappointed that he never spoke about how to ensure you adequately measure those results. I am giving him a pass though, as I assume it was the audience he was speaking to. Regardless, I think this is an extremely important point. Experiment all you want in this world. Tweak things and be curious...but always make sure you can adequately measure your results of the test. If not, then you have no idea what "experiment" really worked. To do this, you need to make sure your data is accurate and accessible; and that you have control of the variables. If not, you can test all you want, but you will have little understanding as to whether your manipulation effected your metrics, or something else effected them.

Email Diva? hmm, stick with email

noreply@blogger.com (J Buser) — Wed, 17 Oct 2007 21:40:00 +0000

I was just handed an article written by the Email Diva (it was hard copy, otherwise I would have the link). Summarizing, the author stated that since there was no standardization in email metrics (citing EEC Whitepaper), one should seek out benchmarks from their Email Service Provider, or Marketing Sherpa, which is close to apples to apples. However, because of non-standardization and other issues, “Comparing your results to industry standards will never tell you whether the effort is worthwhile for your company….The only standard is: did I make money/was I able to acquire new customers at an acceptable cost?”

Well, in regard to the standardization issue, I whole-heartedly agree that currently there is a problem. Going to your provider is a great option. Indices such as The Bulldog Index ensure everything is calculated and treated the same. However, I do not agree that the Marketing Sherpa Guide is close to apples to apples. I think it is a good guide, and important, but by its very nature, it is a survey, and therefore wrought with non-standardization. Again, another reason for the indexes like the Bulldog Index!

The main concern I have is the statement of no reason to use Industry standards (even when standardized). The only standard is money? What IS an acceptable cost? An Industry benchmark helps you decide what your standard SHOULD be and helps you compare yourself to competitors. If your CPL is $35.00 one month and $33.00 the next, great, you improved, but if your industry average is $25.00 you have a lot of work to do, and your standard needs to be improved, otherwise you are losing out to your competitors. The landscape changes dramatically if your industry CPL standard is $45.00. They you can make a decision of, continual improvement on what you are currently doing, or taking resources and going after something else.

Right is Right

noreply@blogger.com (J Buser) — Fri, 12 Oct 2007 16:37:00 +0000

An individual once came into my office after a particularly upbeat argument about statistics and wrote on my board “Right is Right.” I kept that on my board until I moved out of the office. For a while, I thought, yes, I must be that theorist, because I am right, I know theory. It is my job to remain strong in theory. I am not so sure about that anymore.

Basically, sometimes you will have a person on one side trying to argue that conservative, statistical route and the other side; you have a person explaining that you are thinking too analytical and need to focus on the overall goal. In the end, both are right and both are needed. You need a “theory guy” to ensure that what is being done is following the correct models and assumptions. However, a lot of time that theorist can be too involved in theory, and not involved in enough of delivery. That’s where the other person comes into play. The “Strategy” guy. While it is the theorists’ job to bring the right assumptions to the table, it is the strategists’ job to bring the theorist more into the real world. If this can be done well, it can be a great synergy.

Look, it is about being right, statistically. Because if it is not, then no model will work. But it is also about delivery and getting things done. Sometimes you don’t have the correct data to make the perfect model, and if you wait too long, you and your company looses out. It is the organization that has a good synergy between the two that will be the most successful!

Data from Multiple Sources

noreply@blogger.com (J Buser) — Wed, 03 Oct 2007 21:19:00 +0000

I was just working on a presentation for Innotech next week and I got to thinking about something that has always concerned me. Technology has afforded us the ability to capture droves of data, and has also given us a lot of more user-friendly software in which to analyze the data. Some of this is really good for us, and some of it is bad for us. In the wrong hands, bad data and assumptions can bring a company to it's knees pretty quickly. In order to choose the correct model, one must know what the model assumption are. I have seen many analyses completed on data that do not follow the correct assumptions. Unfortunately, software now available compounds this. In the good old days, one really had to at least understand the make-up of data in order to run an analysis. It wouldn't stop anyone from doing the wrong thing, but it was a decent barrier. Now, one can just push and pull data through systems without knowing too much if what they are doing is really right or not. Some software systems have developed barriers, but this still does not stop some weird things (I was once asked why a software system would not allow him to do a multiple regression with over 500 variables!). Am I saying everyone needs to be a statistician. Well, no.... But what I am saying is, if you don't know some of the basics, beware of your results.

Probability of Eye Injuries

noreply@blogger.com (J Buser) — Tue, 02 Oct 2007 21:36:00 +0000

Wow,

OK, sorry everyone. I am back. I had a little mishap. I was running and I actually got stung in the eye with a wasp. Now, I was thinking, what is the probability of THAT happening. So I tried to do a little research on wasp stings and the likelihood of getting stung in the eye. Unfortunately, there is little information out there that can be used. I did find that there was about 9K fireworks related mishaps a year, and 30% of these affecting the eye. Hm, little curious to find out where in the country this happens the most! I also found out that there were about 42,286 work related injuries to the face in 2002 and 70% involved the eye. Dang! Unless I was working with bee keepers, that won't help me.

Do you ever feel like this when trying to calculate what seems to be a simple issue? You can't find the correct data, and you end up chasing the wrong information. Sometimes, in the case of the wasp sting, you may just have to cut your loses and try another day. Otherwise you can reach too hard for honey which turns out to be just jam.

A Kilo no Longer a Kilo

noreply@blogger.com (J Buser) — Thu, 20 Sep 2007 13:59:00 +0000

http://www.cnn.com/2007/TECH/science/09/12/shrinking.kilogram.ap/index.html

I read this article this morning, and my first thought was, wow, what are the drug kingpins going to do now! They are getting jipped!

Actually, my mind immediately came to the thought of making sure that you have good artifacts in which to measure and control off of. It also came to mind that people unlike myself who muse about weird stuff (e.g. regular people) may think...so what? That's what they get for using the metric system! So this piece of metal is losing weight (and perhaps, if a piece of metal can do it, so can I!).

Well, it is much more complicated than that. Most people know that there are universal standards throughout our world. Such as a Kilogram. Because of variance a kilo is never a kilo. So we need to make sure that we all trace back to that artifact. Here in the U.S. we tend to use NIST http://www.nist.gov/, which is the National Institute of Standards and Technology. The idea is that all things are traceable back to a standard and although they never measure or weigh the exact same (due to variance), they are within statistically calculated specifications. Now, if the artifact is degrading, then imagine how hard it is to hit a moving target! 50 Micrograms sounds small, but it is dependent upon the distribution. It could reek havoc. Maybe we will just have to go back to "stones."

Talk like a Pirate

noreply@blogger.com (J Buser) — Wed, 19 Sep 2007 14:48:00 +0000

Ahoy Maite's,

Today is the official talk like a pirate day....For those of you who are interested, go check out what this really is http://www.talklikeapirate.com/

I was thinking about writing like a pirate but when I tried, I realized how brutal this would be. I would like you to check out Seth Godin's blog today http://sethgodin.typepad.com/. He makes some good points, unfortunatly he falls a little short so I want to clear things up.

He states that you should be focusing in on your real distribution when looking at web traffic (or vists to McDonalds). This is true. That you should not focus on Mean, but Median as well. Again, another salient point. However, in the example he gives you, median would have the possibility of not giving you the full story. Media is the middle value. So, let's say you had 4 visitors. if these visitors came to the site 1,1,9, and 10 times, your mean would be 5.25 and your median would be 5. Not much of a differnce...However, your mode would be 1! This may be important to say my although I sometimes have some high number of visits, my most frequently occuring is 1.

Of course different scenarios call for different measures of central tendancy. So yes, he is correct, make sure you measure more than mean, but if you are going to go in the right direction, make sure you go all the way!

The Roe Effect: How NOT to perform a Study

noreply@blogger.com (J Buser) — Tue, 18 Sep 2007 14:20:00 +0000

Wow. Please read http://www.opinionjournal.com/extra/?id=110005277.

This is an awful study and a great example on how one can twist numbers into a "good story." There are so many things wrong with it, I do not know where to begin. I am only going to point out a few... In fact, this would probably be a study I would hand to students to dissect and tell me what they think is wrong. Their issues are typical of bad statistical analysis. You could point out their mathematical flaws for eons. However, their very construct is wrong. They assume a cause and affect relationship.

1) They mention that children "tend" to absorb ideals of their parents...yet they analyze as it is a cause and effect relationship...that they WILL absorb ideals of parents.
2) N=1. "Hey, I know this guy who thought everyone was like their parents so they MUST all share the same political views"
3) Wow, they really cleaned up the issue that always arises of getting the right demographics by asking people if they "knew of anyone..." What a great and cheap way to control for the demographics factor. I wish I had thought of this in the past. Would have saved me oodles of time! Then, again, they assumed cause and affect saying well, those that answered yes must have had the same political leanings so that MUST mean those who they answered yes about had the same political views...they even go so far as to say that 1/3 of liberals are having more abortions...hmm, based off of..."I know someone?"
4) The "significant difference" badge of honor. Love that one...since there was a "significant difference" it must be true...even though everything they did was wrong before that.
5) Finally, they also assume those with abortions will even vote. They did not even take into account the voting percentage of those likely to vote!

Bottom line...asking people if they know someone that had an abortion, and assuming that the person who did have an abortion followed the same political leanings, and then again assuming that if so, these aborted individuals would B) Most definitely Vote and B) Most definitely have the same political leanings is one of the biggest stretches I have seen in a very long time.

Data Mining and Web 2.0

noreply@blogger.com (J Buser) — Mon, 17 Sep 2007 16:55:00 +0000

My first job out of grad school in 1998 was for a statistical software company. I enjoyed it greatly and found myself working with General Linear Models and Neural Networks quite a lot. Around that time, there seemed to be a big push for “data mining” solutions. Not that these did not exist already, but a lot of people started throwing their hats into the ring. It always bothered me (and many others) that statistical analysis was starting to be taken…lightly (or so it seemed). There was no real clear cut definition but ‘everyone was doing it!’ I once interviewed for a data mining position in which after about 30 minutes, I stopped the interview and asked them to define exactly what they thought data mining was….the admission? They had no clue, but knew they needed someone!

In 2001 I jumped over to the manufacturing world and began working with Statistical Process Control. In 2006 I re-entered the world of more complicated analysis. In that time, what had been a few “data mining solutions” has exploded. I did a very quick search this A.M. and by eyeballing it, I came up with 30-40 different vendors. Search each one of these vendors and you can find a large amount of case studies, each touting grand success stories.

What does this mean? Well, it means that a large market exists for these packages (obviously). A market much larger than the amount of skilled analysts (note, I did not say statisticians, because you don’t HAVE to be a statistician, but you must be skilled!) available. It also means that a lot of these packages lack the capabilities to do proper data mining. Combine these together and you have people who lack the skills to do proper analysis, with tools that lack the capabilities. For each success out there, I wonder about how many very expensive failures there are…

So, what does this have in common with Web 2.0? I tend to think Web 2.0 is following a similar path. There is a huge amount of buzz and a lot of people trying to get into the fray…but ask for a definition and good luck! Of course, just like data mining there will be a good amount of success, but with that I wonder how many failures there will be as well…

Can I..Sure...But SHOULD I?

noreply@blogger.com (J Buser) — Fri, 14 Sep 2007 20:43:00 +0000

I can't tell you how many times I get asked by people...can you do this analysis? This question always makes me laugh. When I was in 2nd grade I was browbeat by a very large, intimidating teacher on the difference between can, may, and should.

ME: "Mrs. Brown, can I go to the bathroom"
Mrs. Brown: "I don't know, can you?"
ME: "Mrs. Brown, MAY I got to the bathroom?
Mrs. Brown: "Yes you may!"

This happens all over the country in classrooms, learning the difference between can, may and should. Yet, it amazes me how many don't remember these lessons. I would think they would have considering if you didn't, there was no way you were going to the bathroom, except maybe right there in the room. I say this because I still always get the "Can you do the analysis?" Sure, of course I can...but SHOULD I? Many time, I find myself arguing this very point. They mistaken my answer of "Should not do it" to mean, "Can't do it." Then wonder why they hired a stats guy that can't do math....well, let's get this straight...yes, I CAN do it. I can average 2 numbers together...but SHOULD I average two numbers together? That's a whole different question. Several years ago, I would get into this argument with someone over me, and over and over again it was the same argument, same result. He would argue that I could do the calculation, I would argue that I should not do it (to the point I would give references), and eventually would have to do it anyway. You see, it was never a question of can't....but should.

I see this happen ever more so with the advent of all these new, slick stats packages "made easy." Check my next post for more details on this peice. Needless to say, with these packages...can just about anyone do complicated stats? Yes....Should they? Now that's a whole different discussion!

Smoking Dieters

noreply@blogger.com (J Buser) — Thu, 13 Sep 2007 20:13:00 +0000

http://news.yahoo.com/s/nm/20070913/hl_nm/diets_smokers_dc

Really, this is what they thought they found? That female teenagers who initiate dieting appear at risk for beginning regular smoking?

Hmmm, 21% of the girls were actually overweight but yet 55% were dieters....perhaps, could there be something underlining this? Like perhaps maybe image issues? Did they look at this? Does not look like it to me. Perhaps it was their desire to loose weigth for image issues that also had them increasing in the chance to light up? Just a thought....

These are some of the things that really irk me and give researchers a bad name. Always looking for a simple explanation and ignoring everthing else. You can find a correlation with anything (Statistical), but is it Practical, that's another thing!