The t-Test

The t-test assesses whether the means of two groups are statistically different from each other. This analysis is appropriate whenever you want to compare the means of two groups.

             Figure 1. Idealized distributions for treated and comparison group posttest values.


Figure 1 shows the distributions for the treated (blue) and control (green) groups in a study. The figure indicates where the control and treatment group means are located. The question the t-test addresses is whether the means are statistically different.

What does it mean to say that the averages for two groups are statistically different? Consider the three situations shown in Figure 2. The first thing to notice about the three situations is that the difference between the means is the same in all three. But the three situations don't look the same -- The top example shows a case with moderate variability of scores within each group. The second situation shows the high variability case. The third shows the case with low variability. Clearly, we would conclude that the two groups appear most different or distinct in the bottom or low-variability case. Why? Because there is relatively little overlap between the two bell-shaped curves. In the high variability case, the group difference appears least striking because the two bell-shaped distributions overlap so much.

                                     Figure 2. Three scenarios for differences between means.

This leads us to a very important conclusion: when we are looking at the differences between scores for two groups, we have to judge the difference between their means relative to the spread or variability of their scores. The t-test does just this.

Statistical Analysis of the t-test

The formula for the t-test is a ratio. The top part of the ratio is just the difference between the two means or averages. The bottom part is a measure of the variability or dispersion of the scores. This formula is essentially another example of the signal-to-noise metaphor in research: the difference between the means is the signal that, in this case, we think our program or treatment introduced into the data; the bottom part of the formula is a measure of variability that is essentially noise that may make it harder to see the group difference. Figure 3 shows the formula for the t-test and how the numerator and denominator are related to the distributions.

                                                           Figure 3. Formula for the t-test.

The top part of the formula the difference between the means. The bottom part is called the standard error (SE) of the difference. To compute it, we take the variance for each group and divide it by the number of people in that group. We add these two values and then take their square root. The specific formula is given in Figure 4:

                           Figure 4. Formula for the Standard error of the difference between the means.

The variance is of course simply the square of the standard deviation.

The final formula for the t-test is shown in Figure 5:

                                                         Figure 5. Formula for the t-test.

The t-value will be positive if the first mean is larger than the second and negative if it is smaller. Once we compute the t-value we have to look it up in a table of significance to test whether the ratio is large enough to say that the difference between the groups is not likely to have been a chance finding. To test the significance, we need to set a risk level (called the alpha level). In most social research, the "rule of thumb" is to set the alpha level at .05. This means that five times out of a hundred we would find a statistically significant difference between the means even if there was no such difference (i.e., "chance" occurance). We also need to determine the degrees of freedom (df) for the test. In the t-test, the df is the sum of the persons in both groups minus 2. Given the alpha level, the df, and the t-value, we can look the t-value up in a standard table of significance (given below) to determine whether the t-value is large enough to be significant. If it is, we can conclude that the difference between the means for the two groups is different (even given the variability).

- From http://www.socialresearchmethods.net/kb/index.php

Design of Experiments (DoE)

DoE is a third 'Advanced Tool' for problem solving.
Despite all the efforts by specialists in quality and statistics, Design Of Experiments (DOE) is still not applied as widely as it could and should be, because there is a wrong notion that it is too complex. We just need to know how the system, product or process will react if one factor is changed from one level to another level.


We can divide the experimentation process in four phases: setting-up the experiment, executing the tests, analyzing the results and drawing conclusions. We need to use basic rules from DOE to avoid mistakes.

Rule number one: write down the questions you would like to see answered by the experiment. E.g. does the "red tomato" fertilizer increase my tomato harvest by at least 20% in weight?

Rule number two: don’t forget that characteristics that are not part of the study also need to fulfill requirements.

E.g. as a result of changing fertilizer if we have 20 % more tomatoes, but they should not be of bad taste or small size. So at the end of the experiment we need to measure and evaluate these characteristics

Rule number three: make sure to have a reliable measurement system. You must be aware of the importance of the variation introduced by the measurement system and have to keep it at a minimum.

Rule number four: use statistics and statistical principles upfront. If you want to detect small differences the sample sizes increase drastically. For other cases, it can be smaller.

Rule number five: beware of known enemies. E.g. a tree causes shades on some tomato plants but not on the others. We can place half of them in the sun and half of them in the shade. In DOE this is called "blocking". For every known enemy we have to develop a strategy and keep it constant for the test.

Rule number six: beware of unknown enemies. E.g. In a garden, soil composition, effect of wind, ground water levels, etc. may or may not influence the result of our test. So the experiment is set up in such a way that these factors are distributed randomly, by chance. Randomizing within each block can be done by taking three black and three red playing cards, shuffle them and at each test location within the block pick one card. If it is a black card, treat that plant with "tomato lover", if it is a red card he treat it with "the red tomato".

This is a randomization in location, in many industrial tests, randomization in time is needed. This means that the sequence of executing the tests has to be decided by chance within each block.

Rule number seven: beware of what goes on during testing. With industrial there is no end to what can go wrong during testing. In many cases the people performing the tests have not been part of the team that designed it, they have no idea what it is about or sometimes even why it is done. So keep these two golden rules in mind:

1. He who communicates is king

2. Be where it happens when it happens.

Rule number eight: analyse the results statistically to find the mean and the standard deviation of the two types of treatment. Statistically we test the null hypothesis that the means are equal versus the alternative that the difference between the means is larger than the objective. This is done with a t-test.

If the result is positive, Sam would still have to analyze all the other characteristics that need to fulfill minimum requirements.

Rule number nine: present the results graphically. Since not all people involved in the experiment are knowledgeable of statistics, graphical presentation of results is so important in communicating. Actually, in most cases the graphical output will tell the whole story. Only when there is some doubt left, the correct numbers may be needed to take a final decision.

Conclusion

There is no such thing as a "simple" experiment. No matter how simple it may look, you need to take several rules into account if you want to be able to draw correct conclusions out of your tests. Don’t forget that it is equally expensive to run a bad or a good experiment. The only difference is that the good experiment has a return on investment.

- Ref: http://www.improvementandinnovation.com

Regression Analysis

This is another of the 'Advanced Tools' for problem solving.
Here we apply regression analysis to some fictitious data, and we show how to interpret the results of our analysis.
Note: Regression computations can be done in the Excel spreadsheet also. For this example, however, we will do the computations "manually", to know the details of how it works.

Problem StatementLast year, five randomly selected students took a math aptitude test before they began their statistics course. The Statistics Department has three questions.
 What linear regression equation best predicts statistics performance, based on math aptitude scores?
 If a student made an 80 on the aptitude test, what grade would we expect her to make in statistics?
 How well does the regression equation fit the data?

How to Find the Regression Equation
In the table below, the xi column shows scores on the aptitude test. Similarly, the yi column shows statistics grades. The last two rows show sums and (arithmetic) mean scores that we will use to conduct the regression analysis.

The regression equation is a linear equation of the form: ŷ = b₀ + b₁x . To conduct a regression analysis, we need to solve for b₀ and b₁. Computations are shown below.

Therefore, the regression equation is: ŷ = 26.768 + 0.644x .

How to Use the Regression Equation
Once you have the regression equation, using it is a snap. Choose a value for the independent variable (x), perform the computation, and you have an estimated value (ŷ) for the dependent variable.

In our example, the independent variable is the student's score on the aptitude test. The dependent variable is the student's statistics grade. If a student made an 80 on the aptitude test, the estimated statistics grade would be:

ŷ = 26.768 + 0.644x = 26.768 + 0.644 * 80 = 26.768 + 51.52 = 78.288

Warning: When you use a regression equation, do not use values for the independent variable that are outside the range of values used to create the equation. That is called extrapolation, and it can produce unreasonable estimates.

In this example, the aptitude test scores used to create the regression equation ranged from 60 to 95. Therefore, only use values inside that range to estimate statistics grades. Using values outside that range (less than 60 or greater than 95) is problematic.

How to Find the Coefficient of Determination
Whenever you use a regression equation, you should ask how well the equation fits the data. One way to assess fit is to check the coefficient of determination, which can be computed from the following formula.

where N is the number of observations used to fit the model, Σ is the summation symbol, xi is the x value for observation i, x is the mean x value, yi is the y value for observation i, y is the mean y value, σx is the standard deviation of x, and σy is the standard deviation of y. Computations for the sample problem of this lesson are shown below.

A coefficient of determination equal to 0.48 indicates that about 48% of the variation in statistics grades (the dependent variable) can be explained by the relationship to math aptitude scores (the independent variable). This would be considered a good fit to the data, in the sense that it would substantially improve an educator's ability to predict student performance in statistics class. A coefficient of 1 indicates a perfect or 100% fit. A correlation greater than 0.8 is generally described as strong, whereas a correlation less than 0.5 is generally described as weak. These values can vary based upon the "type" of data being examined. A study utilizing scientific data may require a stronge correlation than a study using social science data.

- Ref AP* (Advanced Placement) Statistics Tutorial, http://stattrek.com/AP-Statistics-1/Regression-Example.aspx