## This site is under construction. Please check back every few weeks for updates

COMMON MISTEAKS MISTAKES IN USING STATISTICS: Spotting and Avoiding Them

# More on Choosing a Confidence Level for a Confidence Interval

Considerations in choosing a confidence level for a confidence interval are essentially the same as those discussed in setting a significance level for a hypothesis test in Type I and II Errors, so the discussion here will be very much like the discussion there; the same examples will be used to illustrate.

• Confidence levels should be chosen before analyzing the data -- preferably before gathering the data.1
• The choice should be based on the consequences of having a large or small confidence level.
Example 1: Two drugs are being compared for effectiveness in treating the same condition. Drug 1 is very affordable, but Drug 2 is extremely expensive.  The confidence interval is for (proportion of patients for whom Drug 2 is effective) - (proportion of patients for whom drug 1 is effective).  If 0 is not in the confidence interval, Drug 2 will be deemed more effective and will be recommended over Drug 1, resulting in much greater cost for the patient than if Drug 1 were used. If 0 is in the confidence interval, this will be taken as of evidence of equal effectiveness, and the less expensive Drug 2 will be recommended. From the patient's perspective, this would be a serious consequence. Thus the patient would consider it important to have a high level of confidence. (Note that a high confidence level corresponds to a small significance level; cf Example 1 in Type I and II Errors.)

Example 2: Two drugs are known to be equally effective for a certain condition. They are also each equally affordable. However, there is some suspicion that Drug 2 causes a serious side-effect in some patients, whereas Drug 1 has been used for decades with no reports of serious side effects. The confidence interval is for (incidence of side effect in Drug 2) - (incidence of side effect in Drug 1). If 0 is in the confidence interval, the investigators will decide that the incidence of the side effects is the same for both drugs. The higher the confidence level, the larger the confidence interval, so the more likely that this decision will be made. If this is the wrong decision, the results could be serious for the patient. Thus the patient would consider a lower confidence level to be preferable to a higher one. (Note that a low confidence level corresponds to a large significance level; cf Example 2 in Type I and II Errors.)