Check out this post from Skeptical Scalpel about uncool tricks with statistical graphs. Editors beware!—Brenda Gregoline, ELS
Q: If there is a column for P values in a table and if a P value “straddles” rows (eg, provides the P value for men vs women), how should this be shown?
A: There are several options, with option 1 being preferred:
1. Center the P value between the items it compares (eg, between the values for men and women) and consider the use of a side brace.
2. If only 2 items are being compared, list the P value on the line giving the overall category (eg, Sex).
3. Use footnotes to indicate the P value for items being compared (eg, use a superscript “a” next to the value for men and the value for women and indicate the P value for this comparison in a footnote labeled “a”).
Q: If some of the confidence intervals given in a table column include negative values, how do you combine the minus sign and the hyphen that would normally be used in such a range in a table?
A: With ranges that include a minus sign, use to to express the range, rather than a hyphen. Carry this style throughout the entire table, even for those values that do not include a minus sign.—Cheryl Iverson, MA
If there is any doubt about whether significant/significance refers to statistical significance, clinical significance, or simply something “important” or “noteworthy,” choose another word or include a modifier that removes any ambiguity for the reader.
The AMA Manual of Style (§20.9, Glossary of Statistical Terms, pp 893-894 in print) includes definitions for statistical significance (the testing of the null hypothesis of no difference between groups; a significant result rejects the null hypothesis) and clinical significance (involves a judgment as to whether the risk factor or intervention studied would affect a patient’s outcome enough to make a difference for the patient; may be used interchangeably with clinical importance). Significant and significance also are used in more general contexts to describe worthiness or importance.
Often the context in which the word appears will make the meaning clear:
▪ Statistical Significance:
• Exposure to the health care system was a significant protective factor for exclusive throat carriage of Staphylococcus aureus (odds ratio, 0.67; P = .001).
• Most associations remained statistically significant at the adjusted significance level (P < .125).
▪ Clinical Significance:
• Low creatinine values in patients with connective tissue diseases were found to be clinically significant.
• The combination of erythromycin and carbamazepine represents a clinically significant drug interaction and should be avoided when possible.
• His appointment as chair of the department was a significant victory for those who appreciated his skill in teaching.
• A journal’s 100th anniversary is significant and should be celebrated.
Sometimes, however, the context does not clarify the meaning and ambiguity results.
▪ The one truly significant adverse effect that has caused carbon dioxide resurfacing to lose favor is hypopigmentation, which can be unpredictable and resistant to treatment.
To avoid the possibility of ambiguity, some have recommended confining the word to only one of its meanings. However, why cheat a word of one of its legitimate meanings when there are ways to retain its richness and yet not confuse the reader?—Cheryl Iverson, MA
I believe we are on an irreversible trend toward more freedom and democracy – but that could change.
In general usage, the concept of trend implies movement. Not only is this implied in its definitions, but the word can be traced to its Middle High German root of trendel, which is a disk or spinning top.1
In scientific writing, when is a trend not a trend? When it is not referring to comparisons of findings across an ordered series of categories or across periods of time. However, this and related terms are often misused in manuscripts and articles.
Most studies are constructed as hypothesis testing. Because an individual study only provides a point estimate of the truth, the researchers must determine before conducting the study an acceptable cutoff for the probability that a finding of an association is due to chance (the α value, most commonly but not universally set at .05 in clinical studies). This creates a dichotomous situation in interpreting the result: the study either does or does not meet this criterion. If the criterion is met, the finding is described as “statistically significant”; if it is not met, the finding is described as “not statistically significant.”
There are many limitations to this approach. Where the α level is set is arbitrary; therefore, in general all findings should be expressed as the study’s point estimate and confidence interval, rather than just the study estimate and the P value. Despite the limitations, if a researcher designs a study on the basis of hypothesis testing, it is not appropriate to change the rules after the results are available, and the results should be interpreted accordingly. The entire study design (such as calculation of the sample size and study power – the ability of a study to detect an actual difference or effect, if one truly exists) is dependent on setting the rules in advance and adhering to them.
If a study does not meet the significance criterion (for example, if the α level was set as < .05, and the P value for the finding was .08), authors sometimes describe the findings as “trending toward significance,” “having a trend toward significance,” “approaching significance,” “borderline significant,” or “nearly significant.” None of these terms is correct. Results do not trend toward significant—they either are or are not statistically significant based on the prespecified study assumptions. Similarly, the results do not include any movement and so cannot “approach” significance; and because of the dichotomous definition, “nearly significant” is no more meaningful than “nearly pregnant.”
When a finding does not meet statistical significance, there are generally 2 possible explanations: (1) There is no real association. (2) There might be an association, but the study was underpowered to detect it, usually because there were not enough participants or outcome events. A finding that does not meet statistical significance may still be clinically important and warrant further consideration.
However, when authors use terms such as trend or approaching significance, they are hedging the interpretation. In effect, they are treating the findings as if the association were statistically significant, or as if it might have been if the study had just gone a little differently. This is not justified. (Lang and Secic2 make the fascinating observation that “Curiously, P values never seem to ‘trend’ away from significance.”)
A proper use of the term trend refers to the results of one of the specific statistical tests for trend, the purpose of which is to estimate the likelihood that differences across 3 or more groups move (increase or decrease) in a meaningful direction more than would be expected by chance. For example, if a population of persons is ranked by evenly divided quintiles based on serum cholesterol level (from lowest to highest), and the risk of subsequent myocardial infarction is measured in each group, the researcher may want to determine whether risk increases in a linear way across the groups. Statistical tests that might be used for analyzing trends include the χ2 test for trend and the Cochran-Armitage test.
Similarly, a researcher may want to test for a directional movement in the values of data over time, such as a month-to-month decrease in prescriptions of a medication following publication of an article describing major adverse effects. A number of analytic approaches can be used for this, including time series and other regression models.
Instead of using these terms, the options are:
1. Delete the reported finding if it is not clinically important or a primary outcome. OR
2. Report the finding with its P value. Describe the result as “not statistically significant,” or “a statistically nonsignificant reduction/increase,” and provide the confidence interval so that the reader can judge whether insufficient power is a likely reason for the lack of statistical significance.
If the finding is considered clinically important, authors should discuss why they believe the results did not achieve statistical significance and provide support for this argument (for example, explaining how the study was underpowered). However, this type of discussion is an interpretation of the finding and should take place in the “Discussion” (or “Comment”) section, not in the “Results” section.
1. The term trend should only be used when reporting the results of statistical tests for trend.
2. Other uses of trend or approaching significance should be removed and replaced with a simple statement of the findings and the phrase not statistically significant (or the equivalent). Confidence intervals, along with point estimates, should be provided whenever possible.—Robert M. Golub, MD
1. Mish FC, ed in chief. Merriam-Webster’s Collegiate Dictionary. 11th ed. Springfield, MA: Merriam-Webster Inc; 2003.
2. Lang TA, Secic M. How to Report Statistics in Medicine: Annotated Guidelines for Authors, Editors, and Publishers. 2nd ed. Philadelphia, PA: American College of Physicans; 2006:56, 58.
In medical contexts, incidence is most often used in its epidemiologic sense, ie, the number of new cases of a disease occurring over a defined period among persons at risk for that disease. When thus used, incidence may be expressed as a percentage (new cases divided by number of persons at risk during the period) or as a rate (number of new cases divided by number of person-years at risk).
Reporting several incidence values in the same sentence can nearly always be accomplished using the singular form (eg, “the incidence of nonfatal myocardial infarction during follow-up was 10% at 6 months, 19% at 12 months, and 26% at 18 months” or “the incidence of clinical stroke decreased significantly, from 7.6 to 5.3 per 1000 person-years in men and from 6.2 to 5.1 per 1000 person-years in women). However, in rare instances, sentence construction may necessitate the use of the plural, which of course is… what, exactly? The understandable urge to simply add an “s” at the end of the word to form the plural results in incidences — a form not found in most dictionaries and a clunker of a word if ever there was one. Writers wishing for a more mellifluous plural sometimes use incidence rates, a valid term but one perhaps best reserved for reporting incidence values expressed as actual rates rather than simple percentages. Moreover, incidences is sometimes used when reporting values either as percentages or as rates, in the latter case missing a valuable opportunity to emphasize that rates rather than percentages are being reported.
Thus, it is perhaps best to use incidences, awkward as it may be, when reporting multiple incidence values as percentages and incidence rates when reporting such values as rates, eg, “at first follow-up, the incidences of falls resulting from frailty, neuromuscular disorders, or improper use of mobility devices were 15% (95% CI, 10%-20%), 12% (95% CI, 7%-17%), and 12% (5%-19%), respectively” or “the incidence rates for falls resulting from frailty, neuromuscular disorders, or improper use of mobility devices were 5.1, 6.3, and 4.6 per person-year, respectively.” Incidentally, these 2 examples report occurrences (falls) rather than diseases or conditions, and so represent 2 instances reporting the incidence of incidents.
To further muddy the waters, incidence is sometimes confused with prevalence, defined as the proportion of persons with a disease at any given time (ie, total number of cases divided by total population). Thus, whereas incidence describes how commonly cases are diagnosed, prevalence describes how widespread the disease already is; on a more personal level, incidence describes one’s risk of developing the disease, whereas prevalence describes the likelihood that one already has it. The confusion between the terms is perhaps attributable to the occasional use of prevalence in place of incidence in the study of rare, chronic diseases for which few newly diagnosed cases are available; however, this circumstance is unusual, and incidence and prevalence should always be distinguished from one another and used appropriately. (See also §20.9, Glossary of Statistical Terms, in the AMA Manual of Style, p 872 in print.)
Whereas prevalence is often used in general contexts to indicate predominance or general acceptance, the circumstances calling for the use of incidence in general contexts are quite few and become fewer still when one takes into account that incidence is often used when incidents (the simple plural of incident) or instance (again denoting an occurrence) would be the better choice. Perhaps incidents or instances was intended but never made it to the page — as is so often the case with homophones and near-homophones, even the careful writer who usually would not confuse incidence, incidents, and instance might one day look back over a hastily typed passage only to see that a wayward incidence has crept in; if the passage is hastily edited to boot, the error might well go unnoticed until the passage is in print and a discerning reader takes pains to point it out in a letter or e-mail. The plural form, incidences, has virtually no use outside of the epidemiologic discussed above, although it has been used to subtly disorienting effect by translators rendering the Kafkaesque works of Russian writer Daniil Kharms (1904-1942) into English, most notably when rendering the 1-word title of Incidences, Kharms’ 1934 collection of absurdist critiques on life in the Soviet Union under Stalin. However, writers who are not political dissidents aiming for absurdist effect — presumably all medical writers — would do well to proofread carefully and often. — Phil Sefton, ELS
Q: If a product name appears in all caps in a company’s product literature (with or without a trademark symbol or registered symbol), must the editor retain the all caps in a journal article? Companies use caps for graphic impact or emphasis, but caps can be distracting and can make the text difficult to read. Would it be acceptable to substitute only an initial cap for an all-cap product name, particularly if the product is the main subject of the manuscript and occurs frequently?
A: Our journals do not require use of the trademark symbol (™) or the registered symbol (®) as the use of the initial cap frequently used on proprietary names indicates the proprietary nature of the name (see 5.6.16, Legal and Ethical Considerations, Intellectual Property: Ownership Access, Rights, and Management, Trademark). There are exceptions to the use of the initial cap (eg, pHisoHex; see section 10.8, “Intercapped” Compounds) and in these cases, as in all others, we advise using the name according to the presentation of the legal trademark. To avoid a plethora of caps—which certainly can be distracting—we would suggest varying the way in which the product is referred to (eg, “this product,” “it”) as long as the meaning remains clear.
Q: Your manual indicates that references should be numbered consecutively with arabic numerals in the order in which they are cited in the text. But what about the distinction between references cited in a range and references cited individually? If an author cites references 1 through 5, does this count as only the citation of reference 1, as the first number in the range, or does it count as citation of all 5 references included in the range?
A: It matters not if the references are cited as part of a range or cited individually. Even if a reference is cited as part of a range, when any one of those references is cited later, it retains the same reference number. This is not specifically stated in the Manual and, perhaps wrongly, we assumed that it would be understood. Thank you for allowing us to clarify this point.
Q: Convention seems to be to use the leading zero in P values, but why is this necessary since P cannot be greater than 1?
A: JAMA and the Archives Journals do not use a zero to the left of the decimal point, since statistically it is not possible to prove or disprove the null hypothesis completely when only a sample of the population is tested (P cannot equal 1 or 0, except by rounding). If convention dictates otherwise, we are unconventional!
Q: I have been unable to find specific rules on the use of nonbreaking hyphens and spaces. Do you have any suggestions for the correct and preferred use of nonbreaking hyphens and spaces?
A: You are right. We do not have any section devoted to this. However, there is information about line breaks scattered throughout the Manual. For example:
There may be other instances like this scattered throughout the Manual where specific guidance is needed. However, individual publishers or clients may have their own preferences that require attention when editing material for their publications.
Q: I am working on a manuscript in which one of the authors has listed the degree MAS (Master of Advanced Studies). This abbreviation is not included in the Manual. Is it acceptable?
A: This is a perfectly acceptable abbreviation. We simply did not have space to list all possible degrees and their abbreviations in the Manual and attempted to list some of the more common ones.—Cheryl Iverson, MA
Sometimes editors (not you or I, of course) obey the rules of their institution’s preferred style manual without fully understanding, or really thinking about, why some of these rules exist. For example, some editors (not you or I, of course) automatically delete (or, if they’re lucky, their editing program deletes for them) the leading zero in a few statistics, but not all. They know exactly when and where to delete the leading zero, but not why. Or they round some statistics, but not all, assuming that all of this has something to do with saving space. It does, of course,1(p830) but this isn’t the only reason we do it.
The AMA Manual of Style defines a P value as “The probability of obtaining the observed data (or data that are more extreme) if the null hypothesis were exactly true.”1(p888) Per AMA style, P values greater than .01 are expressed to a maximum of 2 decimal places and those less than .01 are expressed to a maximum of 3 decimal places. I set out in search of the complicated statistical reason why we use this specific number of decimal places and found that, in addition to saving space, we do it for one simple reason: it’s all we need. Yep, that’s it. It’s all we need to know. P < .00000001 doesn’t tell us any more of value than P < .001. Both tell us that the probability is very low, and that’s good enough. Of course, if the author protests or rounding will make P appear nonsignificant, an exception is made (for example, if P = .046 and significance is set at P = .05).1(pp851-852) Also, studies such as genome-wide association studies report P values of P < .00001 or smaller, often in scientific notation, to address the issue of multiple comparisons; it is essential not to round these. So every rule has exceptions, I guess (remember Spanish class, anyone?).
Why then, you ask, do we not save ourselves the confusion and simply round P < .001 to P = .00? There’s a reason for that, too, and it’s the same reason we don’t use leading zeros with certain probability statistics (ah, you say, it all comes together). If probability is the chance that a given event will occur,2 and we have only surveyed a sample of a given population, probability cannot equal 1.0 or 0 because we can’t say absolutely that a null hypothesis will definitely or definitely not happen in that population.1(p889) And if P can’t equal 1.0 or 0, why include a zero that doesn’t tell us anything new? For this reason, we use P > .99 and P < .001 as the highest and lowest P values. For the same reason, and because they are used often, the leading zero rule applies to α and β probabilities as well. Why? To save space, of course.–Roya Khatiblou, MA
1. Iverson C, Christiansen S, Flanagin A, et al. AMA Manual of Style: A Guide for Authors and Editors. 10th ed. New York, NY: Oxford University Press; 2007.
2. Merriam-Webster’s Collegiate Dictionary. 10th ed. Springfield, MA: Merriam-Webster Inc; 1997.
What’s the difference between an α level and a β level? Do you know your y-axis from your x-axis from your z-axis? What term means the spread or dispersion of data? This month’s quiz, which subscribers can find at http://www.amamanualofstyle.com/, can help you learn the answers to these and other questions on statistical terms. On the basis of your understanding of section 20.9 of the AMA Manual of Style, select the correct answer from the choices listed in the following sample quiz question.
Which of the following terms means the correlation coefficient for bivariate analysis?
So, how did you do? Here’s the answer (use your mouse to highlight the blank line):
Which of the following terms means the correlation coefficient for bivariate analysis?
R is the correlation coefficient for multivariate analysis. r2 is the coefficient of determination for bivariate analysis. R2 is the coefficient to determination for multivariate analysis.
If you want to further test your knowledge of statistical terms, subscribe to the AMA Manual of Style online and take the full quiz. Stay tuned next month for another edition of Quiz Bowl.—Laura King, MA, ELS