What makes a conclusion reliable

To produce valid generalizable results, clearly define the population you are researching e. Ensure that you have enough participants and that they are representative of the population. Reliability should be considered throughout the data collection process. Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations, clearly define how specific behaviours or responses will be counted, and make sure questions are phrased the same way each time. When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions. Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Have a language expert improve your writing. Check your paper for plagiarism in 10 minutes. Do the check. Generate your APA citations for free! APA Citation Generator. Reliability vs validity Reliability Validity What does it tell you?

The extent to which the results can be reproduced when the research is repeated under the same conditions. The extent to which the results really measure what they are supposed to measure. How is it assessed? By checking the consistency of results across time, across different observers, and across parts of the test itself. By checking how well the results correspond to established theories and other measures of the same concept.

How do they relate? A valid measurement is generally reliable: if a test produces accurate results, they should be reproducible. You measure the temperature of a liquid sample several times under identical conditions. The thermometer displays the same temperature every time, so the results are reliable. A doctor uses a symptom questionnaire to diagnose a patient with a long-term medical condition. Several different doctors use the same questionnaire with the same patient but give different diagnoses.

This indicates that the questionnaire has low reliability as a measure of the condition. If a symptom questionnaire results in a reliable diagnosis when answered at different times and with different doctors, this indicates that it has high validity as a measurement of the medical condition.

The thermometer that you used to test the sample gives reliable results. However, the thermometer has not been calibrated properly, so the result is 2 degrees lower than the true value.

Therefore, the measurement is not valid. A group of participants take a test designed to measure working memory. What can proofreading do for your paper? Is this article helpful? So, it is possible that in a study we can conclude that our program and outcome are related conclusion validity and also conclude that the outcome was caused by some factor other than the program i. Another issue is that the relationship we are looking for may be a weak one and seeing it is a bit like looking for a needle in the haystack.

We send an occasional email to keep our users informed about new developments on Conjoint. You can always unsubscribe later. Your email will not be shared with other companies. Request consultation. Looking for a free online survey tool? Get started for free. Trochim hosted by Conjoint. Free Survey Tool Fully-functional online survey tool with various question types, logic, randomisation, and reporting for unlimited number of responses and surveys. Start now View details. List of articles Back.

Knowledge Base Analysis Conclusion Validity. Gans, D. Use of a preliminary test in comparing two sample means. On the confidence interval for the binomial parameter. Bayesian adaptive estimation of arbitrary points on a psychometric function. Testing equivalence with repeated measures: tests of the difference model of two-alternative forced-choice performance. On the discrepant results in synchrony judgment and temporal-order judgment tasks: a quantitative model. A comparison of anchor-item designs for the concurrent calibration of large banks of Likert-type items.

Psychometric functions for detection and discrimination with and without flankers. Statistical inference involving binomial and negative binomial parameters. Girden, E. Evaluating Research Articles. From Start to Finish , 3rd Edn. Thousand Oaks, CA: Sage. Goodwin, C. Research in Psychology. Methods and Design , 6th Edn. Hoboken, NJ: Wiley.

Graybill, F. Determining sample size for a specified width confidence interval. Green, B. The perception of distance and location for dual tactile figures.

Greenwald, A. Hawkins, D. Diagnostics for conformity of paired quantitative measurements. Hayes, A. Further evaluating the conditional decision rule for comparing two independent means. Henson, R. Methodology in our education research culture: toward a stronger collective quantitative proficiency.

Hoekstra, R. Are assumptions of well-known statistical techniques checked, and why not? Howard, G. Isaac, P. Linear regression, structural relations, and measurement error. Jan, S. Jennison, C. Statistical approaches to interim monitoring of clinical trials: a review and commentary. John, L. Measuring the prevalence of questionable research practices with incentives for truth telling.

Kalantar, A. Biases in summary statistics of slopes and intercepts in linear regression with errors in both variables. Talanta 42, — Keselman, H. The new and improved two-sample t test. Ketellapper, R.

On estimating parameters in a simple linear errors-in-variables model. Technometrics 25, 43— Lee, B. Statistical conclusion validity in ex post facto designs: practicality in evaluation. Policy Anal. Lippa, R. The relation between sex drive and sexual attraction to men and women: a cross-national study of heterosexual, bisexual, and homosexual men and women. Lumley, T. The importance of the normality assumption in large public health data sets.

Public Health 23, — Malakoff, D. Science , — Mandansky, A. The fitting of straight lines when both variables are subject to error. Matthews, W. What might judgment and decision making research be like if we took a Bayesian approach to hypothesis testing? Maxwell, S. Sample size planning for statistical power and accuracy in parameter estimation. Maylor, E. Alcohol, reaction time and memory: a meta-analysis. McCarroll, D.

Mehta, C. Adaptive increase in sample size when interim results are promising: a practical guide with examples. Milligan, G. Statistical conclusion validity in experimental designs used in business research. Morse, D. MINSIZE2: a computer program for determining effect size and minimum sample size for statistical significance for univariate, multivariate, and nonparametric tests. Moser, B.

Homogeneity of variance in the two-sample means test. Ng, M. A comparison of two-stage procedures for testing least-squares coefficients under heteroscedasticity. Nickerson, R. Null hypothesis significance testing: a review of an old and continuing controversy. Methods 5, — What authors want from journal reviewers and editors. Nieuwenhuis, S. Erroneous analyses of interactions in neuroscience: a problem of significance.

Nisen, J. A simple method of computing the sample size for chi-square test for the equality of multinomial distributions. Data Anal. Orme, J. Statistical conclusion validity for single-system designs. Ottenbacher, K. Statistical conclusion validity of early intervention research with handicapped children.

How to detect effects: statistical power and evidence-based practice in occupational therapy research. Rankupalli, B. Practicing evidence-based psychiatry: 1. Asian J. Rasch, D. The two-sample t test: pre-testing its assumptions does not pay off.

Riggs, D. Fitting straight lines when both variables are subject to error. Life Sci. Rochon, J. A closer look at the effect of preliminary goodness-of-fit testing for normality for the one-sample t-test. Saberi, K. A detection-theoretic model of echo inhibition. Schucany, W. Preliminary goodness-of-fit tests for normality do not validate the one-sample Student t.

Theory Methods 35, — Shadish, W. Shieh, G. Methods 44, — Shun, Z. Type I error in sample size re-estimations based on observed treatment difference. Simmons, J. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant.

Smith, P. Mask-dependent attentional cuing effects in visual signal detection: the psychometric function for contrast.

Sternberg, R. On civility in reviewing. APS Obs. Stevens, W. Fiducial limits of the parameter of a discontinuous distribution. Biometrika 37, — Strube, M. SNOOP: a program for demonstrating the consequences of premature and repeated null hypothesis testing. Methods 38, 24— Sullivan, L. Robustness of the t test applied to data distorted from normality by floor effects. Treisman, M. Vecchiato, G. The issue of multiple univariate comparisons in the context of neuroelectric brain mapping: an application in a neuromarketing experiment.

Methods , — Vul, E. Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Wagenmakers, E. A practical solution to the pervasive problems of p values. Wald, A. The fitting of straight lines if both variables are subject to error. Sequential Analysis. Wells, C. Dealing with assumptions underlying statistical tests.

Wetherill, G. Sequential Methods in Statistics. London: Chapman and Hall. Wicherts, J. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Wilcox, R. New methods for comparing groups: strategies for increasing the probability of detecting true differences. Modern robust data analysis methods: measures of central tendency.

Methods 8, — Wilkinson, L. The Task Force on Statistical Inference. Statistical methods in psychology journals: guidelines and explanations. Ximenez, C. Xu, E. Rhesus monkeys lack a consistent peak-end effect. Yeshurun, Y. Bias and sensitivity in two-interval forced choice procedures: tests of the difference model.