^{a}

^{b}

Evaluating how an effect-size estimate performs between two continuous variables based on the common-language effect size (CLES) has received increasing attention. While Blomqvist (1950; https://doi.org/10.1214/aoms/1177729754) developed a parametric estimator (q') for the CLES, there has been limited progress in further refining CLES. This study: a) extends Blomqvist’s work by providing a mathematical foundation for Bp (a non-parametric version of CLES) and an analytic approach for estimating its standard error; and b) evaluates the performance of the analytic and bootstrap confidence intervals (CIs) for Bp. The simulation shows that the bootstrap bias-corrected-and-accelerated interval (BCaI) has the best protected Type 1 error rate with a slight compromise in Power, whereas the analytic-t CI has the highest overall Power but with a Type 1 error slightly larger than the nominal value. This study also uses a real-world data-set to demonstrate the applicability of the CLES in measuring the relationship between age and sexual compulsivity.

In psychological research, there has been increasing attention paid to the importance of effect size (ES) estimates and confidence interval (CI) in improving the quality of statistical practices.

ES is mainly measured and quantified based on two theoretical frameworks: the

By contrast, the interpretation of

In light of this, researchers have explored and considered alternative ESs beyond the

While CLES for two group comparisons has received increasing attention among behavioral researchers, the understanding of whether CLES can be used in evaluating the effect between two continuous variables

where _{1}) that belong to the first or third quadrants compared with the total number of observations (

Despite the potential of

_{i}_{i}

where

where “

The PBS between

Given Equation (4), _{1} is defined as the number of _{2} is defined as the number of

where _{1} is the number of observations that belong to the first or third quadrants, and _{2} is the number of observations that belong to the second or fourth quadrants in the

where

Under the special case when

where the left side becomes

In fact, Equation 8 is identical to

Assuming that (

where

where

where

Five distributions were evaluated (_{r}

Six levels of sample sizes—20, 50, 100, 300, 500, and 1000—were evaluated, which comprehensively cover a small to large sample size in behavioral research.

Nine levels of

These factors are combined to produce a design with

For the first type of distribution (BLNC),

where

For the remaining distributions, the simulation was executed in the R package (truncdist;

Bias is used to evaluate the performance of the point estimates for the true PBS (

Coverage probability is defined as the likelihood that the 95% CIs surrounding the

The width is defined as the difference between the upper and lower limits of a CI. A narrower (or wider) CI means that the method can produce a more (or less) precise boundary surrounding the

When

When the assumption of BLNC was met,

Bias | BLNC | PBS-normal | PBS- |
PBS-uniform | PBS-beta | Overall |
---|---|---|---|---|---|---|

.0001 | -.0103 | -.0112 | -.0080 | -.0056 | -.0070 | |

.0018 | .0099 | .0104 | .0076 | .0056 | .0087 | |

Min | -.0069 | -.0428 | -.0435 | -.0306 | -.0243 | -.0435 |

Max | .0042 | .0023 | .0005 | .0032 | .0012 | .0042 |

Of the six methods, the BSI-

Performance | Analytic- |
Analytic- |
BSI- |
BSI- |
BPI | BCaI |
---|---|---|---|---|---|---|

CP | ||||||

.9326 | .9371 | .9668 | .9695 | .9458 | .9278 | |

.0219 | .0195 | .0100 | .0115 | .0410 | .0218 | |

Min | .8120 | .8670 | .9440 | .9460 | .7550 | .8130 |

Max | .9640 | .9700 | .9940 | .9980 | 1.0000 | .9650 |

% within [.925, .975] | .7593 | .8111 | .7926 | .6741 | .6370 | .6926 |

Width | ||||||

.1695 | .1755 | .2029 | .2104 | .2009 | .1978 | |

.1151 | .1241 | .1465 | .1581 | .1433 | .1436 | |

Min | .0372 | .0372 | .0399 | .0399 | .0399 | .0401 |

Max | .4274 | .4581 | .5130 | .5499 | .5064 | .4986 |

Type 1 Error | ||||||

.0547 | .0547 | .0380 | .0359 | .0225 | .0509 | |

.0101 | .0101 | .0075 | .0093 | .0127 | .0072 | |

Min | .0360 | .0360 | .0230 | .0150 | .0010 | .0320 |

Max | .0740 | .0740 | .0500 | .0490 | .0460 | .0660 |

Power | ||||||

.8108 | .8108 | .7895 | .7819 | .7378 | .7361 | |

.2967 | .2967 | .3151 | .3228 | .3631 | .3548 | |

Min | .0580 | .0580 | .0380 | .0320 | .0040 | .0280 |

Max | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |

For the two percentile-based bootstrap CIs, BCaI has the best protected Type 1 error rates, which ranged from .0320 to .0660, with a mean of .0509: very close to the nominal value of .05. On the other hand, the Power of BCaI was found to be the smallest (mean = .7361) relative to all other methods. This finding is understandable because a better protected Type 1 error rate tends to decrease the Power in detecting a significant result. Of the 270 conditions, 187 (or 69.26%) of the CPs fell within the criterion of [.925, .975], but the mean of the CPs was the smallest (.9278) relative to all the other methods. Comparatively, another percentile-based CI (BPI) is more conservative than BCaI, when a researcher is making an inferential-statistical decision. Here, the mean Type 1 error rate was .0225, and the mean Power rate was .7378; both values were small relative to the other methods. Of the 270 conditions, 172 (or 63.70%) of the CPs fell within the criterion of [.925, .975], although the mean CP (.9458) was the closest to the nominal value of .95.

Regarding the two analytic approaches, both the analytic-

Given that sample size is the only factor that researchers can plan and control in practice, this section examines the effects of different sample size levels on the 6 CIs^{1}

There was no obvious difference regarding the effects of the data distributions and true

Performance / |
Analytic- |
Analytic- |
BSI- |
BSI- |
BPI | BCaI |
---|---|---|---|---|---|---|

CP | ||||||

20 | .9180 | .9336 | .9744 | .9835 | .9700 | .8899 |

50 | .9314 | .9381 | .9724 | .9755 | .9634 | .9261 |

100 | .9364 | .9386 | .9675 | .9699 | .9466 | .9267 |

300 | .9374 | .9381 | .9650 | .9656 | .9374 | .9396 |

500 | .9364 | .9380 | .9621 | .9625 | .9318 | .9417 |

1000 | .9358 | .9362 | .9596 | .9597 | .9256 | .9427 |

Width | ||||||

20 | .3746 | .4015 | .4774 | .5118 | .4672 | .4684 |

50 | .2415 | .2478 | .2880 | .2955 | .2863 | .2795 |

100 | .1712 | .1734 | .1978 | .2003 | .1974 | .1917 |

300 | .0989 | .0993 | .1104 | .1109 | .1105 | .1073 |

500 | .0765 | .0767 | .0845 | .0847 | .0847 | .0823 |

1000 | .0541 | .0541 | .0590 | .0591 | .0592 | .0576 |

Type 1 Error | ||||||

20 | .0404 | .0404 | .0302 | .0246 | .0046 | .0544 |

50 | .0668 | .0668 | .0332 | .0298 | .0120 | .0494 |

100 | .0536 | .0536 | .0370 | .0344 | .0180 | .0492 |

300 | .0560 | .0560 | .0396 | .0392 | .0294 | .0492 |

500 | .0578 | .0578 | .0434 | .0432 | .0334 | .0534 |

1000 | .0534 | .0534 | .0446 | .0444 | .0374 | .0500 |

Power | ||||||

20 | .0678 | .0678 | .0500 | .0370 | .0092 | .0350 |

50 | .1282 | .1282 | .0764 | .0680 | .0320 | .0548 |

100 | .1784 | .1784 | .1372 | .1320 | .0866 | .1104 |

300 | .4146 | .4146 | .3696 | .3674 | .3194 | .3280 |

500 | .6000 | .6000 | .5604 | .5592 | .5174 | .5252 |

1000 | .8834 | .8834 | .8696 | .8690 | .8540 | .8502 |

First, when sample size was increased, the CPs obtained from the 6 CIs tended to be closer to the nominal value of .95. Specifically, the analytic-

Second, the mean widths of the 6 CIs became narrower when

Third, BCaI always led to the best protected mean Type 1 error rates (ranging from .0492 to .0544), and the analytic-

Fourth, when

The only influential factor remaining lies in the effect of different

β_{p} |
Analytic- |
Analytic- |
BSI- |
BSI- |
BPI | BCaI |
---|---|---|---|---|---|---|

.55 | .3787 | .3787 | .3439 | .3388 | .3031 | .3173 |

.60 | .6402 | .6402 | .6047 | .5975 | .5580 | .5643 |

.65 | .7740 | .7740 | .7402 | .7302 | .6811 | .6833 |

.70 | .8601 | .8601 | .8340 | .8233 | .7718 | .7701 |

.75 | .9146 | .9146 | .8986 | .8885 | .8390 | .8310 |

.80 | .9509 | .9509 | .9407 | .9328 | .8849 | .8774 |

.85 | .9761 | .9761 | .9678 | .9612 | .9183 | .9074 |

.90 | .9917 | .9917 | .9863 | .9825 | .9460 | .9377 |

_{p} = the true population probability-of-bivariate-superiority (PBS) value; BSI-

In sum, when a study sample is small (

On the basis of sensation-seeking theories,

There is an open-access database that provides a raw SCS data-set for research purposes (the data used for the current analysis is available in the

A lack of bivariate linear correlation does not imply a lack of bivariate relationship. Most behavioral researchers examine a research hypothesis that is based on linear relationships between variables. They typically specify and choose a linear-based statistical model (e.g., Pearson’s correlation), despite the fact that there are many other types of bivariate relationships (e.g., curvilinearity;

The present study is a crucial development in extending and promoting the use of PBS in practice. Researchers are increasingly aware of the importance and usefulness of PBS in examining relationships between variables. Conceptually, both

Most publication manuals in psychology (e.g.,

Our simulation results show that each of the 6 CI methods behave in a manner that may serve for different research purposes. If a researcher prefers to use a method that encompasses many potential study effects, perhaps in the early stages of exploratory research, then the researcher can use the analytic-

In the real-world example, the results led to different conceptual understandings of the pattern of relationships based on

In terms of theory, this study provides the details of the necessary mathematical proof (Equations 3 - 12) for the point estimate,

This research was funded by a University Research Grants Program (URGP) to Johnson Ching-Hong Li in the Department of Psychology at the University of Manitoba (#47094).

The authors have declared that no competing interests exist.

The authors have no support to report.