Skip to content

mean2

Jump to bottom Edit New page
Henrik Bengtsson edited this page Sep 11, 2019 · 3 revisions

matrixStats: Benchmark report

mean2() benchmarks

This report benchmark the performance of mean2() against alternative methods.

Alternative methods

  • mean() + [()
  • mean.default() + [() - avoids method dispatching

as below

> mean2_R_v1 <- function(x, na.rm = FALSE, idxs) {
+     mean(x[idxs], na.rm = na.rm)
+ }

and

> mean2_R_v2 <- function(x, na.rm = FALSE, idxs) {
+     mean.default(x[idxs], na.rm = na.rm)
+ }

Data type "integer"

Data

> rvector <- function(n, mode = c("logical", "double", "integer"), range = c(-100, +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     x
+ }
> rvectors <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rvector(n = scale * 100, ...)
+     data[[2]] <- rvector(n = scale * 1000, ...)
+     data[[3]] <- rvector(n = scale * 10000, ...)
+     data[[4]] <- rvector(n = scale * 1e+05, ...)
+     data[[5]] <- rvector(n = scale * 1e+06, ...)
+     names(data) <- sprintf("n = %d", sapply(data, FUN = length))
+     data
+ }
> data <- rvectors(mode = mode)

Results

n = 1000 vector

All elements

> x <- data[["n = 1000"]]
> gc()
           used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells  3237480 173.0    5709258 305.0  5709258 305.0
Vcells 33440267 255.2   58570544 446.9 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
4 mean.default 0.001515 0.0015645 0.0017326 0.0016040 0.0016535 0.011704
1 mean2 0.001699 0.0017375 0.0019839 0.0017725 0.0018385 0.019023
2 mean2_no_refine 0.001706 0.0017340 0.0018111 0.0017765 0.0018465 0.002247
3 mean 0.002852 0.0029990 0.0032766 0.0030850 0.0032150 0.018680
expr min lq mean median uq max
4 mean.default 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 mean2 1.121452 1.110578 1.145010 1.105050 1.111884 1.6253418
2 mean2_no_refine 1.126073 1.108341 1.045272 1.107544 1.116722 0.1919856
3 mean 1.882508 1.916906 1.891120 1.923317 1.944360 1.5960355

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells  3235334 172.8    5709258 305.0  5709258 305.0
Vcells 11804806  90.1   46856436 357.5 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.001116 0.0011765 0.0014133 0.0012820 0.0013720 0.013952
1 mean2 0.001118 0.0011690 0.0013048 0.0012825 0.0013955 0.001716
4 mean.default+[() 0.001853 0.0021405 0.0023856 0.0022795 0.0024160 0.010225
3 mean+[() 0.003809 0.0040480 0.0044210 0.0041795 0.0043395 0.022491
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.0000000 1.0000000 1.000000 1.000000 1.0000000
1 mean2 1.001792 0.9936252 0.9232418 1.000390 1.017128 0.1229931
4 mean.default+[() 1.660394 1.8193795 1.6880002 1.778081 1.760933 0.7328698
3 mean+[() 3.413082 3.4407140 3.1282062 3.260140 3.162901 1.6120269

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235418 172.8    5709258  305  5709258 305.0
Vcells 11805482  90.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 0.001475 0.0015355 0.0017779 0.0016065 0.0017025 0.016743
2 mean2_no_refine 0.001475 0.0015620 0.0016735 0.0016425 0.0017490 0.002661
4 mean.default+[() 0.002469 0.0026850 0.0028144 0.0027760 0.0028850 0.005091
3 mean+[() 0.004350 0.0045825 0.0050131 0.0047255 0.0049415 0.028994
expr min lq mean median uq max
1 mean2 1.000000 1.000000 1.0000000 1.000000 1.000000 1.0000000
2 mean2_no_refine 1.000000 1.017258 0.9412618 1.022409 1.027313 0.1589321
4 mean.default+[() 1.673898 1.748616 1.5830226 1.727980 1.694567 0.3040674
3 mean+[() 2.949152 2.984370 2.8196851 2.941488 2.902496 1.7317088

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235499 172.8    5709258  305  5709258 305.0
Vcells 11805736  90.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.002204 0.0022815 0.0023979 0.0023940 0.0024480 0.003043
1 mean2 0.002205 0.0022845 0.0025144 0.0024115 0.0025000 0.011565
4 mean.default+[() 0.003401 0.0036100 0.0038942 0.0037255 0.0038805 0.017993
3 mean+[() 0.005348 0.0055400 0.0059136 0.0056730 0.0059210 0.024336
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.000454 1.001315 1.048571 1.007310 1.021242 3.800526
4 mean.default+[() 1.543103 1.582292 1.624014 1.556182 1.585172 5.912915
3 mean+[() 2.426497 2.428227 2.466152 2.369674 2.418709 7.997371

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 10000 vector

All elements

> x <- data[["n = 10000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235512 172.8    5709258  305  5709258 305.0
Vcells 11805522  90.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
4 mean.default 0.008665 0.0087360 0.0088370 0.0087900 0.0088355 0.011673
3 mean 0.010130 0.0103125 0.0106210 0.0104245 0.0105620 0.027637
1 mean2 0.011046 0.0111320 0.0112285 0.0112025 0.0112600 0.013338
2 mean2_no_refine 0.011064 0.0111220 0.0113622 0.0112050 0.0112820 0.022468
expr min lq mean median uq max
4 mean.default 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
3 mean 1.169071 1.180460 1.201879 1.185950 1.195405 2.367600
1 mean2 1.274784 1.274267 1.270628 1.274460 1.274404 1.142637
2 mean2_no_refine 1.276861 1.273123 1.285754 1.274744 1.276894 1.924784

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235661 172.9    5709258  305  5709258 305.0
Vcells 11806907  90.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.004471 0.0045425 0.0046697 0.0046325 0.0047695 0.005352
1 mean2 0.004475 0.0045420 0.0047574 0.0046515 0.0047305 0.013912
4 mean.default+[() 0.006273 0.0066685 0.0070391 0.0068180 0.0070980 0.011162
3 mean+[() 0.008449 0.0088215 0.0095840 0.0090405 0.0093300 0.043923
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.0000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.000895 0.9998899 1.018783 1.004101 0.991823 2.599402
4 mean.default+[() 1.403042 1.4680242 1.507397 1.471775 1.488206 2.085576
3 mean+[() 1.889734 1.9419923 2.052391 1.951538 1.956180 8.206839

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235742 172.9    5709258  305  5709258 305.0
Vcells 11808262  90.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 0.008079 0.0082295 0.0083584 0.0083620 0.0084545 0.009005
2 mean2_no_refine 0.008146 0.0082730 0.0084673 0.0083765 0.0084530 0.017390
4 mean.default+[() 0.011325 0.0115500 0.0118261 0.0116450 0.0118065 0.015502
3 mean+[() 0.013316 0.0136905 0.0144581 0.0138720 0.0141120 0.051134
expr min lq mean median uq max
1 mean2 1.000000 1.000000 1.000000 1.000000 1.0000000 1.000000
2 mean2_no_refine 1.008293 1.005286 1.013034 1.001734 0.9998226 1.931149
4 mean.default+[() 1.401782 1.403487 1.414878 1.392609 1.3964752 1.721488
3 mean+[() 1.648224 1.663588 1.729778 1.658933 1.6691703 5.678401

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235823 172.9    5709258  305  5709258 305.0
Vcells 11810681  90.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 0.015401 0.0155465 0.0157903 0.0156530 0.0157390 0.029470
2 mean2_no_refine 0.015407 0.0155830 0.0156897 0.0156835 0.0158065 0.016155
4 mean.default+[() 0.020783 0.0211130 0.0219483 0.0212785 0.0215335 0.054715
3 mean+[() 0.023065 0.0233810 0.0239698 0.0235295 0.0237565 0.044138
expr min lq mean median uq max
1 mean2 1.000000 1.000000 1.0000000 1.000000 1.000000 1.0000000
2 mean2_no_refine 1.000390 1.002348 0.9936265 1.001948 1.004289 0.5481846
4 mean.default+[() 1.349458 1.358055 1.3899870 1.359388 1.368162 1.8566339
3 mean+[() 1.497630 1.503940 1.5180053 1.503194 1.509403 1.4977265

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 100000 vector

All elements

> x <- data[["n = 100000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235836 172.9    5709258  305  5709258 305.0
Vcells 11810692  90.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 100000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
4 mean.default 0.079641 0.0799895 0.0803480 0.0800865 0.0802345 0.090928
3 mean 0.081914 0.0822575 0.0826470 0.0823535 0.0824645 0.105321
2 mean2_no_refine 0.104137 0.1042020 0.1043205 0.1042390 0.1043350 0.106373
1 mean2 0.104141 0.1042010 0.1045264 0.1042400 0.1043290 0.124704
expr min lq mean median uq max
4 mean.default 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
3 mean 1.028541 1.028354 1.028613 1.028307 1.027794 1.158290
2 mean2_no_refine 1.307580 1.302696 1.298358 1.301580 1.300376 1.169860
1 mean2 1.307630 1.302684 1.300920 1.301593 1.300301 1.371459

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 100000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3235985 172.9    5709258  305  5709258 305.0
Vcells 11817226  90.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.037530 0.0376040 0.0378154 0.0376620 0.0378010 0.048253
1 mean2 0.037525 0.0376205 0.0379368 0.0376650 0.0377360 0.061969
4 mean.default+[() 0.051469 0.0519295 0.0532452 0.0521640 0.0525655 0.104971
3 mean+[() 0.053599 0.0541265 0.0548172 0.0544555 0.0547025 0.074824
expr min lq mean median uq max
2 mean2_no_refine 1.0000000 1.000000 1.000000 1.000000 1.0000000 1.000000
1 mean2 0.9998668 1.000439 1.003209 1.000080 0.9982805 1.284252
4 mean.default+[() 1.3714095 1.380957 1.408028 1.385057 1.3905849 2.175429
3 mean+[() 1.4281641 1.439381 1.449599 1.445900 1.4471178 1.550660

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236066 172.9    5709258  305  5709258 305.0
Vcells 11827802  90.3   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.073510 0.073633 0.0742540 0.0736695 0.0737825 0.120578
1 mean2 0.073459 0.073623 0.0737102 0.0736765 0.0737710 0.074442
4 mean.default+[() 0.097049 0.097575 0.0988462 0.0978705 0.0989045 0.121276
3 mean+[() 0.099286 0.099900 0.1013969 0.1002855 0.1009365 0.131933
expr min lq mean median uq max
2 mean2_no_refine 1.0000000 1.0000000 1.0000000 1.000000 1.0000000 1.0000000
1 mean2 0.9993062 0.9998642 0.9926778 1.000095 0.9998441 0.6173763
4 mean.default+[() 1.3202149 1.3251531 1.3311911 1.328508 1.3404872 1.0057888
3 mean+[() 1.3506462 1.3567286 1.3655416 1.361289 1.3680276 1.0941714

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236147 172.9    5709258  305  5709258 305.0
Vcells 11847856  90.4   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 0.145429 0.1455630 0.1458392 0.1456595 0.1458325 0.151034
2 mean2_no_refine 0.145463 0.1455685 0.1469765 0.1456625 0.1458465 0.210989
4 mean.default+[() 0.189048 0.1899565 0.1995546 0.1904935 0.1911445 0.282780
3 mean+[() 0.191331 0.1924660 0.2076123 0.1931690 0.1967220 0.293314
expr min lq mean median uq max
1 mean2 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 mean2_no_refine 1.000234 1.000038 1.007799 1.000021 1.000096 1.396964
4 mean.default+[() 1.299933 1.304978 1.368320 1.307800 1.310713 1.872294
3 mean+[() 1.315632 1.322218 1.423570 1.326168 1.348959 1.942039

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 100000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 1000000 vector

All elements

> x <- data[["n = 1000000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236160 172.9    5709258  305  5709258 305.0
Vcells 11848060  90.4   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
4 mean.default 0.786145 0.7892925 0.7904622 0.7898235 0.791267 0.814502
3 mean 0.789484 0.7929370 0.7950156 0.7938710 0.795753 0.826365
1 mean2 1.025168 1.0300525 1.0323308 1.0303420 1.032055 1.078533
2 mean2_no_refine 1.004467 1.0301110 1.0315238 1.0304890 1.032379 1.066795
expr min lq mean median uq max
4 mean.default 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
3 mean 1.004247 1.004617 1.005760 1.005125 1.005669 1.014565
1 mean2 1.304044 1.305033 1.305984 1.304522 1.304306 1.324162
2 mean2_no_refine 1.277712 1.305107 1.304963 1.304708 1.304716 1.309751

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236309 172.9    5709258  305  5709258 305.0
Vcells 11908593  90.9   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.365119 0.3783240 0.3926940 0.3874270 0.4087540 0.432756
1 mean2 0.369178 0.3814190 0.3944363 0.3896345 0.4003870 0.568113
4 mean.default+[() 0.511613 0.5561065 0.7117908 0.7692215 0.7890970 0.851525
3 mean+[() 0.517016 0.7588860 0.7335599 0.7762285 0.7968365 0.841731
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.0000000 1.000000
1 mean2 1.011117 1.008181 1.004437 1.005698 0.9795305 1.312779
4 mean.default+[() 1.401223 1.469921 1.812584 1.985462 1.9304936 1.967679
3 mean+[() 1.416020 2.005916 1.868019 2.003548 1.9494280 1.945048

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236390 172.9    5709258  305  5709258 305.0
Vcells 12008649  91.7   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 0.723781 0.7269900 0.7443883 0.7323950 0.7396745 0.983666
2 mean2_no_refine 0.723624 0.7284550 0.7478266 0.7325375 0.7392160 1.297313
4 mean.default+[() 0.967273 0.9789205 1.0765917 0.9902460 1.0600675 1.483031
3 mean+[() 0.970599 0.9862590 1.1480598 0.9992395 1.1701925 7.073463
expr min lq mean median uq max
1 mean2 1.0000000 1.000000 1.000000 1.000000 1.0000000 1.000000
2 mean2_no_refine 0.9997831 1.002015 1.004619 1.000195 0.9993801 1.318855
4 mean.default+[() 1.3364167 1.346539 1.446277 1.352065 1.4331540 1.507657
3 mean+[() 1.3410120 1.356634 1.542286 1.364345 1.5820371 7.190919

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236471 172.9    5709258  305  5709258 305.0
Vcells 12209460  93.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 1.448900 1.454125 1.475360 1.460513 1.468019 2.069696
2 mean2_no_refine 1.449127 1.454113 1.489845 1.460797 1.474863 2.108258
3 mean+[() 1.908811 1.923137 2.391497 1.954579 2.733780 10.532991
4 mean.default+[() 1.895716 1.912886 2.401140 2.190495 2.735460 10.424394
expr min lq mean median uq max
1 mean2 1.000000 1.0000000 1.000000 1.000000 1.000000 1.000000
2 mean2_no_refine 1.000157 0.9999921 1.009818 1.000195 1.004662 1.018632
3 mean+[() 1.317421 1.3225394 1.620958 1.338283 1.862224 5.089149
4 mean.default+[() 1.308383 1.3154898 1.627494 1.499813 1.863368 5.036679

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 1000000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 10000000 vector

All elements

> x <- data[["n = 10000000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236484 172.9    5709258  305  5709258 305.0
Vcells 12209033  93.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
4 mean.default 7.838175 8.027278 8.090957 8.046798 8.062791 12.75987
3 mean 7.845059 8.041952 8.099937 8.062498 8.084361 11.69292
2 mean2_no_refine 10.151019 10.443252 10.466833 10.469959 10.477272 11.38110
1 mean2 10.174916 10.453464 10.458192 10.471350 10.481231 11.15226
expr min lq mean median uq max
4 mean.default 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
3 mean 1.000878 1.001828 1.001110 1.001951 1.002675 0.9163828
2 mean2_no_refine 1.295074 1.300970 1.293646 1.301133 1.299460 0.8919446
1 mean2 1.298123 1.302243 1.292578 1.301306 1.299951 0.8740105

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236633 172.9    5709258  305  5709258 305.0
Vcells 12809566  97.8   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 4.323749 4.378862 4.500883 4.400150 4.499671 5.291595
1 mean2 4.336977 4.380094 4.508883 4.408432 4.538824 5.412560
4 mean.default+[() 6.434353 8.298769 9.269994 8.439051 8.978974 21.420245
3 mean+[() 6.442737 8.317103 9.132680 8.445161 8.762881 22.056313
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.003059 1.000282 1.001777 1.001882 1.008701 1.022860
4 mean.default+[() 1.488142 1.895189 2.059595 1.917901 1.995473 4.047975
3 mean+[() 1.490081 1.899376 2.029087 1.919290 1.947449 4.168179

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236714 172.9    5709258  305  5709258 305.0
Vcells 13810522 105.4   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 mean2 7.550001 7.574736 7.699777 7.588044 7.611163 9.826406
2 mean2_no_refine 7.541923 7.571379 7.660771 7.589351 7.616875 9.472421
4 mean.default+[() 10.923186 14.642793 15.691470 14.743345 14.992078 29.411238
3 mean+[() 10.911724 14.633787 17.767953 14.763592 15.058673 279.960072
expr min lq mean median uq max
1 mean2 1.0000000 1.0000000 1.0000000 1.000000 1.000000 1.0000000
2 mean2_no_refine 0.9989301 0.9995569 0.9949341 1.000172 1.000750 0.9639761
4 mean.default+[() 1.4467794 1.9331095 2.0379123 1.942970 1.969749 2.9930819
3 mean+[() 1.4452613 1.9319205 2.3075934 1.945639 1.978498 28.4905867

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236792 172.9    5709258  305  5709258 305.0
Vcells 15810571 120.7   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 14.58030 14.69894 14.97032 14.72901 14.87608 27.72130
1 mean2 14.55143 14.70976 14.92888 14.77023 14.92913 18.13128
4 mean.default+[() 27.41324 27.63234 36.26475 27.92353 36.05713 295.96439
3 mean+[() 27.57888 27.86948 34.87476 27.99913 38.46810 296.02413
expr min lq mean median uq max
2 mean2_no_refine 1.0000000 1.000000 1.0000000 1.000000 1.000000 1.000000
1 mean2 0.9980203 1.000736 0.9972316 1.002799 1.003566 0.654056
4 mean.default+[() 1.8801563 1.879886 2.4224422 1.895819 2.423832 10.676424
3 mean+[() 1.8915170 1.896019 2.3295929 1.900952 2.585902 10.678579

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on integer+n = 10000000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rvector <- function(n, mode = c("logical", "double", "integer"), range = c(-100, +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     x
+ }
> rvectors <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rvector(n = scale * 100, ...)
+     data[[2]] <- rvector(n = scale * 1000, ...)
+     data[[3]] <- rvector(n = scale * 10000, ...)
+     data[[4]] <- rvector(n = scale * 1e+05, ...)
+     data[[5]] <- rvector(n = scale * 1e+06, ...)
+     names(data) <- sprintf("n = %d", sapply(data, FUN = length))
+     data
+ }
> data <- rvectors(mode = mode)

Results

n = 1000 vector

All elements

> x <- data[["n = 1000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236808 172.9    5709258  305  5709258 305.0
Vcells 21365649 163.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.001716 0.0017420 0.0018221 0.0017765 0.0018495 0.002486
4 mean.default 0.002271 0.0023340 0.0024352 0.0023695 0.0024610 0.005381
1 mean2 0.002752 0.0027915 0.0029928 0.0028320 0.0029040 0.015064
3 mean 0.003761 0.0038445 0.0041719 0.0039075 0.0040475 0.024380
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
4 mean.default 1.323427 1.339839 1.336491 1.333802 1.330630 2.164521
1 mean2 1.603730 1.602468 1.642522 1.594146 1.570154 6.059533
3 mean 2.191725 2.206946 2.289594 2.199550 2.188429 9.806919

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3236957 172.9    5709258  305  5709258 305.0
Vcells 17367366 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.001116 0.0011850 0.0015905 0.0012985 0.0013785 0.027222
1 mean2 0.001332 0.0014060 0.0015384 0.0015205 0.0016235 0.002200
4 mean.default+[() 0.002196 0.0024480 0.0027586 0.0025645 0.0027325 0.017722
3 mean+[() 0.004281 0.0044515 0.0048223 0.0045720 0.0047375 0.026206
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.0000000 1.000000 1.000000 1.0000000
1 mean2 1.193548 1.186498 0.9672118 1.170967 1.177729 0.0808170
4 mean.default+[() 1.967742 2.065823 1.7344059 1.974971 1.982227 0.6510176
3 mean+[() 3.836022 3.756540 3.0319017 3.520986 3.436707 0.9626772

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237038 172.9    5709258  305  5709258 305.0
Vcells 17367522 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.001498 0.0015725 0.0016954 0.0016760 0.0017665 0.002954
1 mean2 0.001916 0.0019735 0.0022293 0.0020475 0.0021570 0.016848
4 mean.default+[() 0.002896 0.0030865 0.0032503 0.0032045 0.0033220 0.005748
3 mean+[() 0.004805 0.0051075 0.0055593 0.0052530 0.0054005 0.033333
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.279039 1.255008 1.314915 1.221659 1.221059 5.703453
4 mean.default+[() 1.933244 1.962798 1.917123 1.911993 1.880555 1.945836
3 mean+[() 3.207610 3.248013 3.279053 3.134248 3.057175 11.284022

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 1000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237119 172.9    5709258  305  5709258 305.0
Vcells 17367776 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.002210 0.0023095 0.0024092 0.0023870 0.0024695 0.003308
1 mean2 0.003052 0.0031350 0.0033802 0.0032550 0.0033345 0.014260
4 mean.default+[() 0.004130 0.0044505 0.0049668 0.0045410 0.0046905 0.022495
3 mean+[() 0.006186 0.0064710 0.0068803 0.0066005 0.0067905 0.027571
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.380996 1.357437 1.403032 1.363636 1.350273 4.310762
4 mean.default+[() 1.868778 1.927040 2.061622 1.902388 1.899372 6.800181
3 mean+[() 2.799095 2.801905 2.855856 2.765186 2.749747 8.334643

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 10000 vector

All elements

> x <- data[["n = 10000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237132 172.9    5709258  305  5709258 305.0
Vcells 17367349 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.011101 0.0112655 0.0114860 0.0113290 0.0114125 0.025110
4 mean.default 0.016336 0.0164650 0.0165716 0.0165110 0.0165920 0.019920
3 mean 0.018064 0.0182985 0.0186318 0.0184075 0.0185445 0.038576
1 mean2 0.021431 0.0215550 0.0216686 0.0216180 0.0217070 0.024017
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
4 mean.default 1.471579 1.461542 1.442760 1.457410 1.453844 0.7933094
3 mean 1.627241 1.624295 1.622123 1.624812 1.624929 1.5362804
1 mean2 1.930547 1.913364 1.886513 1.908200 1.902037 0.9564715

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237281 172.9    5709258  305  5709258 305.0
Vcells 17369779 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.004530 0.0046335 0.0047866 0.0047750 0.0048600 0.005406
1 mean2 0.006605 0.0067270 0.0070908 0.0068555 0.0069750 0.021635
4 mean.default+[() 0.008672 0.0090210 0.0093464 0.0091230 0.0093920 0.014210
3 mean+[() 0.010983 0.0113345 0.0128229 0.0115885 0.0118505 0.075978
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.458057 1.451818 1.481377 1.435707 1.435185 4.002035
4 mean.default+[() 1.914349 1.946908 1.952618 1.910576 1.932510 2.628561
3 mean+[() 2.424503 2.446207 2.678909 2.426911 2.438375 14.054384

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237362 172.9    5709258  305  5709258 305.0
Vcells 17370835 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.008216 0.0083835 0.0086482 0.0084560 0.0085430 0.026035
1 mean2 0.012365 0.0125090 0.0126900 0.0126325 0.0127325 0.015401
4 mean.default+[() 0.015329 0.0158385 0.0163284 0.0160000 0.0162725 0.029203
3 mean+[() 0.017643 0.0181820 0.0189174 0.0183485 0.0185935 0.057063
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 mean2 1.504990 1.492098 1.467362 1.493910 1.490401 0.5915498
4 mean.default+[() 1.865750 1.889247 1.888076 1.892148 1.904776 1.1216824
3 mean+[() 2.147395 2.168784 2.187444 2.169879 2.176460 2.1917803

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 10000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237443 172.9    5709258  305  5709258 305.0
Vcells 17372889 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.015579 0.0156895 0.0158413 0.0158020 0.0159090 0.018410
1 mean2 0.023814 0.0239480 0.0241550 0.0240315 0.0241345 0.035644
4 mean.default+[() 0.028507 0.0294860 0.0303152 0.0297440 0.0300290 0.060273
3 mean+[() 0.030630 0.0318755 0.0323503 0.0321665 0.0324370 0.052719
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.528596 1.526371 1.524809 1.520789 1.517034 1.936122
4 mean.default+[() 1.829835 1.879346 1.913677 1.882293 1.887548 3.273927
3 mean+[() 1.966108 2.031645 2.042148 2.035597 2.038909 2.863607

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 100000 vector

All elements

> x <- data[["n = 100000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237456 172.9    5709258  305  5709258 305.0
Vcells 17372462 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 100000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.104148 0.1042345 0.1044468 0.1042690 0.1043975 0.106605
4 mean.default 0.155528 0.1556080 0.1558212 0.1556980 0.1557955 0.158118
3 mean 0.157820 0.1580230 0.1583918 0.1581295 0.1582500 0.179802
1 mean2 0.207014 0.2070770 0.2074356 0.2071165 0.2071970 0.227233
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
4 mean.default 1.493336 1.492865 1.491873 1.493234 1.492330 1.483214
3 mean 1.515343 1.516034 1.516484 1.516553 1.515841 1.686619
1 mean2 1.987691 1.986646 1.986042 1.986367 1.984693 2.131542

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 100000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237605 173.0    5709258  305  5709258 305.0
Vcells 17378996 132.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.038247 0.0383865 0.0385506 0.0384785 0.0385855 0.041967
1 mean2 0.059953 0.0600960 0.0608196 0.0601635 0.0602835 0.118204
4 mean.default+[() 0.073587 0.0743345 0.0764475 0.0747785 0.0754460 0.158558
3 mean+[() 0.075907 0.0770315 0.0789902 0.0774320 0.0784345 0.104829
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.567522 1.565550 1.577656 1.563562 1.562336 2.816594
4 mean.default+[() 1.923994 1.936475 1.983041 1.943384 1.955294 3.778159
3 mean+[() 1.984652 2.006734 2.048998 2.012345 2.032745 2.497891

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237686 173.0    5709258  305  5709258 305.0
Vcells 17390612 132.7   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.073593 0.0736945 0.0746640 0.0737765 0.0738880 0.138898
1 mean2 0.114935 0.1150195 0.1155257 0.1150885 0.1152495 0.142700
4 mean.default+[() 0.138086 0.1407560 0.1420396 0.1415605 0.1423410 0.167275
3 mean+[() 0.140265 0.1436130 0.1453005 0.1443560 0.1456825 0.170358
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.561765 1.560761 1.547274 1.559962 1.559786 1.027373
4 mean.default+[() 1.876347 1.909993 1.902384 1.918775 1.926443 1.204301
3 mean+[() 1.905956 1.948761 1.946058 1.956666 1.971667 1.226497

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 100000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237767 173.0    5709258  305  5709258 305.0
Vcells 17410666 132.9   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.145511 0.1456025 0.1536907 0.1458415 0.1471675 0.378968
1 mean2 0.227776 0.2279420 0.2312377 0.2281725 0.2289890 0.357522
4 mean.default+[() 0.258598 0.2767635 0.3536514 0.2889915 0.4310290 0.728359
3 mean+[() 0.261920 0.2794995 0.3521816 0.2892785 0.4352335 0.489634
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 mean2 1.565352 1.565509 1.504566 1.564524 1.555975 0.9434095
4 mean.default+[() 1.777171 1.900816 2.301059 1.981545 2.928833 1.9219538
3 mean+[() 1.800001 1.919606 2.291496 1.983513 2.957402 1.2920194

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 100000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 1000000 vector

All elements

> x <- data[["n = 1000000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237780 173.0    5709258  305  5709258 305.0
Vcells 17410239 132.9   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 1.026754 1.051454 1.056996 1.053252 1.058614 1.114728
4 mean.default 1.553472 1.594251 1.610358 1.597915 1.606791 2.191001
3 mean 1.578921 1.608677 1.617133 1.614062 1.619892 1.758759
1 mean2 2.074304 2.097707 2.106379 2.104033 2.109192 2.167240
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
4 mean.default 1.512993 1.516235 1.523522 1.517125 1.517825 1.965503
3 mean 1.537779 1.529955 1.529932 1.532455 1.530201 1.577747
1 mean2 2.020254 1.995054 1.992797 1.997654 1.992409 1.944187

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 1000000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3237929 173.0    5709258  305  5709258 305.0
Vcells 17470772 133.3   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.478477 0.590530 0.6289313 0.6343935 0.6685945 0.977233
1 mean2 0.837596 1.017808 1.0559042 1.0646500 1.1080095 1.224932
4 mean.default+[() 0.889853 1.505241 1.6474787 1.5413070 1.5781900 14.127943
3 mean+[() 0.893893 1.522968 1.6783895 1.5526815 1.5994640 13.921335
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
1 mean2 1.750546 1.723550 1.678886 1.678217 1.657222 1.25347
4 mean.default+[() 1.859761 2.548966 2.619489 2.429576 2.360459 14.45709
3 mean+[() 1.868205 2.578986 2.668637 2.447505 2.392278 14.24567

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238010 173.0    5709258  305  5709258 305.0
Vcells 17570828 134.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 0.812045 0.8609835 0.9198856 0.9050885 0.9483435 1.443393
1 mean2 1.303331 1.3698990 1.4474710 1.4304645 1.4754555 2.345792
4 mean.default+[() 1.633059 2.3182440 2.7593362 2.3722785 2.4915760 13.780499
3 mean+[() 1.970608 2.3483825 2.5528565 2.4054115 2.4512435 13.615577
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.604998 1.591086 1.573534 1.580469 1.555824 1.625193
4 mean.default+[() 2.011045 2.692553 2.999651 2.621046 2.627293 9.547295
3 mean+[() 2.426723 2.727558 2.775189 2.657653 2.584763 9.433035

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 1000000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238091 173.0    5709258  305  5709258 305.0
Vcells 17770882 135.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 1.492015 1.505714 1.553038 1.527435 1.542582 1.869643
1 mean2 2.343350 2.363042 2.444962 2.396588 2.426274 3.024941
3 mean+[() 2.847850 4.294592 5.083393 4.362610 4.468501 16.288315
4 mean.default+[() 2.837037 4.305288 4.652314 4.422275 4.497725 14.846236
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.570594 1.569383 1.574310 1.569028 1.572865 1.617924
3 mean+[() 1.908727 2.852196 3.273194 2.856167 2.896766 8.711992
4 mean.default+[() 1.901480 2.859300 2.995622 2.895230 2.915712 7.940680

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 1000000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

n = 10000000 vector

All elements

> x <- data[["n = 10000000"]]
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238104 173.0    5709258  305  5709258 305.0
Vcells 17770455 135.6   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, refine = TRUE), mean2_no_refine = mean2(x, refine = FALSE), 
+     mean = mean(x), mean.default = mean.default(x), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000000+all data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 9.843419 10.50587 10.50763 10.51808 10.53710 12.21150
4 mean.default 15.060048 15.96673 16.00241 15.97704 15.99398 16.90293
3 mean 15.642657 15.98317 16.02730 15.99309 16.00777 18.10020
1 mean2 19.696074 20.95110 20.95150 21.02333 21.04277 21.73061
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
4 mean.default 1.529961 1.519791 1.522933 1.519008 1.517873 1.384181
3 mean 1.589149 1.521356 1.525301 1.520533 1.519181 1.482226
1 mean2 2.000938 1.994228 1.993933 1.998781 1.997016 1.779520

Figure: Benchmarking of mean2(), mean2_no_refine(), mean() and mean.default() on n = 10000000+all data. Outliers are displayed as crosses. Times are in milliseconds.

A 20% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.2
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238253 173.0    5709258  305  5709258 305.0
Vcells 18372857 140.2   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.2 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 6.369286 6.667874 6.829916 6.825528 6.937186 7.48577
1 mean2 11.112320 11.645808 11.874461 11.771455 11.946345 15.70384
4 mean.default+[() 10.565307 16.034094 18.237393 16.562583 17.300082 30.12160
3 mean+[() 12.679728 16.201599 17.512428 16.699506 17.161988 28.88841
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.744673 1.746555 1.738595 1.724622 1.722074 2.097826
4 mean.default+[() 1.658790 2.404679 2.670222 2.426564 2.493819 4.023848
3 mean+[() 1.990761 2.429800 2.564077 2.446625 2.473912 3.859110

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.2 data. Outliers are displayed as crosses. Times are in milliseconds.

A 40% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.4
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238325 173.0    5709258  305  5709258 305.0
Vcells 19372898 147.9   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.4 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 8.65734 8.711063 8.969746 8.755019 9.03013 10.30212
1 mean2 14.24664 14.307604 14.694644 14.346621 14.68712 19.78128
3 mean+[() 16.96485 23.945018 34.132319 25.419415 35.89771 289.60112
4 mean.default+[() 23.77888 24.251441 31.447632 25.419867 35.32692 287.64227
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.645614 1.642464 1.638245 1.638674 1.626457 1.920117
3 mean+[() 1.959591 2.748806 3.805271 2.903411 3.975326 28.110823
4 mean.default+[() 2.746673 2.783982 3.505967 2.903462 3.912116 27.920682

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.4 data. Outliers are displayed as crosses. Times are in milliseconds.

A 80% subset

> x <- data[["n = 10000000"]]
> subset
[1] 0.8
> idxs <- sort(sample(length(x), size = subset * length(x), replace = FALSE))
> gc()
           used  (Mb) gc trigger (Mb) max used  (Mb)
Ncells  3238412 173.0    5709258  305  5709258 305.0
Vcells 21372962 163.1   37485149  286 87357391 666.5
> stats <- microbenchmark(mean2 = mean2(x, idxs = idxs, refine = TRUE), mean2_no_refine = mean2(x, 
+     idxs = idxs, refine = FALSE), `mean+[()` = mean2_R_v1(x, idxs = idxs), `mean.default+[()` = mean2_R_v2(x, 
+     idxs = idxs), unit = "ms")

Table: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.8 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 mean2_no_refine 15.08422 15.27005 15.88685 15.88614 16.36884 18.64600
1 mean2 23.90745 24.43465 25.22848 25.28174 25.91634 26.79744
3 mean+[() 37.11935 44.55973 63.80948 53.92368 59.40594 312.33525
4 mean.default+[() 30.28500 44.12684 74.11212 53.98984 66.86120 312.48600
expr min lq mean median uq max
2 mean2_no_refine 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 mean2 1.584931 1.600168 1.588010 1.591434 1.583272 1.437168
3 mean+[() 2.460807 2.918112 4.016497 3.394385 3.629208 16.750788
4 mean.default+[() 2.007727 2.889763 4.664998 3.398550 4.084663 16.758873

Figure: Benchmarking of mean2(), mean2_no_refine(), mean+[()() and mean.default+[()() on double+n = 10000000+0.8 data. Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 1.91 mins.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('mean2')

Copyright Henrik Bengtsson. Last updated on 2019-09-10 21:08:46 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer

Benchmark reports

Clone this wiki locally