Skip to content

colRowCummins

Jump to bottom Edit New page
Henrik Bengtsson edited this page Sep 11, 2019 · 6 revisions

matrixStats: Benchmark report

colCummins() and rowCummins() benchmarks

This report benchmark the performance of colCummins() and rowCummins() against alternative methods.

Alternative methods

  • apply() + cummin()

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094809 165.3    5709258 305.0  5709258 305.0
Vcells 5749188  43.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093535 165.3    5709258 305.0  5709258 305.0
Vcells 5745576  43.9   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.001104 0.0013695 0.0017350 0.001548 0.001846 0.011595
2 apply+cummin 0.024364 0.0253780 0.0269594 0.025588 0.025956 0.127639
expr min lq mean median uq max
1 colCummins 1.00000 1.00000 1.0000 1.00000 1.00000 1.00000
2 apply+cummin 22.06884 18.53085 15.5388 16.52972 14.06067 11.00811

Table: Benchmarking of rowCummins() and apply+cummin() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.001038 0.0016775 0.0023967 0.002035 0.002758 0.012162
2 apply+cummin 0.025164 0.0258005 0.0356321 0.026401 0.040775 0.136773
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 24.24277 15.38033 14.86709 12.97346 14.78426 11.24593

Figure: Benchmarking of colCummins() and apply+cummin() on integer+10x10 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 1.104 1.3695 1.73497 1.548 1.846 11.595
2 rowCummins 1.038 1.6775 2.39671 2.035 2.758 12.162
expr min lq mean median uq max
1 colCummins 1.0000000 1.0000 1.000000 1.000000 1.000000 1.0000
2 rowCummins 0.9402174 1.2249 1.381413 1.314599 1.494041 1.0489

Figure: Benchmarking of colCummins() and rowCummins() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3092090 165.2    5709258 305.0  5709258 305.0
Vcells 5362067  41.0   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3092084 165.2    5709258 305.0  5709258 305.0
Vcells 5367110  41.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.016729 0.0173910 0.0188918 0.017871 0.0183840 0.044086
2 apply+cummin 0.163873 0.1665145 0.1830672 0.171454 0.1831925 0.332822
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+cummin 9.795744 9.574751 9.690283 9.593979 9.964779 7.549381

Table: Benchmarking of rowCummins() and apply+cummin() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.012460 0.012819 0.0134192 0.0131660 0.0135165 0.022524
2 apply+cummin 0.166404 0.168387 0.1737151 0.1689845 0.1706210 0.336706
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 13.35506 13.13574 12.94522 12.83492 12.62316 14.94877

Figure: Benchmarking of colCummins() and apply+cummin() on integer+100x100 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 12.460 12.819 13.41924 13.166 13.5165 22.524
1 colCummins 16.729 17.391 18.89183 17.871 18.3840 44.086
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.00000 1.000000 1.00000
1 colCummins 1.342616 1.356658 1.407817 1.35736 1.360115 1.95729

Figure: Benchmarking of colCummins() and rowCummins() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3092823 165.2    5709258 305.0  5709258 305.0
Vcells 5365579  41.0   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3092814 165.2    5709258 305.0  5709258 305.0
Vcells 5370617  41.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.017532 0.017917 0.0202069 0.0183240 0.0186820 0.123854
2 apply+cummin 0.097569 0.101966 0.1185105 0.1028265 0.1051975 0.562017
expr min lq mean median uq max
1 colCummins 1.000000 1.00000 1.000000 1.000000 1.000000 1.000000
2 apply+cummin 5.565195 5.69102 5.864844 5.611575 5.630955 4.537738

Table: Benchmarking of rowCummins() and apply+cummin() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.013103 0.0137600 0.0192005 0.0141275 0.014651 0.114058
2 apply+cummin 0.097622 0.1017285 0.1150768 0.1029030 0.104333 0.567213
expr min lq mean median uq max
1 rowCummins 1.000000 1.00000 1.000000 1.000000 1.00000 1.000000
2 apply+cummin 7.450355 7.39306 5.993428 7.283879 7.12122 4.973022

Figure: Benchmarking of colCummins() and apply+cummin() on integer+1000x10 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 13.103 13.760 19.20050 14.1275 14.651 114.058
1 colCummins 17.532 17.917 20.20693 18.3240 18.682 123.854
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colCummins 1.338014 1.302108 1.052417 1.297045 1.275135 1.085886

Figure: Benchmarking of colCummins() and rowCummins() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093009 165.2    5709258 305.0  5709258 305.0
Vcells 5366262  41.0   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093003 165.2    5709258 305.0  5709258 305.0
Vcells 5371305  41.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.013583 0.0152280 0.0205396 0.0165205 0.0207220 0.099007
2 apply+cummin 0.758790 0.8367295 0.8983289 0.8579140 0.8898655 1.708476
expr min lq mean median uq max
1 colCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 55.86321 54.94678 43.73648 51.93027 42.94303 17.25611

Table: Benchmarking of rowCummins() and apply+cummin() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.012011 0.0131105 0.0140269 0.013643 0.0141655 0.027034
2 apply+cummin 0.740832 0.8246865 0.8358010 0.842213 0.8594920 0.912199
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 61.67946 62.90275 59.58541 61.73224 60.67502 33.74266

Figure: Benchmarking of colCummins() and apply+cummin() on integer+10x1000 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 12.011 13.1105 14.02694 13.6430 14.1655 27.034
1 colCummins 13.583 15.2280 20.53958 16.5205 20.7220 99.007
expr min lq mean median uq max
2 rowCummins 1.00000 1.000000 1.000000 1.000000 1.00000 1.000000
1 colCummins 1.13088 1.161512 1.464295 1.210914 1.46285 3.662314

Figure: Benchmarking of colCummins() and rowCummins() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093193 165.2    5709258 305.0  5709258 305.0
Vcells 5366745  41.0   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093184 165.2    5709258 305.0  5709258 305.0
Vcells 5416783  41.4   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.158854 0.167977 0.1750069 0.1740525 0.1790625 0.222749
2 apply+cummin 1.454637 1.568151 1.8673674 1.6085750 1.6323595 13.776363
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.00000 1.000000 1.000000 1.00000
2 apply+cummin 9.157069 9.335513 10.67025 9.241895 9.116144 61.84703

Table: Benchmarking of rowCummins() and apply+cummin() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.105732 0.114414 0.1213062 0.1178415 0.1232105 0.263937
2 apply+cummin 1.492538 1.573149 1.9593756 1.6149460 1.7454645 15.716241
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 14.11624 13.74962 16.15231 13.70439 14.16652 59.54543

Figure: Benchmarking of colCummins() and apply+cummin() on integer+100x1000 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 105.732 114.414 121.3062 117.8415 123.2105 263.937
1 colCummins 158.854 167.977 175.0069 174.0525 179.0625 222.749
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colCummins 1.502421 1.468151 1.442687 1.477005 1.453305 0.8439476

Figure: Benchmarking of colCummins() and rowCummins() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093386 165.3    5709258 305.0  5709258 305.0
Vcells 5367315  41.0   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093380 165.3    5709258 305.0  5709258 305.0
Vcells 5417358  41.4   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.162515 0.1788920 0.1856476 0.1839145 0.1947555 0.240225
2 apply+cummin 0.820809 0.8661755 1.0246917 0.8842115 0.9292215 6.773607
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 apply+cummin 5.050666 4.841891 5.519552 4.807731 4.771221 28.19693

Table: Benchmarking of rowCummins() and apply+cummin() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.11277 0.1201740 0.1315573 0.1256540 0.138152 0.212913
2 apply+cummin 0.84190 0.9042695 1.1115397 0.9316595 1.070702 6.950486
expr min lq mean median uq max
1 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 apply+cummin 7.465638 7.524668 8.449092 7.414483 7.750177 32.64472

Figure: Benchmarking of colCummins() and apply+cummin() on integer+1000x100 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 112.770 120.174 131.5573 125.6540 138.1520 212.913
1 colCummins 162.515 178.892 185.6476 183.9145 194.7555 240.225
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colCummins 1.441119 1.488608 1.411154 1.463658 1.409719 1.128278

Figure: Benchmarking of colCummins() and rowCummins() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), na_prob = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         x <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         x <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         x <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(x) <- mode
+     if (na_prob > 0) 
+         x[sample(n, size = na_prob * n)] <- NA
+     dim(x) <- c(nrow, ncol)
+     x
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093594 165.3    5709258 305.0  5709258 305.0
Vcells 5483033  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093579 165.3    5709258 305.0  5709258 305.0
Vcells 5483161  41.9   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.000989 0.0014075 0.0018475 0.0016745 0.0020315 0.010712
2 apply+cummin 0.025063 0.0258085 0.0276814 0.0262400 0.0266105 0.127135
expr min lq mean median uq max
1 colCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 25.34176 18.33641 14.98352 15.67035 13.09894 11.86847

Table: Benchmarking of rowCummins() and apply+cummin() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.000998 0.001470 0.0018586 0.001811 0.0019680 0.013349
2 apply+cummin 0.024260 0.025191 0.0266444 0.025475 0.0258035 0.126610
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.000000
2 apply+cummin 24.30862 17.13673 14.33566 14.06681 13.11153 9.484606

Figure: Benchmarking of colCummins() and apply+cummin() on double+10x10 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 989 1407.5 1847.46 1674.5 2031.5 10712
2 rowCummins 998 1470.0 1858.61 1811.0 1968.0 13349
expr min lq mean median uq max
1 colCummins 1.0000 1.000000 1.000000 1.000000 1.0000000 1.000000
2 rowCummins 1.0091 1.044405 1.006035 1.081517 0.9687423 1.246172

Figure: Benchmarking of colCummins() and rowCummins() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093772 165.3    5709258 305.0  5709258 305.0
Vcells 5483858  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093766 165.3    5709258 305.0  5709258 305.0
Vcells 5493901  42.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.022917 0.0233255 0.0244423 0.0238125 0.0242985 0.061007
2 apply+cummin 0.205522 0.2091620 0.2179182 0.2148140 0.2176295 0.355906
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.000000 1.00000 1.000000 1.000000
2 apply+cummin 8.968102 8.967096 8.915618 9.02106 8.956499 5.833855

Table: Benchmarking of rowCummins() and apply+cummin() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.010483 0.011007 0.0117950 0.0113605 0.0118355 0.020724
2 apply+cummin 0.164181 0.167555 0.1827565 0.1686450 0.1707395 0.540866
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 15.66164 15.22259 15.49439 14.84486 14.42605 26.09853

Figure: Benchmarking of colCummins() and apply+cummin() on double+100x100 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 10.483 11.0070 11.79501 11.3605 11.8355 20.724
1 colCummins 22.917 23.3255 24.44230 23.8125 24.2985 61.007
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colCummins 2.186111 2.119151 2.072258 2.096078 2.053018 2.943785

Figure: Benchmarking of colCummins() and rowCummins() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093962 165.3    5709258 305.0  5709258 305.0
Vcells 5484756  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3093956 165.3    5709258 305.0  5709258 305.0
Vcells 5494799  42.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.024701 0.025327 0.0322288 0.026026 0.029233 0.318618
2 apply+cummin 0.142403 0.144577 0.1739306 0.148498 0.173823 0.645888
expr min lq mean median uq max
1 colCummins 1.00000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+cummin 5.76507 5.708414 5.396751 5.705756 5.946123 2.027155

Table: Benchmarking of rowCummins() and apply+cummin() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.011523 0.0123295 0.0134754 0.012906 0.013494 0.045830
2 apply+cummin 0.101787 0.1032515 0.1087779 0.105236 0.108124 0.201244
expr min lq mean median uq max
1 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
2 apply+cummin 8.833377 8.374346 8.072336 8.154037 8.012746 4.391097

Figure: Benchmarking of colCummins() and apply+cummin() on double+1000x10 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 11.523 12.3295 13.47539 12.906 13.494 45.830
1 colCummins 24.701 25.3270 32.22877 26.026 29.233 318.618
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.00000 1.000000
1 colCummins 2.143626 2.054179 2.391676 2.016581 2.16637 6.952171

Figure: Benchmarking of colCummins() and rowCummins() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094151 165.3    5709258 305.0  5709258 305.0
Vcells 5484882  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094145 165.3    5709258 305.0  5709258 305.0
Vcells 5494925  42.0   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.013835 0.015416 0.0177544 0.0165575 0.0189475 0.031691
2 apply+cummin 0.742210 0.818772 0.8809942 0.8505135 0.8700975 1.631995
expr min lq mean median uq max
1 colCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
2 apply+cummin 53.64727 53.11183 49.62106 51.36727 45.92149 51.49711

Table: Benchmarking of rowCummins() and apply+cummin() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.010761 0.0122055 0.0132655 0.0128365 0.0135120 0.022581
2 apply+cummin 0.740283 0.8152195 0.8307643 0.8303000 0.8441475 0.974474
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.00000 1.00000 1.00000 1.0000
2 apply+cummin 68.79314 66.79116 62.62603 64.68274 62.47391 43.1546

Figure: Benchmarking of colCummins() and apply+cummin() on double+10x1000 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 10.761 12.2055 13.26548 12.8365 13.5120 22.581
1 colCummins 13.835 15.4160 17.75444 16.5575 18.9475 31.691
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
1 colCummins 1.285661 1.263037 1.338394 1.289877 1.402272 1.403436

Figure: Benchmarking of colCummins() and rowCummins() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094335 165.3    5709258 305.0  5709258 305.0
Vcells 5485934  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094326 165.3    5709258 305.0  5709258 305.0
Vcells 5585972  42.7   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.215863 0.225439 0.2463868 0.2402555 0.260260 0.344222
2 apply+cummin 1.877418 1.977949 2.4935509 2.1007760 2.409504 13.640032
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.00000 1.000000 1.000000 1.00000
2 apply+cummin 8.697266 8.773764 10.12047 8.743925 9.258065 39.62568

Table: Benchmarking of rowCummins() and apply+cummin() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.088136 0.102374 0.2194747 0.111642 0.1239655 10.65944
2 apply+cummin 1.488863 1.670075 2.1552832 1.889243 2.0837420 12.44535
expr min lq mean median uq max
1 rowCummins 1.00000 1.00000 1.000000 1.00000 1.00000 1.000000
2 apply+cummin 16.89279 16.31347 9.820189 16.92233 16.80905 1.167543

Figure: Benchmarking of colCummins() and apply+cummin() on double+100x1000 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 88.136 102.374 219.4747 111.6420 123.9655 10659.441
1 colCummins 215.863 225.439 246.3868 240.2555 260.2600 344.222
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.00000 1.000000 1.000000 1.0000000
1 colCummins 2.449203 2.202112 1.12262 2.152017 2.099455 0.0322927

Figure: Benchmarking of colCummins() and rowCummins() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094528 165.3    5709258 305.0  5709258 305.0
Vcells 5486062  41.9   22267496 169.9 56666022 432.4
> colStats <- microbenchmark(colCummins = colCummins(X), `apply+cummin` = apply(X, MARGIN = 2L, FUN = cummin), 
+     unit = "ms")
> X <- t(X)
> gc()
          used  (Mb) gc trigger  (Mb) max used  (Mb)
Ncells 3094522 165.3    5709258 305.0  5709258 305.0
Vcells 5586105  42.7   22267496 169.9 56666022 432.4
> rowStats <- microbenchmark(rowCummins = rowCummins(X), `apply+cummin` = apply(X, MARGIN = 1L, FUN = cummin), 
+     unit = "ms")

Table: Benchmarking of colCummins() and apply+cummin() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colCummins 0.234394 0.2622935 0.2848296 0.281013 0.2960725 0.562482
2 apply+cummin 0.845419 0.9405685 1.1813183 1.023181 1.0878530 6.278719
expr min lq mean median uq max
1 colCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.00000
2 apply+cummin 3.606829 3.585939 4.147457 3.641043 3.674279 11.16252

Table: Benchmarking of rowCummins() and apply+cummin() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 rowCummins 0.091668 0.0989035 0.1663113 0.1115535 0.116757 5.822179
2 apply+cummin 0.874420 0.9913885 1.1461032 1.0478420 1.108747 6.220861
expr min lq mean median uq max
1 rowCummins 1.000000 1.0000 1.000000 1.000000 1.000000 1.000000
2 apply+cummin 9.538989 10.0238 6.891311 9.393179 9.496197 1.068476

Figure: Benchmarking of colCummins() and apply+cummin() on double+1000x100 data as well as rowCummins() and apply+cummin() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colCummins() and rowCummins() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowCummins 91.668 98.9035 166.3113 111.5535 116.7570 5822.179
1 colCummins 234.394 262.2935 284.8295 281.0130 296.0725 562.482
expr min lq mean median uq max
2 rowCummins 1.000000 1.000000 1.000000 1.000000 1.000000 1.0000000
1 colCummins 2.556988 2.652014 1.712629 2.519087 2.535801 0.0966102

Figure: Benchmarking of colCummins() and rowCummins() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R version 3.6.1 Patched (2019-08-27 r77078)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/R-3-6-branch/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] microbenchmark_1.4-6    matrixStats_0.55.0-9000 ggplot2_3.2.1          
[4] knitr_1.24              R.devices_2.16.0        R.utils_2.9.0          
[7] R.oo_1.22.0             R.methodsS3_1.7.1       history_0.0.0-9002     

loaded via a namespace (and not attached):
 [1] Biobase_2.45.0       bit64_0.9-7          splines_3.6.1       
 [4] network_1.15         assertthat_0.2.1     highr_0.8           
 [7] stats4_3.6.1         blob_1.2.0           robustbase_0.93-5   
[10] pillar_1.4.2         RSQLite_2.1.2        backports_1.1.4     
[13] lattice_0.20-38      glue_1.3.1           digest_0.6.20       
[16] colorspace_1.4-1     sandwich_2.5-1       Matrix_1.2-17       
[19] XML_3.98-1.20        lpSolve_5.6.13.3     pkgconfig_2.0.2     
[22] genefilter_1.66.0    purrr_0.3.2          ergm_3.10.4         
[25] xtable_1.8-4         mvtnorm_1.0-11       scales_1.0.0        
[28] tibble_2.1.3         annotate_1.62.0      IRanges_2.18.2      
[31] TH.data_1.0-10       withr_2.1.2          BiocGenerics_0.30.0 
[34] lazyeval_0.2.2       mime_0.7             survival_2.44-1.1   
[37] magrittr_1.5         crayon_1.3.4         statnet.common_4.3.0
[40] memoise_1.1.0        laeken_0.5.0         R.cache_0.13.0      
[43] MASS_7.3-51.4        R.rsp_0.43.1         tools_3.6.1         
[46] multcomp_1.4-10      S4Vectors_0.22.1     trust_0.1-7         
[49] munsell_0.5.0        AnnotationDbi_1.46.1 compiler_3.6.1      
[52] rlang_0.4.0          grid_3.6.1           RCurl_1.95-4.12     
[55] cwhmisc_6.6          rappdirs_0.3.1       labeling_0.3        
[58] bitops_1.0-6         base64enc_0.1-3      boot_1.3-23         
[61] gtable_0.3.0         codetools_0.2-16     DBI_1.0.0           
[64] markdown_1.1         R6_2.4.0             zoo_1.8-6           
[67] dplyr_0.8.3          bit_1.1-14           zeallot_0.1.0       
[70] parallel_3.6.1       Rcpp_1.0.2           vctrs_0.2.0         
[73] DEoptimR_1.0-8       tidyselect_0.2.5     xfun_0.9            
[76] coda_0.19-3

Total processing time was 24.27 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colCummins')

Copyright Henrik Bengtsson. Last updated on 2019-09-10 20:38:14 (-0700 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAA21BMVEUAAAAAAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8AAP8BAf4CAv0DA/wdHeIeHuEfH+AgIN8hId4lJdomJtknJ9g+PsE/P8BAQL9yco10dIt1dYp3d4h4eIeVlWqWlmmXl2iYmGeZmWabm2Tn5xjo6Bfp6Rb39wj4+Af//wA2M9hbAAAASXRSTlMAAQIJCgsMJSYnKD4/QGRlZmhpamtsbautrrCxuru8y8zN5ebn6Pn6+///////////////////////////////////////////LsUNcQAAAS9JREFUOI29k21XgkAQhVcFytdSMqMETU26UVqGmpaiFbL//xc1cAhhwVNf6n5i5z67M2dmYOyfJZUqlVLhkKucG7cgmUZTybDz6g0iDeq51PUr37Ds2cy2/C9NeES5puDjxuUk1xnToZsg8pfA3avHQ3lLIi7iWRrkv/OYtkScxBIMgDee0ALoyxHQBJ68JLCjOtQIMIANF7QG9G9fNnHvisCHBVMKgSJgiz7nE+AoBKrAPA3MgepvgR9TSCasrCKH0eB1wBGBFdCO+nAGjMVGPcQb5bd6mQRegN6+1axOs9nGfYcCtfi4NQosdtH7dB+txFIpXQqN1p9B/asRHToyS0jRgpV7nk4nwcq1BJ+x3Gl/v7S9Wmpp/aGquum7w3ZDyrADFYrl8vHBH+ev9AUASW1dmU4h4wAAAABJRU5ErkJggg==" document.getElementsByTagName('head')[0].appendChild(link); </script>
Add a custom footer

Benchmark reports

Clone this wiki locally