Skip to content

Commit

Permalink
[Chapter2] Rewrite. part7
Browse files Browse the repository at this point in the history
  • Loading branch information
dendibakh committed Sep 4, 2024
1 parent d09f798 commit 2fd0810
Show file tree
Hide file tree
Showing 4 changed files with 2 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The most straightforward way to compare two performance distributions is to take

Data scientists often present measurements by plotting the distributions. This eliminates biased conclusions and allows readers to interpret the data for themselves. One of the popular ways to plot distributions is by using box plots (also known as a box-and-whisker plots). In Figure @fig:BoxPlot, we visualized performance distributions of two versions of the same functional program ("before" and "after"). There are 70 performance data points in each distribution.

![Performance of three versions of a program presented as box plots. Box plots better visualize performance distribution and enable more accurate comparisons with other distributions.](../../img/measurements/BoxPlots.png){#fig:BoxPlot width=80%}
![Performance of three versions of a program presented as box plots. Box plots better visualize performance distribution and enable more accurate comparisons with other distributions.](../../img/measurements/BoxPlots.png){#fig:BoxPlot width=90%}

Let's describe the terms indicated on the image:

Expand Down
3 changes: 1 addition & 2 deletions chapters/2-Measuring-Performance/2-5 SW and HW Timers.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

To benchmark execution time, engineers usually use two different timers, which all the modern platforms provide:

- **System-wide high-resolution timer**: this is a system timer that is typically implemented as a simple count of the number of ticks that have transpired since an arbitrary starting date, called the [epoch](https://en.wikipedia.org/wiki/Epoch_(computing))[^1]. This clock is monotonic; i.e., it always goes up. System time can be retrieved from the OS with a system call.[^2] Accessing the system timer on Linux systems is possible via the `clock_gettime` system call. The system timer has a nanosecond resolution, is consistent between all the CPUs and is independent of CPU frequency. Even though the system timer can return timestamps with a nanosecond accuracy, it is not suitable for measuring short-running events because it takes a long time to obtain the timestamp via the `clock_gettime` system call. But it is OK to measure events with a duration of more than a microsecond. The standard way of accessing system timer in C++ is using `std::chrono` as shown in [@lst:Chrono].
- **System-wide high-resolution timer**: this is a system timer that is typically implemented as a simple count of the number of ticks that have transpired since an arbitrary starting date, called the [epoch](https://en.wikipedia.org/wiki/Epoch_(computing))[^1]. This clock is monotonic; i.e., it always goes up. System time can be retrieved from the OS with a system call. Accessing the system timer on Linux systems is possible via the `clock_gettime` system call. The system timer has a nanosecond resolution, is consistent between all the CPUs and is independent of the CPU frequency. Even though the system timer can return timestamps with a nanosecond accuracy, it is not suitable for measuring short-running events because it takes a long time to obtain the timestamp via the `clock_gettime` system call. But it is OK to measure events with a duration of more than a microsecond. The standard way of accessing system timer in C++ is using `std::chrono` as shown in [@lst:Chrono].

Listing: Using C++ std::chrono to access system timer

Expand Down Expand Up @@ -43,5 +43,4 @@ To benchmark execution time, engineers usually use two different timers, which a
Choosing which timer to use is very simple and depends on how long the thing is that you want to measure. If you measure something over a very short time period, TSC will give you better accuracy. Conversely, it's pointless to use the TSC to measure a program that runs for hours. Unless you need cycle accuracy, the system timer should be enough for a large proportion of cases. It's important to keep in mind that accessing the system timer usually has a higher latency than accessing TSC. Making a `clock_gettime` system call can be easily ten times slower than executing `RDTSC` instruction, which takes 20+ CPU cycles. This may become important for minimizing measurement overhead, especially in the production environment. A performance comparison of different APIs for accessing timers on various platforms is available on a [wiki page](https://gitlab.com/chriscox/CppPerformanceBenchmarks/-/wikis/ClockTimeAnalysis)[^3] of the CppPerformanceBenchmarks repository.
[^1]: Unix epoch starts on 1 January 1970 00:00:00 UT: [https://en.wikipedia.org/wiki/Unix_epoch](https://en.wikipedia.org/wiki/Unix_epoch).
[^2]: Retrieving system time - [https://en.wikipedia.org/wiki/System_time#Retrieving_system_time](https://en.wikipedia.org/wiki/System_time#Retrieving_system_time)
[^3]: CppPerformanceBenchmarks wiki - [https://gitlab.com/chriscox/CppPerformanceBenchmarks/-/wikis/ClockTimeAnalysis](https://gitlab.com/chriscox/CppPerformanceBenchmarks/-/wikis/ClockTimeAnalysis)
2 changes: 0 additions & 2 deletions chapters/2-Measuring-Performance/2-6 Microbenchmarks.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@


## Microbenchmarks

Microbenchmarks are small self-contained programs that people write to quickly test a hypothesis. Usually, microbenchmarks are used to choose the best implementation of a certain relatively small algorithm or functionality. Nearly all modern languages have benchmarking frameworks. In C++, one can use the Google [benchmark](https://github.com/google/benchmark)[^3] library, C# has [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet)[^4] library, Julia has the [BenchmarkTools](https://github.com/JuliaCI/BenchmarkTools.jl)[^5] package, Java has [JMH](http://openjdk.java.net/projects/code-tools/jmh/etc)[^6] (Java Microbenchmark Harness), etc.
Expand Down
Binary file modified img/measurements/BoxPlots.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 2fd0810

Please sign in to comment.