Merge pull request roced-scheduler#19 from uschnoor/ben_benchmark

Ben benchmark

publication/Draft_CompSoftwBigScience/include/Benchmarks.tex

@@ -1,35 +1,33 @@
-Besides the use of the legacy HEP-SPEC06 (HS06) benchmark \cite{Hepspec}, the performance of the compute resources is furthermore evaluated with two fast benchmarks
-developed to provide real-time information of the WLCG performance and available in the CERN benchmark suite \cite{Alef:2017jyx}; the Dirac Benchmark 2012 DB12 \cite{DB12}
-and the ATLAS Kit Validation KV \cite{DeSalvo:2010zza}. The DB12 program is evaluating the performance of CPUs through floating-point arithmetic operations, while the KV benchmark
-is making use of the simulation toolkit GEANT4 \cite{Agostinelli:2002hh} to simulate the interactions of single muon events in the detector of the ATLAS experiment. As our primary
-target is to measure performances of CPUs in the context of High Energy Physics applications, the KV benchmark constitutes a realistic workload. The DB12 benchmark is using
-the HS06 units, and the KV output provides the number of events produced per second. \\
+Besides the use of the legacy HEP-SPEC06 (HS06) benchmark \cite{Hepspec}, the performance of the compute resources is furthermore evaluated with the ATLAS Kit Validation
+KV \cite{DeSalvo:2010zza}, a fast benchmark developed to provide real-time information of the WLCG performance and available in the CERN benchmark suite \cite{Alef:2017jyx}.
+The KV benchmark is making use of the simulation toolkit GEANT4 \cite{Agostinelli:2002hh} to simulate the interactions of single muon events in the detector of the ATLAS experiment
+and provides as ouput the number of events produced per second. As our primary target is to measure performances of CPUs in the context of High Energy Physics applications, the KV benchmark
+constitutes a realistic workload. \\
 
-To assess the impact of the virtualisation, the performance of the same hardware configuration (20 cores Intel Xeon E5-2630 CPUs) has been determined either deployed via
-the standard bare metal operation on the NEMO cluster (NEMO bare metal) and on the local ATLAS Tier-3 centre in Freiburg (ATLAS-BFG bare metal), or as virtual machines on the
-NEMO cluster (NEMO VM). On the ATLAS-BFG bare metal and on the virtual machines running on the NEMO cluster, hyper-threading (HT) technology is activated. Both are using Scientific
-Linux 6 \cite{SL6} as operating system. The NEMO bare metal has no HT activated due to the more general use case of the system, and uses CentOS7 as operating system \cite{CentOS7}. % \\
-The scores of the HEP-SPEC06, DB12, and KV benchmarks have been determined for these three configurations as a function of the number of cores actually used by the benchmarking processes.
-This number ranges from 2 to 40 for the ATLAS-BFG bare metal and for the NEMO VM, for which HT is enabled, and from 2 to 20 for the NEMO bare metal,
-for which HT is not implemented. The results have been determined by step of two core units. The benchmarks have been run 20 times for each core multiplicity value, and the means
-and standard deviations of the corresponding distributions have been extracted. \\
+To assess the impact of the virtualization, the performance of the same hardware configuration (20 cores Intel Xeon E5-2630 CPUs) has been determined either deployed via
+the standard bare metal operation on the NEMO cluster (NEMO bare metal) and on the ATLAS Tier-3 center in Freiburg (ATLAS Tier-3 bare metal), or as virtual machines on the
+NEMO cluster (NEMO VM). On the ATLAS Tier-3 bare metal and on the virtual machines running on the NEMO cluster, hyper-threading (HT) technology is activated. Both are using Scientific
+Linux 6 \cite{SL6} as operating system. The NEMO bare metal has no HT activated due to the more general use case of the system, and uses CentOS7 as operating system \cite{CentOS7}. 
+The scores of the HEP-SPEC06 and KV benchmarks have been determined for these three configurations as a function of the number of cores actually used by the benchmarking processes.
+This number ranges from 2 to 40 for the ATLAS Tier-3 bare metal and for the NEMO VM, for which HT is enabled, and from 2 to 20 for the NEMO bare metal, for which HT is not implemented.
+The benchmarks have been run 20 times for each core multiplicity value, and the means and standard deviations of the corresponding distributions have been extracted. \\
 
 \begin{figure}[htbp]
 \centering
 \includegraphics[width=0.44\textwidth]{figures/benchmark-hepspec06-total.pdf}
-\includegraphics[width=0.44\textwidth]{figures/DB12-BFG-BM-NEMO-VM-NEMO-BM-total.pdf} 
 \includegraphics[width=0.44\textwidth]{figures/kv-BFG-BM-NEMO-VM-NEMO-BM-total.pdf} 
-\caption{Total score as a function of the core multiplicity for the HEP-SPEC06 (top), DB12 (middle) and KV (bottom) benchmarks
-for the ATLAS-BFG bare metal (blue open circles), the NEMO VMs (red full circles) and the NEMO bare metal (black open squares). The data points represent
-the average values of the benchmarks for each core multiplicity, and the vertical bars show the associated standard deviations.}
+\caption{Total score as a function of the core multiplicity for the HEP-SPEC06 (top) and KV (bottom) benchmarks for the ATLAS Tier-3 bare metal (blue open circles),
+the NEMO VMs (red full circles) and the NEMO bare metal (black open squares). The data points represent the average values of the benchmarks for each core multiplicity,
+and the vertical bars show the associated standard deviations.}
 \label{bmk-total}
 \end{figure}
 
-The total scores are presented in Fig.~\ref{bmk-total} for the three benchmarks and the three configurations considered, except for the KV software for which the NEMO bare metal
-results are not yet available. The total scores observed for the different benchmarks are increasing until the maximum number of physical cores has been reached, and are characterized by a flattening increase
-or a constant behaviour afterwards. The benchmark scores of the virtual machines running on the NEMO cluster are only slightly lower than those obtained for the NEMO bare metal operation, and the lost of performance
-due to the virtualisation does not exceed 10$\%$. For the ATLAS-BFG bare metal and the VMs running on the NEMO cluster, the interplay between the virtualisation and the different operating systems leads
-to similar benchmark behaviours for the two configurations.
+The HEP-SPEC06 and KV results are presented in figure \ref{bmk-total} for the three configurations considered.
+The total scores of the two benchmarks are increasing until the maximum number of physical cores has been reached, and are characterized by a flattening increase afterwards.
+The scores of the virtual machines running on the NEMO cluster are only slightly lower than those obtained for the NEMO bare metal, and the lost of performance
+due to the virtualization does not exceed 10$\%$.
+For the VMs running on the NEMO cluster and the ATLAS Tier-3 bare metal, the interplay between the virtualization and the different operating systems leads to very similar scores
+for the two configurations, particularly for the KV benchmark, and the lost of performance is smaller than 10$\%$ as well.
 
 % References for HEPSPEC
 %%https://indico.cern.ch/event/49388/contributions/2014772/attachments/960838/1363966/20090127_NewCPUbenchmark_final.pdf

publication/Draft_CompSoftwBigScience/include/PackerPuppet.tex

@@ -6,3 +6,6 @@
 
 \Packer's interface to \Puppet allows a fully automated image generation with up-to-date and version-controlled configuration. At the end of the generation run, the image is automatically transferred to the \\\Openstack image server.
 
+
+
+

publication/Draft_CompSoftwBigScience/nemo-virtualization.tex

@@ -89,7 +89,7 @@
 %\titlerunning{Short form of title}        % if too long for running head
 
 \author{Felix B\"uhrer \and Anton Gamel \and Thomas Hauth \and Michael Janczyk \and
-  Benoit Roland \and
+  Beno\^it Roland \and
   Markus Schumacher \and
   Ulrike Schnoor \and Dirk von Suchodoletz \and Bernd Wiebelt
   % etc.
@@ -128,7 +128,7 @@
 
 %Insert your abstract here. Include keywords, PACS and mathematical
 %subject classification numbers as needed.
-\keywords{Virtualization \and Particle Physics \and Grid Computing }
+\keywords{Virtualization \and Particle Physics \and Grid Computing \and Benchmarks}
 % \PACS{PACS code1 \and PACS code2 \and more}
 % \subclass{MSC code1 \and MSC code2 \and more}
 \end{abstract}
@@ -233,7 +233,7 @@ \subsection{Benchmarks}
 %$\to$ATLAS Freiburg 
 %
 \subsection{Usage statistics}
-Fig. \ref{fig-frplots} show the utilization of virtual machines which were orchestrated by \Roced depending on the resource demands of the users of the KIT group.
+Fig. \ref{fig-frplots} shows the utilization of virtual machines which were orchestrated by \Roced depending on the resource demands of the users of the KIT group.
 At peak times, up to 9000 virtual cores were filled with user jobs, consuming more than a half of the initial 16000 NEMO cores.
 
 \begin{figure}
@@ -251,7 +251,7 @@ \subsection{Usage statistics}
 a standard bare-metal job.
 
 At the beginning the cluster was only containing the operating system and some
-basic development tools, scientific software was added after the  cluster was
+basic development tools, scientific software was added after the cluster was
 already in production mode. Since the VRE for the CMS project was already
 available when the NEMO cluster started, it could already use the whole cluster
 while other groups still had to wait for the required scientific software to be deployed on