Skip to content

Commit

Permalink
Merge pull request #2 from cbielow/support_unicode
Browse files Browse the repository at this point in the history
Support unicode
  • Loading branch information
cbielow authored Apr 7, 2023
2 parents e03be86 + a32e1cd commit f590078
Show file tree
Hide file tree
Showing 14 changed files with 349 additions and 217 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

V1.1 - 2023/04/07
- Unicode Support for command line arguments
- report local time for Creation/Exit time of process instead of UTC
- fix ExampleTarget RAM usage in Release mode (RAM allocation was not effective)
- switch from dll-injection to external querying of RAM usage

V1.0 - 2023/02/13
- initial release
7 changes: 1 addition & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ cmake_minimum_required(VERSION 3.12)
project(WinTimeAll VERSION 1.0)

# specify the C++ standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD 20)

## requires a 'project()' call first!
if (CMAKE_SIZEOF_VOID_P EQUAL 4)
Expand All @@ -17,12 +17,7 @@ endif()
message(STATUS "Target Arch is ${ARCH}")

## the order needs to be exactly this (due to internal references)
add_subdirectory(NamedPipeLib)
add_subdirectory(WinTime)
add_subdirectory(WinTimeDll) ## needs to go last, since it refs WinTime

add_subdirectory(ExampleTarget)

## run that here again (already done in WinTime), but now WinTimeDll data is available
file(REMOVE "${WinTime_BINARY_DIR}/config.h")
configure_file("${WinTime_SOURCE_DIR}/config.h.in" "${WinTime_BINARY_DIR}/config.h" @ONLY)
16 changes: 9 additions & 7 deletions ExampleTarget/ExampleTarget.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,17 @@ int main(int argc, char** argv)
if (argc == 1)
{
std::cerr << "Usage " << argv[0] << " <MiB alloc> <ms sleep> <stats>\n"
" MiB alloc: [number] MiB to allocate\n"
" ms sleep: [number, optional] Milliseconds to sleep\n"
" stats: [any value, optional] Print memory usage using internal functions (useful to estimate overhead of external WinTime)\n";
" MiB alloc: [number] MiB to allocate (this may take some time, especially for larger allocations)\n"
" ms sleep: [number, optional] Milliseconds to sleep (in addition to the time it takes to allocate -- see above)\n"
" stats: [any value, optional] Print memory usage using internal functions (useful to compare against the results of an external WinTime call)\n";
}
if (argc >= 2)
{
size_t mb = std::abs(atoi(argv[1]));
std::cerr << " -- Allocating " << mb << " Mb\n";
volatile auto ptr = malloc(mb * 1024 * 1024 + 1);
((char*)ptr)[0] = '\n';
std::cerr << " -- Deallocating\n";
free(ptr);
size_t bytes_to_allocate = mb * 1024 * 1024;
std::string large_data(bytes_to_allocate, 0);
volatile char* vd = large_data.data();
}
if (argc >= 3)
{
Expand All @@ -103,4 +102,7 @@ int main(int argc, char** argv)
{
getMemoryInfo(GetCurrentProcessId());
}
std::cerr << "-- end of ExampleTarget\n\n";
}


16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,13 @@ where `[wintime_options]` is the place for optional arguments to WinTime itself.

## Features

- peak **RAM** usage
- total **CPU time** (wall, kernel, user) with high resolution
- PageFaultCount
- PeakPagefileUsage
- reports:
- peak **RAM** usage
- total **CPU time** (wall, kernel, user) with high resolution
- PageFaultCount
- PeakPagefileUsage
- log file output
- supports Unicode program names and arguments via UTF-8 encoding
- similar command line interface as /usr/bin/time

## Installation
Expand Down Expand Up @@ -129,16 +131,16 @@ Open a [bug report, feature request](https://github.com/cbielow/wintime/issues)

## Technical details

WinTime makes heavy use of Windows API functions. It works by launching the target process and immediatedly injecting a Dll into it. Upon termination, the Dll will get notified that the target process is about to finish, and take the opportunity to measure peak ram usage and other metrics. This data will be send to WinTime using named pipes. WinTime itself will measure the CPU time of the target process, which can be done even after the process has exited iff you know the process ID (which we do since we spawned the process).
WinTime makes heavy use of Windows API functions. WinTime will invoke the target process and measure the CPU time and RAM usage of the target process. This can be done even after the process has exited iff you know the process ID (which we do since we spawned the process).
The `-o` option allows to write/append the data to a log, which requires some file locking magic to ensure that concurrent access to the log does not mangle its content.

## How accurate is the data?

##### RAM
The RAM usage of the target process is slightly increased by the injected Dll and the Windows functions it calls -- about 1.3 MiB. Similarly, PageFaultCount increases by about 344 (which makes sense for a 4kb page, i.e. 344*4 Kb = 1376 Kb). Those are worst case numbers. If the target process natively calls GetProcessMemoryInfo(), these numbers will go down (due to reused pages). At a bare minimum, the RAM usage will increase by the size of the injected Dll, which is currently ~50 Kb.
Same as `GetProcessMemoryInfo()` (part of the Windows API).

##### CPU
The CPU time (wall time, kernel time, user time) are high resolution, but include some minor overhead of injecting the Dll and time of calling the RAM usage function inside the target process -- on my system that is at most 5 milliseconds (this is how long a process executes which does absolutely nothing except execution of an empty `main()` function plus the overhead of the injected WinTime.dll).
The CPU time (wall time, kernel time, user time) are high resolution.

## License
MIT
Expand Down
17 changes: 9 additions & 8 deletions WinTime/Arch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,16 @@


#include "Arch.h"
#include "Process.h"

using namespace std;

namespace WinTime
{
Arch getArch(LPCWSTR path_to_exe)
Arch getArch(const std::string& path_to_exe)
{
DWORD result;
if (!GetBinaryTypeW(path_to_exe, &result))
if (!GetBinaryTypeW(&widen(path_to_exe)[0], &result))
{ // not an executable
return Arch::NOT_EXECUTABLE;
}
Expand All @@ -55,26 +56,26 @@ namespace WinTime
}
}

std::wstring getArchMatchedExplanation(const ArchMatched what, LPCWSTR path_to_target_exe)
std::string getArchMatchedExplanation(const ArchMatched what, LPCSTR path_to_target_exe)
{
switch (what)
{
break; case ArchMatched::SAME:
return std::wstring(L"Architectures match") + (sizeof(void*) == 8 ? L" (64bit)" : L" (32bit)");
return std::string("Architectures match") + (sizeof(void*) == 8 ? " (64bit)" : " (32bit)");
break; case ArchMatched::MIXED:
if (hostIs64Bit())
return std::wstring(L"WinTime is 64 bit, but target (") + path_to_target_exe + L") is 32 bit. Please use 32 bit version of WinTime.";
return std::string("WinTime is 64 bit, but target (") + path_to_target_exe + ") is 32 bit. Please use 32 bit version of WinTime.";
else
return std::wstring(L"WinTime is 32 bit, but target (") + path_to_target_exe + L") is 64 bit. Please use 64 bit version of WinTime.";
return std::string("WinTime is 32 bit, but target (") + path_to_target_exe + ") is 64 bit. Please use 64 bit version of WinTime.";
break; case ArchMatched::TARGET_UNKNOWN:
return std::wstring(L"Target Architecture (of ") + path_to_target_exe + L") is neither 32bit nor 64bit Windows, but something else.";
return std::string("Target Architecture (of ") + path_to_target_exe + ") is neither 32bit nor 64bit Windows, but something else.";
default:
throw std::logic_error("Missed a case? Please report a bug!");
};

}

ArchMatched checkMatchingArch(LPCWSTR target_exe)
ArchMatched checkMatchingArch(const std::string& target_exe)
{
const auto target_arch = getArch(target_exe);
constexpr const Arch this_arch = hostIs64Bit() ? Arch::x64 : Arch::x86;
Expand Down
6 changes: 3 additions & 3 deletions WinTime/Arch.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ namespace WinTime

/// Get architecture (32 or 64 bit) of executable
/// Return Arch::NOT_EXECUTABLE on error
Arch getArch(LPCWSTR path_to_exe);
Arch getArch(const std::string& path_to_exe);

constexpr bool hostIs64Bit()
{
Expand All @@ -52,9 +52,9 @@ namespace WinTime
TARGET_UNKNOWN ///< we surely know the host target (that's us; but target might be Arch::OTHER etc)
};

std::wstring getArchMatchedExplanation(const ArchMatched what, LPCWSTR path_to_target_exe);;
std::string getArchMatchedExplanation(const ArchMatched what, LPCSTR path_to_target_exe);;

/// Does the architecture of @p target_exe match the architecture of the current process?
ArchMatched checkMatchingArch(LPCWSTR target_exe);
ArchMatched checkMatchingArch(const std::string& target_exe);

} // namespace
7 changes: 2 additions & 5 deletions WinTime/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,8 @@ set(WINTIME_EXE "WinTime${ARCH}" CACHE STRING "Name of the exe" FORCE)
set(WINTIME_EXE_OTHERARCH "WinTime${ARCHOTHER}" CACHE STRING "Name of the exe of other bitness" FORCE)


## run that here, so the file is present... (but run configure_file later, since we need WinTimeDLL in there as well)
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/config.h" "")
add_executable(${WINTIME_EXE} WinTime.cpp args.hxx Time.h Time.cpp FileLog.h FileLog.cpp Console.h Console.cpp Arch.h Arch.cpp Process.h Process.cpp "${CMAKE_CURRENT_BINARY_DIR}/config.h")

target_link_libraries(${WINTIME_EXE} PRIVATE ${NAMEDPIPELIB})
configure_file("${WinTime_SOURCE_DIR}/config.h.in" "${WinTime_BINARY_DIR}/config.h" @ONLY)
add_executable(${WINTIME_EXE} WinTime.cpp args.hxx Time.h Time.cpp FileLog.h FileLog.cpp Console.h Console.cpp Arch.h Arch.cpp Memory.h Process.h Process.cpp "${CMAKE_CURRENT_BINARY_DIR}/config.h")

## include the binary dir to have access to config.h
target_include_directories(${WINTIME_EXE} PRIVATE "${CMAKE_CURRENT_BINARY_DIR}")
Expand Down
34 changes: 21 additions & 13 deletions WinTime/FileLog.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,13 @@

#include "FileLog.h"

#include "Memory.h"
#include "Process.h"

#include <io.h> // for _chsize_s()
#include <chrono>
#include <iostream>
#include <filesystem>
#include <thread>

using namespace std;
Expand All @@ -39,27 +44,30 @@ namespace WinTime
: filename_(filename),
openmode_(mode)
{
// First, make sure the file exists:
auto wfile = widen(filename_);
if (!std::filesystem::exists(wfile))
{
std::wofstream of(wfile);
if (!of.is_open())
{ // this may print bogus characters if non-ascii letters are printed, but
// printing to wcerr << wfile will not show non-ascii either, unless fiddling with
// _setmode, which breaks ascii output... well done Microsoft!
std::cerr << "Could not open file " << filename_ << ". Do you have permission to create it in this directory?\n";
throw std::runtime_error("Could not open file.");
}
}
}

bool LockedFile::tryLock()
{
// First, make sure the file exists:
// Opens for reading and appending; creates the file first if it doesn't exist.
FILE* tmp = _fsopen(filename_.c_str(), "a", _SH_DENYWR);

if (!tmp)
{ // could not create file. Is it locked by someone else?
return false;
}
fclose(tmp); // close the lock, otherwise we cannot lock it again below

// at this point, the file exists!
// at this point, the file exists! -- see C'tor

// open file for reading and writing (so we can move to the end, depending on 'mode_'.
// We cannot query for filesize before, because another process might write stuff to the file
// immediately afterwards (before we _fsopen for just writing in case we found the file being empty; this
// would overwrite the other process' data)
is_locked_ = ((stream_ = _fsopen(filename_.c_str(), "r+", _SH_DENYWR)) != NULL);
is_locked_ = ((stream_ = _wfsopen(&widen(filename_)[0], L"r+", _SH_DENYWR)) != NULL);

if (!is_locked_) return false;

Expand Down Expand Up @@ -131,7 +139,7 @@ namespace WinTime
std::stringstream content;
if (lockf.isFileEmpty())
{ // at start of file .. write header
printLineToStream(content, '\t', [](auto type, const char sep) { return type.printHeader(sep); }, Command(), PTime(), ClientProcessMemoryCounter(PROCESS_MEMORY_COUNTERS()));
printLineToStream(content, '\t', [](auto type, const char sep) { return type.printHeader(sep); }, Command(), time, pmc);
}
printLineToStream(content, '\t', [](auto type, const char sep) { return type.print(sep); }, Command{ cmd }, time, pmc);
lockf.write(content.str().c_str());
Expand Down
2 changes: 2 additions & 0 deletions WinTime/FileLog.h
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ namespace WinTime
/// Truncates the file to the last write position and closes the stream
~LockedFile();
};

struct ClientProcessMemoryCounter;

class FileLog
{
Expand Down
124 changes: 124 additions & 0 deletions WinTime/Memory.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/*
* Copyright (c) 2023-2023 Chris Bielow <chris[dot]bielow[at]fu-berlin[dot].de>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/

#pragma once

#include <array>
#include <iomanip>
#include <iostream>
#define WIN32_LEAN_AND_MEAN // Exclude rarely-used stuff from Windows headers
#define _CRT_SECURE_NO_WARNINGS 1
#include <windows.h>
#include <sstream>

// To ensure correct resolution of symbols, add Psapi.lib to TARGETLIBS
// and compile with -DPSAPI_VERSION=1
#define PSAPI_VERSION 1
#include <psapi.h> // for PROCESS_MEMORY_COUNTERS
#pragma comment (lib, "Psapi.lib")

namespace WinTime
{

/// convert bytes to a human readable unit (TiB, GiB, MiB, KiB)
inline std::string toHumanReadable(uint64_t bytes)
{
std::array units{ "byte", "KiB", "MiB", "GiB", "TiB", "PiB" };

const int divisor = 1024;
double db = double(bytes);
for (const auto u : units)
{
if (db < divisor)
{
std::stringstream ss;
ss << std::setprecision(4) /* 4 digits overall, i.e. 1000 or 1.456 */ << db << ' ' << u;
return ss.str();
}
db /= divisor;
}
// wow ... you made it here...
return std::string("Congrats. That's a lot of bytes: ") + std::to_string(bytes);
}

/// A serializable wrapper around a 'PROCESS_MEMORY_COUNTERS' struct
struct ClientProcessMemoryCounter
{
explicit ClientProcessMemoryCounter(HANDLE hProcess)
: data_{}
{
bool res = GetProcessMemoryInfo(hProcess, &data_, sizeof(data_));
if (!res)
{
throw std::runtime_error("Could not get memory info!");
}
}

void print() const
{ // only report data which does not depend on the time of measurement (which was when unloading the Dll). Only report maximia (peaks).
std::cerr << "PageFaultCount: " << (data_.PageFaultCount) << '\n';
std::cerr << "PeakWorkingSetSize: " << toHumanReadable(data_.PeakWorkingSetSize) << '\n';
//std::cerr << "WorkingSetSize: " << toHumanReadable(data_.WorkingSetSize) << '\n';
std::cerr << "QuotaPeakPagedPoolUsage: " << toHumanReadable(data_.QuotaPeakPagedPoolUsage) << '\n';
//std::cerr << "QuotaPagedPoolUsage: " << toHumanReadable(data_.QuotaPagedPoolUsage) << '\n';
std::cerr << "QuotaPeakNonPagedPoolUsage: " << toHumanReadable(data_.QuotaPeakNonPagedPoolUsage) << '\n';
//std::cerr << "QuotaNonPagedPoolUsage: " << toHumanReadable(data_.QuotaNonPagedPoolUsage) << '\n';
//std::cerr << "PagefileUsage: " << toHumanReadable(data_.PagefileUsage) << '\n';
std::cerr << "PeakPagefileUsage: " << toHumanReadable(data_.PeakPagefileUsage) << '\n';
}

std::string print(const char separator) const
{
std::stringstream where;
where << data_.PageFaultCount << separator
<< (data_.PeakWorkingSetSize) << separator
<< toHumanReadable(data_.PeakWorkingSetSize) << separator
//<< toHumanReadable(data_.WorkingSetSize) << separator
<< toHumanReadable(data_.QuotaPeakPagedPoolUsage) << separator
//<< toHumanReadable(data_.QuotaPagedPoolUsage) << separator
<< toHumanReadable(data_.QuotaPeakNonPagedPoolUsage) << separator
//<< toHumanReadable(data_.QuotaNonPagedPoolUsage) << separator
//<< toHumanReadable(data_.PagefileUsage) << separator
<< toHumanReadable(data_.PeakPagefileUsage);
return where.str();
}

static std::string printHeader(const char separator)
{ // only report data which does not depend on the time of measurement (which was when unloading the Dll). Only report maximia (peaks).
std::stringstream where;
where << "PageFaultCount" << separator
<< "PeakWorkingSetSize (bytes)" << separator
<< "PeakWorkingSetSize" << separator
//<< "WorkingSetSize" << separator
<< "QuotaPeakPagedPoolUsage" << separator
//<< "QuotaPagedPoolUsage" << separator
<< "QuotaPeakNonPagedPoolUsage" << separator
//<< "QuotaNonPagedPoolUsage" << separator
//<< "PagefileUsage" << separator
<< "PeakPagefileUsage";
return where.str();
}
private:
PROCESS_MEMORY_COUNTERS data_;
};

} // namespace
Loading

0 comments on commit f590078

Please sign in to comment.