Skip to content

Commit

Permalink
refactor: Introduce FilesystemStatistics for adding runtime counters (#…
Browse files Browse the repository at this point in the history
…145)

Summary:
X-link: facebookincubator/velox#12424

Pull Request resolved: #145

Introduce FilesystemStatistics as a free form thread-safe structure used to create populate counters based off of Velox's RuntimeMetrics. The underlying data structure is a map with key being whatever client would like and value being the RuntimeMetrics.

The motivation behind this refactor is that previous PR introduced iostats into filesystems.h interface; however the iostats is specific to dwio and potentially not generic for other filesystem implementations. This PR addresses this by not using iostats and introducing `FilesystemStatistics` to serve the same purpose and also allow generic free form stats for user to add.

Reviewed By: xiaoxmeng, Yuhta

Differential Revision: D70007870
  • Loading branch information
yuandagits authored and facebook-github-bot committed Feb 26, 2025
1 parent ec60a45 commit e5580cd
Showing 1 changed file with 5 additions and 4 deletions.
9 changes: 5 additions & 4 deletions dwio/nimble/common/tests/TestUtils.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#include "folly/Random.h"
#include "folly/Synchronized.h"
#include "velox/common/file/File.h"
#include "velox/common/file/FileSystems.h"
#include "velox/common/memory/Memory.h"

// Utilities to support testing in nimble.
Expand Down Expand Up @@ -294,30 +295,30 @@ class InMemoryTrackableReadFile final : public velox::ReadFile {
uint64_t offset,
uint64_t length,
void* buf,
velox::io::IoStatistics* stats = nullptr) const final {
velox::filesystems::File::IoStats* stats = nullptr) const final {
chunks_.wlock()->push_back({offset, length});
return file_.pread(offset, length, buf, stats);
}

std::string pread(
uint64_t offset,
uint64_t length,
velox::io::IoStatistics* stats = nullptr) const final {
velox::filesystems::File::IoStats* stats = nullptr) const final {
chunks_.wlock()->push_back({offset, length});
return file_.pread(offset, length, stats);
}

uint64_t preadv(
uint64_t /* offset */,
const std::vector<folly::Range<char*>>& /* buffers */,
velox::io::IoStatistics* stats = nullptr) const final {
velox::filesystems::File::IoStats* stats = nullptr) const final {
NIMBLE_NOT_SUPPORTED("Not used by Nimble");
}

uint64_t preadv(
folly::Range<const velox::common::Region*> regions,
folly::Range<folly::IOBuf*> iobufs,
velox::io::IoStatistics* stats = nullptr) const override {
velox::filesystems::File::IoStats* stats = nullptr) const override {
VELOX_CHECK_EQ(regions.size(), iobufs.size());
uint64_t length = 0;
for (size_t i = 0; i < regions.size(); ++i) {
Expand Down

0 comments on commit e5580cd

Please sign in to comment.