Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to the latest tatami interface. #6

Merged
merged 36 commits into from
Sep 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7ca4c4f
Begin the slow process of updating.
LTLA Sep 23, 2024
a44a68b
Got the dense tests to pass, mostly.
LTLA Sep 23, 2024
6f531c1
Slowly crawl towards getting the sparse thing to work.
LTLA Sep 23, 2024
3125a46
Continue the grind.
LTLA Sep 23, 2024
1ebabb9
Further inching along.
LTLA Sep 24, 2024
e8d2cdc
Continue the grind.
LTLA Sep 24, 2024
4a9272e
Get a bit closer.
LTLA Sep 24, 2024
52f78e6
Inch a bit closer to getting it to all compile.
LTLA Sep 24, 2024
88f0a09
Got most tests up and running.
LTLA Sep 24, 2024
7708749
Got all the tests to pass, more or less.
LTLA Sep 24, 2024
c14e5ca
Do some preparatory work to handle untyped'ness.
LTLA Sep 24, 2024
ee6edd2
Merge branch 'master' into update
LTLA Sep 24, 2024
d4c3730
Avoid test issues from sporadic deadlocks during parallelization.
LTLA Sep 24, 2024
ccdcdfc
Updated the CI action to run tests more efficiently.
LTLA Sep 24, 2024
97220d9
Again try to lock down the parallelization.
LTLA Sep 24, 2024
563e44a
Removed the instrumentation.
LTLA Sep 24, 2024
1c43ef3
Shift the uncached analysis to a separate test suite.
LTLA Sep 24, 2024
4cc7f1d
WHAT A MEGAGRIND.
LTLA Sep 26, 2024
84a23f7
Comment edit.
LTLA Sep 26, 2024
8500073
Restored the typed vector.
LTLA Sep 26, 2024
0537e13
Dense matrix now auto-adapts to the input type, directly fills cache.
LTLA Sep 26, 2024
720090b
Get even closer.
LTLA Sep 26, 2024
a4f9535
Inch a little bit closer.
LTLA Sep 27, 2024
cabb135
Got most of it to compile.
LTLA Sep 27, 2024
a0d0628
Got it all to compile.
LTLA Sep 27, 2024
4b1e1e8
Got the tests to pass again.
LTLA Sep 27, 2024
f2f219f
Check that the variable dimensions are computed correctly w/o overflow.
LTLA Sep 27, 2024
659b0a4
HArdended the tests.
LTLA Sep 27, 2024
73c7bd4
More obvious explanation for the various integer shenanigans.
LTLA Sep 27, 2024
3580ea4
Added more tests for the variably typed utilities.
LTLA Sep 28, 2024
c317a3b
Bugfix and bump out the rigor of the tests.
LTLA Sep 28, 2024
07259fe
Stripped out the Solo code to simplify matters.
LTLA Sep 28, 2024
c3ae439
Added codecov secret to get some codecov goodness.
LTLA Sep 28, 2024
ea7efa1
Remove unnecessary cases.
LTLA Sep 28, 2024
6bb7497
Updated the README.
LTLA Sep 28, 2024
2282e87
Further README fixes.
LTLA Sep 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .github/workflows/run-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ jobs:
}

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Get latest CMake
uses: lukka/get-cmake@latest

- uses: actions/cache@v3
- uses: actions/cache@v4
id: cache
with:
path: libtiledb
Expand All @@ -51,7 +51,15 @@ jobs:
- name: Run the tests
run: |
cd build
ctest
# Avoid using ctest because it's so slow; it starts a
# new process for each individual test, which is crazy.
for exe in $(ctest --show-only=json-v1 | jq -r ".tests[] | .command | .[0]" | sort | uniq)
do
echo "#### RUNNING ${exe} ####"
echo
${exe} --gtest_brief=1
echo
done

- name: Generate code coverage
if: ${{ matrix.config.cov }}
Expand All @@ -61,6 +69,8 @@ jobs:

- name: Upload to Codecov
if: ${{ matrix.config.cov }}
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
directory: build/tests/CMakeFiles/
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
17 changes: 5 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,18 @@ Any matrix stored as a 2-dimensional TileDB array (dense or sparse) can be repre
```cpp
#include "tatami_tiledb/tatami_tiledb.hpp"

tatami_tiledb::TileDbDenseMatrix<double, int> dense_mat("some_dir", "attr_name");
tatami_tiledb::TileDbSparseMatrix<double, int> sparse_mat("some_dir", "attr_name");
```

If the dense/sparse nature is not known beforehand, we can use the `make_TileDbMatrix()` function to decide for us instead:

```cpp
auto some_mat = tatami_tiledb::make_TileDbMatrix("some_dir", "attr_name");
tatami_tiledb::DenseMatrix<double, int> dense_mat("some_dir", "attr_name");
tatami_tiledb::SparseMatrix<double, int> sparse_mat("some_dir", "attr_name");
```

Advanced users can fiddle with the options, in particular the size of the tile cache.
For example, we could force the matrix to use no more than 200 MB of memory in the cache:

```cpp
tatami_tiledb::TileDbOptions opt;
tatami_tiledb::DenseMatrixOptions opt;
opt.maximum_cache_size = 200000000;
opt.require_minimum_cache = false; // don't allow the cache to automatically expand.

auto some_mat2 = tatami_tiledb::make_TileDbMatrix("some_dir", "attr_name", opt);
tatami_tiledb::DenseMatrix<double, int> some_mat2("some_dir", "attr_name", opt);
```

Check out the [reference documentation](https://tatami-inc.github.io/tatami_tiledb) for more details.
Expand All @@ -50,7 +43,7 @@ If you're using CMake, you just need to add something like this to your `CMakeLi
include(FetchContent)

FetchContent_Declare(
tatami
tatami_tiledb
GIT_REPOSITORY https://github.com/tatami-inc/tatami_tiledb
GIT_TAG master # or any version of interest
)
Expand Down
Loading
Loading