Added `StoreCache` #3701

cschuchardt88 · 2025-01-29T06:24:23Z

Description

This is a benchmark of my StoreCache vs. DataCache. Let know if I am missing functionality or features.

StorageCache

Note: this class needs more features and optimizations for better performance.

Features

Has expiration tracking for items.
Cache throughputs to IStore data.

Expiration Tracking

Remove item from the cache when they expire automatically.
Resets expiration time when accessed.

@shargon @Jim8y @nan01ab
I need some input on this for new feature and what you think.

Benchmark

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.2894)
Intel Core i7-8700K CPU 3.70GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 9.0.102
  [Host]     : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2
  DefaultJob : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2

Method	Mean	Error	StdDev	Gen0	Gen1	Allocated
TestStoreCacheAddAndUpdate	34.91 ns	0.620 ns	0.518 ns	0.0063	-	40 B
TestStoreCacheRemove	11.76 ns	0.054 ns	0.045 ns	-	-	-
TestStoreCacheGetAlreadyCachedData	25.06 ns	0.082 ns	0.076 ns	-	-	-
TestStoreCacheGetNonCachedData	922.97 ns	6.598 ns	5.849 ns	1.3189	0.0410	8280 B
TestDataCacheAddAndUpdate	51.73 ns	0.396 ns	0.351 ns	-	-	-
TestDataCacheRemove	73.13 ns	1.266 ns	1.184 ns	-	-	-
TestDataCacheGetAlreadyCachedData	76.74 ns	0.691 ns	0.577 ns	-	-	-
TestDataCacheGetNonCachedData	731.02 ns	13.061 ns	12.217 ns	1.3123	0.0305	8240 B

Type of change

Optimization (the change is only an optimization)
Style (the change is only a code style for better maintenance or standard purpose)
Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

UT_StoreCache.TestAddAndGetSync
UT_StoreCache.TestStoreCacheGetNonCachedData

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

Jim8y · 2025-01-29T07:04:21Z

do you mind to briefly explain why your cache is more efficient?

cschuchardt88 · 2025-01-29T07:54:03Z

do you mind to briefly explain why your cache is more efficient?

Weak references are useful for objects that use a lot of memory, but can be recreated easily if they are reclaimed by garbage collection.

Suppose a tree view in a Windows Forms application displays a complex hierarchical choice of options to the user. If the underlying data is large, keeping the tree in memory is inefficient when the user is involved with something else in the application.

When the user switches away to another part of the application, you can use the WeakReference class to create a weak reference to the tree and destroy all strong references. When the user switches back to the tree, the application attempts to obtain a strong reference to the tree and, if successful, avoids reconstructing the tree.

Reference: https://learn.microsoft.com/en-us/dotnet/standard/garbage-collection/weak-references

Also uses a write-through caching to the StoreProvider and none of this build your own bs tracking.

nan01ab · 2025-01-29T10:09:26Z

src/Neo/Collections/Caching/StoreCache.cs

+
+namespace Neo.Collections.Caching
+{
+    public class StoreCache<TKey, TValue>(IStore store) : ICollection<KeyValuePair<TKey, TValue>>, IEnumerable<KeyValuePair<TKey, TValue>>, IEnumerable, IDictionary<TKey, TValue>, IReadOnlyCollection<KeyValuePair<TKey, TValue>>, IReadOnlyDictionary<TKey, TValue>, ICollection, IDictionary


This line is too long.

tests/Neo.UnitTests/Collections/Caching/UT_StoreCache.cs

shargon · 2025-01-29T12:29:14Z

Changing these keys, values and comparer is not enough?

 public class MemoryStore : IStore
 {
     private readonly ConcurrentDictionary<byte[], byte[]> _innerData = new(ByteArrayEqualityComparer.Default);

Why create a new Store?

shargon · 2025-01-29T12:41:18Z

If your plan is optimize DataCache is good to me, but if you want to add a new kind of class for storage, we have ClonedCache , DataCache, ISnapshot, IStore, SnapShotCache, in my opinion, we don't need more, we can optimize or reduce the current ones.

cschuchardt88 · 2025-01-29T17:35:50Z

If your plan is optimize DataCache is good to me, but if you want to add a new kind of class for storage, we have ClonedCache , DataCache, ISnapshot, IStore, SnapShotCache, in my opinion, we don't need more, we can optimize or reduce the current ones.

The goal is to deprecate those classes. I would of added this in leveldb and rocksdb store. However, we don't ever have direct access to the Store class, only the interface (IStore). More needs to be done with StoreCache class, like allowing snapshots or making it immutable.

shargon · 2025-01-29T17:45:27Z

benchmarks/Neo.Benchmarks/Collections/Caching/Benchmarks.StoreCache.cs

+    [MemoryDiagnoser]  // Enabling Memory Diagnostics
+    [CsvMeasurementsExporter]  // Export results in CSV format
+    [MarkdownExporter]  // Exporting results in Markdown format
+    public class Benchmarks_StoreCache


Why the benchmarks are not the same but changing the interface? it looks like the methods are totally different

Methods do the same thing. However, StoreCache class is a collection, so you can convert StoreCache to any collection you want. This is how DataCache and other classes in neo should of been developed.

StoreCache inherits from

ICollection<KeyValuePair<TKey, TValue>> IEnumerable<KeyValuePair<TKey, TValue>> IEnumerable IDictionary<TKey, TValue> IReadOnlyCollection<KeyValuePair<TKey, TValue>> IReadOnlyDictionary<TKey, TValue> ICollection IDictionary

…8/neo into fix/storage-caching

cschuchardt88 · 2025-01-30T00:56:28Z

This is with optimizations to code.

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.2894)
Intel Core i7-8700K CPU 3.70GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 9.0.102
  [Host]   : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2
  .NET 9.0 : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2

Job=.NET 9.0  OutlierMode=DontRemove  Runtime=.NET 9.0  
Concurrent=True  Force=False  Server=True

Method	Mean	Error	StdDev	Median	Min	Max
TestStoreCacheAdd	13.9222 ns	0.2720 ns	0.2544 ns	13.9250 ns	13.6381 ns	14.4881 ns
TestDataCacheAdd	0.0144 ns	0.0218 ns	0.0204 ns	0.0103 ns	0.0000 ns	0.0736 ns
TestStoreCacheUpdate	0.0049 ns	0.0063 ns	0.0059 ns	0.0022 ns	0.0000 ns	0.0199 ns
TestDataCacheUpdate	0.0197 ns	0.0130 ns	0.0122 ns	0.0215 ns	0.0000 ns	0.0399 ns
TestStoreCacheRemove	25.6244 ns	0.3467 ns	0.3243 ns	25.4620 ns	25.2082 ns	26.2017 ns
TestDataCacheRemove	36.0025 ns	0.7469 ns	1.2884 ns	35.5903 ns	34.8762 ns	41.3266 ns
TestStoreCacheGet	0.0023 ns	0.0059 ns	0.0055 ns	0.0000 ns	0.0000 ns	0.0201 ns
TestDataCacheGet	0.0058 ns	0.0140 ns	0.0149 ns	0.0000 ns	0.0000 ns	0.0578 ns

Running on a Raspberry Pi 4 model B

BenchmarkDotNet v0.14.0, Debian GNU/Linux 11 (bullseye)
Broadcom BCM2711 ARM Cortex-A72, 1 CPU, 4 Cores
.NET SDK 9.0.102
  [Host]   : .NET 9.0.1 (9.0.124.61010), Arm64 RyuJIT AdvSIMD
  .NET 9.0 : .NET 9.0.1 (9.0.124.61010), Arm64 RyuJIT AdvSIMD

Job=.NET 9.0  OutlierMode=DontRemove  Runtime=.NET 9.0  
Concurrent=True  Force=False  Server=True

Method	Mean	Error	StdDev	Median	Min	Max
TestStoreCacheAdd	61.4663 ns	0.1621 ns	0.1516 ns	61.5318 ns	61.1207 ns	61.6121 ns
TestDataCacheAdd	0.0032 ns	0.0131 ns	0.0123 ns	0.0000 ns	0.0000 ns	0.0475 ns
TestStoreCacheUpdate	0.0002 ns	0.0004 ns	0.0003 ns	0.0000 ns	0.0000 ns	0.0009 ns
TestDataCacheUpdate	0.0004 ns	0.0007 ns	0.0007 ns	0.0000 ns	0.0000 ns	0.0019 ns
TestStoreCacheRemove	131.5770 ns	0.1443 ns	0.1349 ns	131.5936 ns	131.2911 ns	131.8276 ns
TestDataCacheRemove	173.8043 ns	0.0710 ns	0.0665 ns	173.8081 ns	173.7009 ns	173.9238 ns
TestStoreCacheGet	0.0002 ns	0.0004 ns	0.0004 ns	0.0000 ns	0.0000 ns	0.0011 ns
TestDataCacheGet	0.0003 ns	0.0005 ns	0.0005 ns	0.0000 ns	0.0000 ns	0.0016 ns

shargon · 2025-01-30T07:50:16Z

If the key is faster with StorageKey, we should force it in IStore to be StorageKey first

…8/neo into fix/storage-caching

cschuchardt88 · 2025-01-30T23:42:46Z

I added StorageCache class with fallback to IStore.

StorageCache

Note: this class needs more features and optimizations for better performance.

Features

Has expiration tracking for items.
Cache throughputs to IStore data.

Expiration Tracking

Remove item from the cache when they expire automatically.
Resets expiration time when accessed.

BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.3037)
Intel Core i7-8700K CPU 3.70GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 9.0.102
  [Host]   : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2
  .NET 9.0 : .NET 9.0.1 (9.0.124.61010), X64 RyuJIT AVX2

Job=.NET 9.0  OutlierMode=DontRemove  Runtime=.NET 9.0  
Concurrent=True  Force=False  Server=True

Method	Mean	Error	StdDev	Median	Min	Max
TestStoreCacheAdd	428.9682 ns	8.1727 ns	7.6447 ns	428.8080 ns	416.8425 ns	442.8046 ns
TestDataCacheAdd	0.0117 ns	0.0146 ns	0.0136 ns	0.0046 ns	0.0000 ns	0.0387 ns
TestStoreCacheUpdate	398.7848 ns	4.7262 ns	4.4209 ns	396.5581 ns	390.5663 ns	405.7355 ns
TestDataCacheUpdate	0.0363 ns	0.0293 ns	0.0401 ns	0.0194 ns	0.0000 ns	0.1821 ns
TestStoreCacheDelete	52.6298 ns	0.8885 ns	0.8311 ns	52.4582 ns	51.2603 ns	54.8418 ns
TestDataCacheDelete	36.1396 ns	0.7226 ns	0.8602 ns	35.9421 ns	35.2178 ns	38.2860 ns
TestStoreCacheGet	0.0168 ns	0.0165 ns	0.0155 ns	0.0149 ns	0.0002 ns	0.0555 ns
TestDataCacheGet	0.0109 ns	0.0103 ns	0.0097 ns	0.0096 ns	0.0000 ns	0.0324 ns

Added StoreCache

dc4effe

cschuchardt88 added Discussion Initial issue state - proposed but not yet accepted Suggestion Ideas and suggestions that could be used for improving clarity, organization or even performance. labels Jan 29, 2025

cschuchardt88 requested review from nan01ab, Jim8y and shargon January 29, 2025 06:26

nan01ab reviewed Jan 29, 2025

View reviewed changes

shargon reviewed Jan 29, 2025

View reviewed changes

tests/Neo.UnitTests/Collections/Caching/UT_StoreCache.cs Outdated Show resolved Hide resolved

Update tests/Neo.UnitTests/Collections/Caching/UT_StoreCache.cs

27e4ebc

shargon reviewed Jan 29, 2025

View reviewed changes

cschuchardt88 added 2 commits January 29, 2025 19:40

Updates and fixes

0406d34

Merge branch 'fix/storage-caching' of https://github.com/cschuchardt8…

a4d3b89

…8/neo into fix/storage-caching

Jim8y and others added 3 commits January 30, 2025 20:07

Merge branch 'master' into fix/storage-caching

1baf832

Added StorageCache

b99d252

Merge branch 'fix/storage-caching' of https://github.com/cschuchardt8…

7e0aa54

…8/neo into fix/storage-caching

cschuchardt88 added 3 commits January 30, 2025 21:46

typo

10a69d8

Bug fixes and optimizations

51572d8

Fixed memory allocation problems

a724168

cschuchardt88 mentioned this pull request Jan 31, 2025

[Improve]: scan operations for IReadOnlyStoreView and fix of Find with all 0xff keyPrefix #3688

Closed

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added `StoreCache` #3701

Added `StoreCache` #3701

cschuchardt88 commented Jan 29, 2025 •

edited

Loading

Jim8y commented Jan 29, 2025

cschuchardt88 commented Jan 29, 2025 •

edited

Loading

nan01ab Jan 29, 2025

shargon commented Jan 29, 2025 •

edited

Loading

shargon commented Jan 29, 2025

cschuchardt88 commented Jan 29, 2025

shargon Jan 29, 2025

cschuchardt88 Jan 29, 2025

cschuchardt88 commented Jan 30, 2025 •

edited

Loading

shargon commented Jan 30, 2025

cschuchardt88 commented Jan 30, 2025 •

edited

Loading

Added StoreCache #3701

Are you sure you want to change the base?

Added StoreCache #3701

Conversation

cschuchardt88 commented Jan 29, 2025 • edited Loading

Description

StorageCache

Features

Expiration Tracking

Benchmark

Type of change

How Has This Been Tested?

Checklist:

Jim8y commented Jan 29, 2025

cschuchardt88 commented Jan 29, 2025 • edited Loading

nan01ab Jan 29, 2025

Choose a reason for hiding this comment

shargon commented Jan 29, 2025 • edited Loading

shargon commented Jan 29, 2025

cschuchardt88 commented Jan 29, 2025

shargon Jan 29, 2025

Choose a reason for hiding this comment

cschuchardt88 Jan 29, 2025

Choose a reason for hiding this comment

cschuchardt88 commented Jan 30, 2025 • edited Loading

Running on a Raspberry Pi 4 model B

shargon commented Jan 30, 2025

cschuchardt88 commented Jan 30, 2025 • edited Loading

StorageCache

Features

Expiration Tracking

Added `StoreCache` #3701

Added `StoreCache` #3701

cschuchardt88 commented Jan 29, 2025 •

edited

Loading

cschuchardt88 commented Jan 29, 2025 •

edited

Loading

shargon commented Jan 29, 2025 •

edited

Loading

cschuchardt88 commented Jan 30, 2025 •

edited

Loading

cschuchardt88 commented Jan 30, 2025 •

edited

Loading