Comparing Opensearch and Marten in Benchmark testing efficiency for inserting and searching data.

[Amit-Limbasiya-27]

Marten DocumentDB vs Opensearch

we are comparing the performance of marten and opensearch in terms of searching and inserting the data.
Here is the code for the benchmark testing.

What is the goal?
The goal is to test the DocumentDB of Marten for searching and compare it with the Opensearch.

We have the entity named PostEntity with having the structure like

In this Benchmark Testing we have 4 benchmark scenario to compare MartenDB vs Opensearch

1. Insert Comment
2. Get Document By PostId
3. Get Post
4. Insert Post

And the results are

Time                |Method                         | Mean       | Error     | StdDev    | Median     | Gen0      | Gen1     | Gen2     | Allocated   |
2023-11-24 11:11:04 |------------------------------ |-----------:|----------:|----------:|-----------:|----------:|---------:|---------:|------------:|
2023-11-24 11:11:04 | InsertCommentInMartenDb       | 138.283 ms | 2.9905 ms | 8.7234 ms | 137.456 ms |  750.0000 | 500.0000 |        - | 11143.67 KB |
2023-11-24 11:11:04 | InsertCommentInOpenSearch     |  10.674 ms | 0.4349 ms | 1.2687 ms |  10.501 ms |   93.7500 |  31.2500 |        - |   877.12 KB |
2023-11-24 11:11:04 | GetDocumentByPostIdMartenDb   |  16.268 ms | 0.2628 ms | 0.3324 ms |  16.145 ms |  562.5000 | 500.0000 |  62.5000 |  6636.38 KB |
2023-11-24 11:11:04 | GetDocumentByPostIdOpenSearch |   2.546 ms | 0.0493 ms | 0.0674 ms |   2.532 ms |    7.8125 |        - |        - |    77.09 KB |
2023-11-24 11:11:04 | GetPostFromMartenDb           | 183.935 ms | 3.6034 ms | 7.4416 ms | 182.433 ms | 1333.3333 | 666.6667 |        - | 15003.48 KB |
2023-11-24 11:11:04 | GetPostFromOpenSearch         |  24.136 ms | 0.4008 ms | 0.3553 ms |  24.156 ms |  437.5000 | 312.5000 | 156.2500 |  4635.55 KB |
2023-11-24 11:11:04 | InsertPostInMartenDb          |   3.766 ms | 0.0731 ms | 0.2000 ms |   3.709 ms |  171.8750 |  54.6875 |        - |  1619.29 KB |
2023-11-24 11:11:04 | InsertPostInOpenSearch        |   5.582 ms | 0.0983 ms | 0.0871 ms |   5.572 ms |  171.8750 |  85.9375 |        - |   1611.9 KB |

Question:
Why the MartenDB postgresql is not more efficient than the Opensearch.
What can we do to improve the performance of the Marten-DocumentDB.

[mysticmind]

@Amit-Limbasiya-27 See my thoughts/response below:

• First of all, why do you need to compare OpenSearch and Marten which caters to and solves completely different problem domains. I am really not sure why you are doing a benchmark comparing these two. In short, you are comparing "apples" to "oranges"
• I honestly dislike your click-bait issue title Opensearch Outperforms Marten in Benchmark testing efficiency for inserting and searching data
• Marten is a fancy client/functionality built on top of Postgres database. So to have efficient search/querying, the data which is being queried will need specific type of indexes to be setup and depending on what type of search you want, you may want to use the full text search etc. It is quite driven by the use case for which something need to be optimized. Please check https://martendb.io/documents/indexing/ in our docs. You will need to setup indexes on the data/fields being queried.

[mysticmind]

I am moving this to discussion.

[gfoidl]

Some random ideas (extending what @mysticmind said):

• use compiled queries
• use LightweightSession instead of LightweightSerializableSession (there's no need for serializable transactions)