Implementatation of the stream functionality #12

rgallor · 2024-09-03T13:39:30Z

This pull request introduces a Rust application that initiates an Astarte device, establishes an MQTT connection to a running Astarte instance, and sends data generated from a set of predefined mathematical functions.

Key Features:

Connection Configuration: The application requires a config.toml file, which contains the necessary information to connect the device to the Astarte instance.
Data Generation: Users can define mathematical functions and their parameters either through command-line interface (CLI) options or by setting the corresponding environment variables.

This work is inspired by and builds upon what has been done in the stream-qt5-test repository.

Closes #3

The interface allows generic sensors to stream samples data in double format. The interface is the same as the one defined in the stream-qt5-test astarte-platform repository Signed-off-by: Riccardo Gallo <[email protected]>

Introduce: - `color_eyre` to have a structured report handler - `tracing_subscriber` to collect scoped, structured and async-aware log data Signed-off-by: Riccardo Gallo <[email protected]>

codecov · 2024-09-03T13:52:37Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

astarte-device-DEVICE_ID_HERE-conf/config.json

src/main.rs

astarte-device-conf/.env

astarte-device-conf/config.toml

src/astarte.rs

.cargo/config.toml

scripts/run.sh

joshuachp · 2024-10-17T15:28:23Z

src/astarte.rs

+use tracing::{debug, error, info};
+
+/// Stream Rust test node identifier
+const STREAM_RUST_TEST_NODE_UUID: uuid::Uuid = uuid::uuid!("d72a6187-7cf1-44cc-87e8-e991936166dc");


Shouldn't this be overridable? So you can test multiple connections to the message hub

As a first implementation, we decided to embed the UUID, consequently having only once connection to the message hub. However, I think this can be changed (either in this PR or in a new one) if we want. What do you think @harlem88?

Yep, you can change it in a new PR.

src/astarte.rs

src/main.rs

joshuachp · 2024-10-17T15:58:03Z

src/math.rs

+}
+
+fn random_spikes_sin(value: f64) -> f64 {
+    if value.sin() + rand::random::<f64>() / f64::MAX * 0.1 + rand::random::<f64>() / f64::MAX * 0.1


Pls explain

confused crab noises

In rust the range for float [0,1)

https://docs.rs/rand/latest/rand/fn.random.html

Yep, the division by f64::MAX can be avoided

Btw, if I understood correctly from the stream-qt5-test code, the random_spike_sin function is a spike (spike(x) = 1 if 0<=x<1, 0 otherwise) computed on the value of a noise sine (which is sin(x) + rand())

joshuachp · 2024-10-17T16:10:31Z

src/math.rs

+}
+
+fn noise_sin(value: f64) -> f64 {
+    value.sin() + rand::random::<f64>() / f64::MAX * 0.2


Same as above

joshuachp · 2024-10-17T16:13:49Z

src/math.rs

+    {
+        100.0
+    } else {
+        0.0


I think it's better to document this function since it's not clear from the name what it does.

From it, I would expect the sin(value) would be return, but I think the function actually returns spikes with PI frequency

Also I don't think the function is actually correct, looking at the original code:

} else if (m_function == QStringLiteral("randomspikessin")) { value = sin(x) + (((double) qrand()) / ((double) RAND_MAX) * 0.1) + ((((double) qrand()) / ((double) RAND_MAX)) > 0.999 ? 100 : 0);

joshuachp · 2024-10-18T08:48:31Z

I would check the other functions like sinc they don't seem to do the same thing as in the original code

rgallor · 2024-11-07T13:37:12Z

I would check the other functions like sinc they don't seem to do the same thing as in the original code

Yes, for this one I used the original definition of the normalized sinc function, which differs from the implementation in the stream-qt5-test

This required also the introduction of some environment variables and structures to handle the generation of the samples to be sent to Astarte. For simplicity, the math functions are the same as the ones used in the stream-qt5-test repository. Signed-off-by: Riccardo Gallo <[email protected]>

Use clap to start the application, also taking into account possible environment variables set by the user. Signed-off-by: Riccardo Gallo <[email protected]>

Now the configuration information to connect a device to Astarte are provided inside a `astarte-device-DEVICE_ID_HERE-conf` directory, in a `config.toml` file. Instead, all the information necessary to generate samples to be sent to Astarte are provided via CLI or environment variables. Also the README have been update to explain how to configure the application and build/run it. Signed-off-by: Riccardo Gallo <[email protected]>

Signed-off-by: Riccardo Gallo <[email protected]>

Provide the Atsarte configuration via environment variables or config.toml file, prioritizing env vars Signed-off-by: Riccardo Gallo <[email protected]>

Signed-off-by: Riccardo Gallo <[email protected]>

Give more flexibility to choose the mqtt or grpc Astarte connection configuration by using the CLI, environment variables and a config.toml file Signed-off-by: Riccardo Gallo <[email protected]>

Signed-off-by: Riccardo Gallo <[email protected]>

rgallor added 2 commits August 20, 2024 16:52

feat(interface): define a device-owned interface to send data

9fb9ed2

The interface allows generic sensors to stream samples data in double format. The interface is the same as the one defined in the stream-qt5-test astarte-platform repository Signed-off-by: Riccardo Gallo <[email protected]>

feat(deps): add deps for logging and error reporting

1d17222

Introduce: - `color_eyre` to have a structured report handler - `tracing_subscriber` to collect scoped, structured and async-aware log data Signed-off-by: Riccardo Gallo <[email protected]>

rgallor requested review from joshuachp and harlem88 September 4, 2024 12:48

rgallor force-pushed the feat/implementatation branch from 0ca718d to 5830171 Compare September 5, 2024 15:33

harlem88 requested changes Sep 9, 2024

View reviewed changes

astarte-device-DEVICE_ID_HERE-conf/config.json Outdated Show resolved Hide resolved

rgallor force-pushed the feat/implementatation branch 2 times, most recently from 0fcd3ec to a6f84a6 Compare September 9, 2024 13:31

harlem88 requested changes Sep 9, 2024

View reviewed changes

src/main.rs Outdated Show resolved Hide resolved

rgallor force-pushed the feat/implementatation branch from a6f84a6 to 498dd70 Compare September 17, 2024 15:02

rgallor requested a review from harlem88 September 19, 2024 14:11

rgallor force-pushed the feat/implementatation branch 2 times, most recently from d775c12 to 7c6b1b5 Compare September 25, 2024 13:15

harlem88 requested changes Sep 30, 2024

View reviewed changes

astarte-device-conf/.env Outdated Show resolved Hide resolved

astarte-device-conf/config.toml Outdated Show resolved Hide resolved

src/astarte.rs Show resolved Hide resolved

harlem88 requested changes Sep 30, 2024

View reviewed changes

src/astarte.rs Show resolved Hide resolved

rgallor requested a review from harlem88 October 1, 2024 15:57

rgallor force-pushed the feat/implementatation branch 2 times, most recently from 835921f to 04d4e6a Compare October 4, 2024 15:18

harlem88 approved these changes Oct 11, 2024

View reviewed changes

rgallor force-pushed the feat/implementatation branch from 04d4e6a to 34ba2a2 Compare October 17, 2024 15:01

joshuachp suggested changes Oct 18, 2024

View reviewed changes

rgallor added 6 commits November 7, 2024 15:37

feat(clap): add CLI to start the application

c0b1a87

Use clap to start the application, also taking into account possible environment variables set by the user. Signed-off-by: Riccardo Gallo <[email protected]>

chore(deps): update sdk to 0.8.4

e61140a

Signed-off-by: Riccardo Gallo <[email protected]>

feat(atsarte): provide astarte cfg

6393b2d

Provide the Atsarte configuration via environment variables or config.toml file, prioritizing env vars Signed-off-by: Riccardo Gallo <[email protected]>

refactor(log): use EnvFilter to log only the application logs

5edc336

Signed-off-by: Riccardo Gallo <[email protected]>

rgallor added 7 commits November 7, 2024 15:37

refactor(mod): simplify main by adding cli and astarte modules

3ba7e12

Signed-off-by: Riccardo Gallo <[email protected]>

feat(msghub): allow connection to the astarte message hub

6ae284e

Signed-off-by: Riccardo Gallo <[email protected]>

refactor(connection): decide which connection to astarte use

d59721d

Give more flexibility to choose the mqtt or grpc Astarte connection configuration by using the CLI, environment variables and a config.toml file Signed-off-by: Riccardo Gallo <[email protected]>

chore(ci): update Windows check with sqlite-static

391f51b

Signed-off-by: Riccardo Gallo <[email protected]>

feat(test): implement tests for the astarte config builder

06c0eb4

Signed-off-by: Riccardo Gallo <[email protected]>

refactor(log): simplify the set up of EnvFilter

67bf216

Signed-off-by: Riccardo Gallo <[email protected]>

fix(math): correct and simplify some math functions

ba6c64d

Signed-off-by: Riccardo Gallo <[email protected]>

rgallor force-pushed the feat/implementatation branch from 34ba2a2 to ba6c64d Compare November 7, 2024 14:51

rgallor requested review from joshuachp and harlem88 November 7, 2024 14:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementatation of the stream functionality #12

Implementatation of the stream functionality #12

rgallor commented Sep 3, 2024 •

edited

Loading

codecov bot commented Sep 3, 2024

joshuachp Oct 17, 2024

rgallor Nov 6, 2024

harlem88 Nov 8, 2024

joshuachp Oct 17, 2024

joshuachp Oct 17, 2024

rgallor Nov 7, 2024

rgallor Nov 7, 2024

joshuachp Oct 17, 2024 •

edited

Loading

joshuachp Oct 17, 2024

joshuachp Oct 18, 2024

joshuachp commented Oct 18, 2024

rgallor commented Nov 7, 2024

Implementatation of the stream functionality #12

Are you sure you want to change the base?

Implementatation of the stream functionality #12

Conversation

rgallor commented Sep 3, 2024 • edited Loading

codecov bot commented Sep 3, 2024

Welcome to Codecov 🎉

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshuachp Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshuachp commented Oct 18, 2024

rgallor commented Nov 7, 2024

rgallor commented Sep 3, 2024 •

edited

Loading

joshuachp Oct 17, 2024 •

edited

Loading