[FR] Ability to remove node from db

[pdxlocations]

It's important to have the option to remove a device from your db in case of a PKE key mismatch.

Draft started in https://github.com/pdxlocations/contact/tree/rm-node-from-db
The node is removed from the device db but I can't keep it from repopulating in the node list without restarting the app. I'm using Ctrl-d on the node list as the trigger.
@rfschmid any ideas?

[rfschmid]

Not sure what might be going on but it looks like it might not be something in the app that's keeping the node around. I stuck some logging around the call to globals.interface.localNode.removeNode() - printing len(globals.interface.nodes)} before and after that call give me the same number, so it seems like the interface object's node list still has the node even after calling remove. It also doesn't seem to be an issue of it just taking a second for the interface to update, as if I delete a second node after some seconds, the first one stays gone despite us refreshing the node list.

Some nuance to the usage of removeNode() that we're not understanding? Maybe we have to "commit" changes somehow first? I'm not noting anything obvious in the Python CLI implementation:
https://github.com/meshtastic/python/blob/master/meshtastic/__main__.py#L450

[pdxlocations]

I think I've narrowed it down to:

contact/message_handlers/rx_handler.py

Lines 27 to 31 in c100539

	# Assume any incoming packet could update the last seen time for a node
	changed = refresh_node_list()
	if(changed):
	draw_node_list()

[pdxlocations]

OK, realizing my last comment doesn't actually narrow anything down. I am noticing that the node sort (last heard) only updates when a nodeinfo is received, not for telemetry or position, which is interesting... Are we not always getting new information from the radio when calling get_node_list()?

[rfschmid]

@pdxlocations it really seems like either a bug with the Python Meshtastic API or something we're missing in its usage. I also read through node.py and mesh_interface.py and don't see anything obvious about updating the node list in response to eg the node being removed.

I tried this test script last night (you'll probably have to modify slightly to use the serial interface instead of TCP):

import meshtastic.tcp_interface
import time

iface = meshtastic.tcp_interface.TCPInterface("meshtastic.local")

idToRemove = list(iface.nodes)[2]
nodeToRemove = iface.nodes[idToRemove]

print(f"len node list: {len(iface.nodes)}")
print(f"calling remove on node: {nodeToRemove['num']}")
iface.localNode.removeNode(nodeToRemove['num'])
print(f"len node list after remove: {len(iface.nodes)}")
if(idToRemove in iface.nodes):
    print("We still found removed node in the node list")

print("Sleeping for 10 seconds")
time.sleep(10)

print(f"After sleep, len node list: {len(iface.nodes)}")

if(idToRemove in iface.nodes):
    print("Id still in interface nodes list")
else:
    print("Id no longer in interface nodes list")

iface = meshtastic.tcp_interface.TCPInterface("meshtastic.local")
print(f"Created new interface, len of node list: {len(iface.nodes)}")
if(idToRemove in iface.nodes):
    print("Id still in interface nodes list")
else:
    print("Id no longer in interface nodes list")

Output:

len node list: 73
calling remove on node: 861678720
len node list after remove: 73
We still found removed node in the node list
Sleeping for 10 seconds
After sleep, len node list: 73
Id still in interface nodes list
Created new interface, len of node list: 72
Id no longer in interface nodes list

So we still have the same number of nodes, and the ID is still in the nodes list after calling removeNode(), and also 10 seconds later in case we were waiting for some sort of confirmation or periodic update or something. It's only after re-creating the mesh interface that we see the results reflected. I hope we don't have to recreate the interface every time we do anything.

I am noticing that the node sort (last heard) only updates when a nodeinfo is received, not for telemetry or position, which is interesting... Are we not always getting new information from the radio when calling get_node_list()?

This has been bugging me too - the nodes list doesn't seem to update as much as I'd like it to, we can definitely get into situations where I've received messages or gotten a traceroute to/through a node in the last minute, but the lastHeard field in the node table on the interface still shows 30 minutes ago or whatever. I'm pretty sure if I close the app and re-open it, the lastHeard is updated - thus it kinda seems like that also depends on re-creating the mesh interface.

[pdxlocations]

Perhaps we could create a local node db dictionary that we reference and modify while the app is running. I suspect this is how the android/apple clients work since the their node list grows beyond the 100 node cap in the radio db.

[pdxlocations]

It's only after re-creating the mesh interface that we see the results reflected.

Hmm, we are only subscribed to meshtastic.receive, maybe we need to subscribe to meshtastic.node as well:
https://github.com/meshtastic/python/blob/03ac32258361d13139c0ab955265cbf9de6262ad/meshtastic/__init__.py#L36