Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev/repair develop #270

Closed
wants to merge 49 commits into from
Closed

Dev/repair develop #270

wants to merge 49 commits into from

Conversation

JonasFrey96
Copy link
Collaborator

@JonasFrey96 JonasFrey96 commented Aug 16, 2023

Keeping track of changes:
Improved the logging of the feature_extraction node, which makes it easy to keep track if the node is up and running.
image

  • Fixed small bug and avoided loading the model at the first timestep given that variable was not inited correctly before
  • Node continues running if the sim-time jumps back

The pipeline is alive:
image

  • Added within the status message how many nodes are valid

Slic features also works nicely:
image

  • rescaled traversability in only node was never published :) Upps. setting it to out_trav
  • removed old optical flow things given that we now never used them
  • Integrated the anomaly detection by robin - to have the model running setting the experiment params model.name to LinearRnvp. in the rosparams set scale_traversability to false - currently the network is not learning so this is maybe something to fix - also it uses a fix lower and upper bound that is hard coded for the traversability scale.

@@ -136,7 +139,8 @@ def inference(self, img: torch.tensor, interpolate: bool = False):
pad = int((W - H) / 2)
features = F.interpolate(features, new_size, mode="bilinear", align_corners=True)
features = F.pad(features, pad=[pad, pad, 0, 0])

# Optionally turn on image feature smoothing
# features = filter2d(features, self.mean_kernel, "replicate")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did smoothening feature help in some cases?

Comment on lines 30 to 31
std = 400
mean = 550
Copy link
Collaborator

@RobinSchmid7 RobinSchmid7 Aug 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This we should set in the LossAnomalyParams. These parameters were found with a test dataset per environment. We should update this during online learning with a rolling window from the traversed features.

@@ -26,13 +26,10 @@ segmentation_type: "random"
feature_type: "dino"
dino_patch_size: 16 # DINO only
confidence_std_factor: 4.0
scale_traversability: True
scale_traversability: False # This parameter needs to be false when using the anomaly detection model
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should add logic to set to false automatically

@@ -273,5 +360,16 @@ def load_model(self):
if __name__ == "__main__":
node_name = "wvn_feature_extractor_node"
rospy.init_node(node_name)

if True:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hacky :)

@RobinSchmid7
Copy link
Collaborator

Working now with slic components:
Screenshot from 2023-08-24 16-24-47

@RobinSchmid7
Copy link
Collaborator

RobinSchmid7 commented Aug 24, 2023

  • binary thresholding of images or rescaling
  • try different sampling approaches during online learning
  • comparison with SimpleMLP
  • comparison with dino 8 vs 16 features

@RobinSchmid7 RobinSchmid7 linked an issue Aug 30, 2023 that may be closed by this pull request
14 tasks
@RobinSchmid7 RobinSchmid7 linked an issue Sep 6, 2023 that may be closed by this pull request
@RobinSchmid7
Copy link
Collaborator

RobinSchmid7 commented Sep 11, 2023

Working version with anomaly detection, running average loss computation and online learning

Screenshot from 2023-09-11 10-03-47

@RobinSchmid7 RobinSchmid7 marked this pull request as ready for review September 11, 2023 08:05
@JonasFrey96 JonasFrey96 deleted the branch develop February 18, 2024 11:38
@mmattamala mmattamala deleted the dev/repair_develop branch March 9, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add packae dependencies Small issues to fix
2 participants