Question

[rvegele]

Hi Daniel,

Thanks for all the great and fun content you produce! I really appreciate and learn a lot from it every time!

I had a question with regards to the neuro-flappy bird example. And I hope you'll have the minute to reflect on this.

I'm trying to understand how the model actually fits/trains. It's not clear to me to what extent does the learning actually happen and when. I'm reading up on Reinforcement Learning and if I somewhat understand the learning process there I'm a bit confused here.

Does the frame by frame 'situation' matter to the ML model in this case, since the mutation only happens when the bird hits the wall? It seems that memory of past frames is not something that's taken into consideration. Or is it?

I'm guessing that the model here needs to see all (as much as possible) situations with regards to inputs to then be able to predict reliably it's actions. If the model has seen hole positions from top to bottom and it's own positions/velocities with relation to them it'll do well in those situations. But if any of the parameters would change e.g. canvas height - the model would have to be re-trained.

So again if you could reflect on how the ML part + GA in this example is actually working, with comparison to a more 'standard' ML approach with let's say a gradient descent or even Reinforcement Learning, I'd really appreciate it! Since I'm trying to understand why this works so quick and well and where the corners are cut comparing to the 'common' ML approaches.

Thanks again!

[rvegele]

I've tried removing the following lines from bird.js following my intuition and the model seems to do quite well.

inputs[0] = map(closest.x, this.x, width, 0, 1);

My thinking here is that the bird doesn't care how far it is from the hole during any time, since when the mutation happens we only take the current height with relation to the closest hole into consideration.