Change Memory Usage to macOS top(1)'s Method #1578
Conversation
|
On macOS Sequoia 15.2 |
|
I assume the bottom instance is the htop version from this pull request. I'll see what I can do when I get home to get it working the same as regular top before it will even be remotely ready for consideration. I'll be home tomorrow later in the day. If i don't do it by tomorrow it shouldn't be long after that depending on how difficult this is. |
|
This post on my fastfetch pr is relevent:
As you can see from your own testing, my What's even more weird is that not only is my sysinfo test program producing different output values for used and free memory than my pull requests for fastfetch and htop, both my fastfetch and htop pull requests are producing different values from each other as well. I find it really odd that I am using the same exact code in 3 different projects and yet it is somehow managing to produce three different and unique results in each one. Perhaps you know why any of this is so? |
Yes. From the titles of the terminal windows you can tell. The instance on the top (no pun intended) is the "original", i.e. before your patch, and the bottom instance is with your patch. Note that I can't tell whether it works correctly. As mentioned in the discussion in #1439 I take a neutral position. Except that the "18.0G/16.0G" on the memory bar meter looked really weird. |
My personal opinion is that you should not insist that your way to do it is the "right" way as people can have different ideas for presenting memory stats. The only useful, "canon" reference here is the top(1) utility from Apple. Also, as one of the contributors to htop, I am not interested in your |
Without context That being said, let's address the elephant in the room. As @Explorer09 explained, there's no single "right" or "wrong" way to display information, so everybody is doing it in a way that fits their intended message best. With Linux and thus top, the memory usage includes things like the filesystem cache, as these are "used" by the system. While this is true to the literal meaning of "used memory" it neglects the somewhat more important information for the user: How much memory is left to be used for actually doing work. The whole mess somewhat started with a small change after htop 3.0 when I experimented with tmpfs a bit and one problem with it is, that tmpfs was marked as filesystem cache. Thus I could have an old htop open showing a totally "fine" system, with barely a page to use left. Thus a change was introduced, that shifted the memory usage to a slightly more useful metric: What partition of memory is there actually available for user data that does not require any used memory to be freed up/swapped out before it can be used. This definition changed tmpfs mounts (of which modern systems use quite a few) to be counted as used memory, as in a low-memory situation these have to be swapped out instead of simply discarding them like you could do with the normal filesystem cache. And while this metric still isn't perfect, it gives somewhat more intuitive results when looking at the memory bar: The filesystem cache section now really shows how much can be freed basically for free, while other memory accounting covers memory that needs more work in low-memory situations. This change was primarily done for Linux, but is reflected on other platforms in a similar manner to be consistent for what users will see when they look at the meter. The last thing is to have a meter show one thing on one platform and something else on a different one. This isn't 100% possible with the many differences that memory accounting has across different OSes, but keeping it as close as possible sure helps with documenting the expected meaning for the user. Then, so why not stick with what top(1) is showing: Because this is discarding some insight that having used memory be presented properly solves. Showing this information allows the user to avoid low-memory situations a bit better, than they could do if you basically always showed them their memory was nearly full (as would be the case with the filesystem cache as that's taking up all the unused pages). I hope this explains where the values from htop are coming from and why we chose to divert from how top(1) does it. |
BenBE
added
needs-discussion 🤔
|
@Explorer09 I think you missed the point of why I shared my sysinfo program completely. It was a demonstration that how top calculates used memory can be easily copied and pasted into a different project and produce the same expected output value as top, which my sysinfo does. You are right, this ticket is about how htop does it and nothing else, I was just puzzled why the same code that produces the same results in two different projects (top and sysinfo) is producing a bogus value when it is put into htop. Since the question here is whether or not we want to use the same code as top in htop, it would make sense naturally why I'd raise the question: why is htop producing a bogus value, when we can already plainly see it doesn't do that in other projects, including top itself? By "bogus value" I do mean what this pull request is showing. I'm sorry for ever saying any of these other methods for getting RAM usage were wrong. I had no way of knowing there was no "technically correct" way of doing this. I'm not used to things in the programming world having multiple right answers that contradict each other. Edit: Explorer09, After re-reading your comment, it appears you perhaps did understand what I was trying to say the first time, but it sounds like you don't agree with doing this how top does it? Please feel free to clarify this. It's clear @BenBE doesn't agree with how top does it, wanted to be sure you both are on the same page. |
|
I see no further use in trying to make this pull request properly match what top shows if you all agree that is not the approach you wish to take. If you agree we should do what top does, then I can fix this pull request so that it goes that route. It's still close enough to the holiday season that I don't want you to feel pressured into giving an answer any time soon. We'll come back to this later. |
|
My position is neutral as I said. I would rather hear more reasons for changing the "used" value formula than "top(1) does it". |
Just that quickly I sort of forgot both of you had said that. My bad.
Well, that depends, since htop is cross-platform, it would make more sense to make it use the same method on all platforms, at least where possible. So whether you adopt what top does as the method of choice or not, a lot of that has to do with personal preference. I prefer what top does because it is what most unix-likes provide out of the box, but no worries if that reason isn't good enough to suit you. |
See here: https://github.com/apple-opensource/top/blob/e7979606cf63270663a62cfe69f82d35cef9ba58/globalstats.c#L433-L435
Edit:
Please note I'm away from home currently and won't be able to test on my M2 Mac Mini Pro until tomorrow at the earliest. I'm hoping the code change I made here does actually match what top(1) shows now.