Linux

45778 readers

1177 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
No misinformation
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago

MODERATORS

[email protected]

Auto kill memory leaking processes before swap death loop (lemmy.world)

submitted 2 months ago by [email protected] to c/[email protected]

14 comments fedilink hide all child comments

I'm using linux mint 21.3, and a process (brave aka chrome) sometimes memory leaking, so eats all the RAM, and then linux goes into swap death loop, when everything freezes (sometimes the mouse cursor is moving), and nothing can't be done, i can just see the HDD led blinking, and do a reset. Is there a way to make the system automatically detect swap death loop, and close the biggest ram user process, and so on?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 6 points 2 months ago* (last edited 2 months ago) (1 children)

This doesn't work to avoid thrashing. The kernel may invoke the OOM killer slightly quicker if you have no swap, so I guess that can sort of help, but it doesn't properly solve the problem.

On Linux, there's a thing called the page cache (aka disk cache): Every time (part of) a file gets read to or written from, that (part of) the file gets copied to RAM. The file is then kept there unless that RAM is needed for something more important. It is cached in RAM. But since it is also on disk, the kernel can drop the file from RAM anytime it wants.

If you're low on RAM, the kernel therefore evicts all of the disk cache, because it can, because those pages can be reloaded from disk if needed. This means it will drop all the programs you're running, the binary code. So any program you're running is constantly interrupted, because its code is not in RAM.

So it runs a couple of instructions, but oh no! Call to function foo() from glibc, but guess what? That's on disk. Queue wait for the kernel to load that. Oh now it wants function bar() from zlib, shit! Need to load that. Since loading stuff from disk is about as slow as running like a gazillion instructions, all your programs are like 1000x slower now.

This happens even with zero swap.

The correct advice is the one from @[email protected]: install/enable systemd-oomd or earlyoom.

[–] [email protected] 3 points 2 months ago (1 children)

Well that's technically correct, but if you're so dependent on disk cache for system performance that you can't live without it then you really need to look at doing an upgrade.

When a box swap deaths, it usually struggles to actually fill swap enough to have the kernel still OOM kill it at any point. Generally the massive performance impact of swapping just slows the app down to the point of being useless, along with the entire rest of the box. Disk cache should not be a concern during these abnormal events.

[–] [email protected] 7 points 2 months ago (1 children)

I'll make an appeal to authority (kernel developer working on memory management):

Disabling swap does not prevent disk I/O from becoming a problem under memory contention, it simply shifts the disk I/O thrashing from anonymous pages to file pages. Not only may this be less efficient, as we have a smaller pool of pages to select from for reclaim, but it may also contribute to getting into this high contention state in the first place.

And then he goes on to say what I said, that it can make the OOM killer quicker to react.

https://chrisdown.name/2018/01/02/in-defence-of-swap.html

[–] [email protected] 3 points 2 months ago

Interesting, thanks for the link!