28
8
I just updated to to macOS Mojave version 10.14 Beta and I have noticed a new process called YaraScanService. The process is consuming too much RAM (around 10GB ). I killed the process using Activity Monitor but it came back an hour later.
Farabi Abdelwahed
Posted 2018-07-09T21:19:22.250
Reputation: 283
18
MRT/YaraScan is a MacOS prodvided antivirus-copyright tool. The reason for it's obscene memory usage is basically why OSX doesn't have a formal 'antivirus'.
More simply, YaraScan is one part of the 'volatility suite' here; https://www.volatilityfoundation.org/about
Do realise that a virus and illegally pirated material both are only detected by a 'signature' set of code paths and both often reliant on bugs, exploits and weak patching, so it's only to be expected that the strongest modern antivirus was grown from a copyright infringement detection tool.
YaraScan runs once after Mojave update, and then deletes itself. It has also been seen to persist on certain MacOS systems within MRT. The reason it uses so much memory is because unless otherwise programmed (as in it's an opt-out), a process that has to scan an large amount of files for an unknown sized file that might be encrypted into said searched files will use a large amount of inactive memory to save all decrypted scanned files for a limited amount of time incase they are needed again. Why? Because empty RAM is wasted RAM, I mean you still have to give it watts so why delete the stuff on it when something else doesn't want to be there? It takes 100x longer to get it back.
More importantly, if you Filevault or APFS, ALL of that data is encrypted and must be decrypted to be read. Many apps actually need launching and then scanning when they are loaded as many files can come together to form a threat in memory space as a single 'concurrent file'. Viruses can be partially stored in a dylib for a completely unrelated app.
The amount of time is actively decided by Grand Central Dispatch in your mac and as soon as you attempt to use a program that needs that logical RAM it will try to clear it. Note that Virtual Memory in this case should be large, as all that decrypted stuff is better stored there until you're literally out of space than deleted on a secondary pass shortly after creation repeatedly.
This is new behavior in the age of SSDs to maximize drive life over responsiveness. Current GCD behavior suggests that the slowdowns are from a fast CPU creating decrypted data faster than it can be written to disk and other requests to RAM having to wait for SSD/HDD to finish.
user1901982
Posted 2018-07-09T21:19:22.250
Reputation: 739
10So Apple is snooping on people's storage without their knowledge? How is this not front-page news a-la Sony DRM-gate? – dhchdhd – 2018-07-19T04:37:43.190
4YaraScan runs once after Mojave update, and then deletes itself.
It's running for me after a kernel panic, so I assume the system loads it from the recovery drive as needed. Just FYI. – Shelton – 2018-08-21T20:27:40.977
1Good to know this is a run-once deal. I just refreshed my OS after have a series of issues and then seeing an unfamiliar virus related program was troubling. – Dan Loughney – 2018-08-30T16:01:01.090
7It definitely did not delete itself after running on my machine. At peak, activity monitor showed it using 80.50 GB of RAM (my box has 32 GB physical ram). I let it run until the process disappeared. The executable still exists at /System/Library/CoreServices/MRT.app/Contents/XPCServices/YaraScanService.xpc. This is on macOS 10.13.6. – pjv – 2018-09-20T11:49:25.673
1YaraScan is now part of XProtect/MRT/Gatekeeper. MRT being literally Malware Removal Tool. Prior to 10.13 I believe it's in the installer, but I am on the same system and do not have it present. I will update this answer to reflect this. As for the large RAM usage, please bear in mind this is virtual memory, AKA a file on the disk of X size (with the bonus of that file always deleting on close). It will be much bigger than RAM due to the fact that MRT won't use much RAM to store decrypted bytes, just dump it onto the disk until you run out of space then worry about delete. – user1901982 – 2018-10-26T15:35:04.750
Windows "Malicious Software Removal Tool" has been running on most Windows PCs every month since 2005, so it's not unusual for consumer OSes to include a non-intrusive antivirus scan regularly. Windows does it monthly (and then gets reinfected quickly). – Christopher Hostage – 2018-10-26T15:59:14.083
1I bumped into this after finding out that yarascan was massively abusing my HDD basically reading twice the amount of data that is occupied. How do we shut it down. – Kaki Master Of Time – 2018-11-04T15:34:56.917
1Wait. It will do it itself. Forcing it to quit is basically ineffective.
As for reading twice the amount of data that is occupied? My only guess is it could be scanning empty sectors. Believe me you can delete it but it will be back. – user1901982 – 2018-11-07T17:44:41.730
7
It's running on 10.13.6 (17G65) also.
1054 66.3 2.1 62395936 359328 ?? Us 11:48AM 10:39.14 /System/Library/CoreServices/MRT.app/Contents/XPCServices/YaraScanService.xpc/Contents/MacOS/YaraScanService
Looks likely https://github.com/virustotal/yara
https://apple.stackexchange.com/questions/296339/mrt-process-using-large-unbounded-amount-of-memory
dhchdhd
Posted 2018-07-09T21:19:22.250
Reputation: 198
3Indeed. From what I can tell YaraScanService is only part of MRT, and at least in 10.13.6 MRT seems to run after every reboot. It may not always run the YaraScanService part of itself, but in my experience it does. – Greg A. Woods – 2018-08-08T23:28:59.710
4Such a great way of increasing obsolescence of older macs… Me: "My Mac is slow because of Yara!" Apple: "So you want to have viruses? Better upgrade your mac" – w00t – 2018-08-18T08:53:04.890
2I ended up removing it bc I never run untrusted code. Apple adding performance-impacting features without providing an easy way to disable them (eg plist setting managed by a new Security prefpane field) seems a bit uncool. – dhchdhd – 2018-08-31T11:56:44.447
2
It does not really consumes your RAM. It likely uses memory mapped I/O when reading those files, but that only means that file content is mapped to virtual memory space, it doesn't actually mean that physical memory is used. For actual usage you need to look at "Real memory size in Activity Monitor.
Matt K.
Posted 2018-07-09T21:19:22.250
Reputation: 21
1Actually it does use RAM (and also memory mapped I/O uses RAM too -- that's the whole idea!), causing already-used RAM to be compressed and/or pushed out to swap, thus further exacerbating the usability of the system while it runs – Greg A. Woods – 2018-08-08T23:18:02.323
That's not how memory mapped file I/O works. It just maps the file to memory address space, it doesn't actually allocate any memory for it. The address space on 64bit platforms is 2^64 large (in theory, depending on platform some bits may have special purpose), which is way beyond the capacity of physical memory. – Matt K. – 2018-08-10T06:16:53.567
1Memory-mapped I/O most definitely does "allocate" physical memory -- and thus it causes memory "pressure", forcing the kernel to compress and/or page-out otherwise active memory still used for other purposes. You can't put something into memory if you don't have somewhere specific in real physical memory allocated to copy it to. Virtual memory address space is directly mapped to physical memory whenever a process accesses it in any way, even if that particular area is backed by secondary storage as in memory-mapped I/O. – Greg A. Woods – 2018-08-10T22:17:53.357
Of course memory mapped files are accessed through physical memory, but those blocks are treated like a cache and they are first to go under memory pressure. No sane operating system paging implementation would compress or swap active memory pages while keeping significant amount of mmaped pages.
In this case the YaraScanService on my computer had over 20 gigabytes mapped but only used around 300MB of physical memory.
Basically claiming that memory mapped file I/O causes memory pressure is same as claiming disk cache causes swapping and memory pressure. – Matt K. – 2018-08-11T23:17:09.583
1Memory mapped I/O requires a lot more physical memory than the equivalent buffer cache, especially with some access patterns. If you look at the amount of memory pressure (in the graph in the Activity Monitor "Memory" tab), and the growth of compressed memory and/or swap usage while YaraScanService runs on any machine with a large and full filesystem and lots of other large processes running you will see that it causes a severe amount of disruption to the extent that it sometimes causes thrashing. Even when Chrome leaks memory and grows huge it isn't nearly such a pig as YSS. – Greg A. Woods – 2018-08-12T01:12:19.567
I suspect that it is mostly YaraScanService's own virtual memory that gets compressed -- nothing else can account for the increase in compressed memory while it runs. Meanwhile other processes will also of course have some of their VM compressed, while yet more is tossed out to swap. Unfortunately if those other processes are also busy then that forces the kernel to uncompress and page-in those needed pages, thus thrashing. – Greg A. Woods – 2018-08-12T01:16:20.790
Okay, so I mmaped a 33GB file and read it byte by byte. The result is ~300KB of memory used by the application 36.89GB Virtual Memory Size, 5.71GB Real Memory Size. There was absolutely no increase in memory pressure, so the 5.71GB Real Memory Size is all cache.The overal cache size before and after test is approximately 6.5GB. With 16GB RAM and 6GB cache seems pretty normal. So far very much as expected for memory mapped IO. That said, I'm not 100% confident if this is exactly same pattern as YaraScanService, I haven't seen it running in quite a while. – Matt K. – 2018-08-12T02:39:29.607
Running the test on multiple files simultaneously results in real memory usage in each halved. When one process is killed the other takes up back all available cache, and when relaunched the real memory usage of former process goes down. It works exactly as expected, no compressed pages, no thrashing, no additional swap used, no memory pressure increase. Pretty much behavior that one would expect from a sane paging implementation. Yes, caching almost 6GB is quite aggressive, but macOS is known for not wasting any available memory. – Matt K. – 2018-08-12T02:50:57.913
And to get back, memory mapped I/O at least on macOS really doesn't result in additional memory allocated. The memory used for it belongs to "Cached Files" section and basically takes all available (unused) memory. Which makes sense, memory not used is wasted. macOS is also quite quick to purge these pages if needed, so no problem there. – Matt K. – 2018-08-12T02:54:57.350
Well I'm quite confident that your simple test did not in any way even remotely mimic the I/O and memory access patterns abused by YaraScanService. That much is clear just from watching it run and drive other memory use into thrashing. – Greg A. Woods – 2018-08-12T19:31:30.833
BTW, in the last 30 days my iMac has never used more than 75% of available physical RAM for any purpose whatsoever. – Greg A. Woods – 2018-08-12T19:32:13.157
1
It's from the Apple Malware Removal Tool (/System/Library/CoreServices/MRT.app/). See In 10.14 Beta 2 - what does YaraScanService do?. See How do I remove Yarascan from osx? and what is the MRT app?. It's a result of the Mojave update and should stop scanning eventually. Don't remove the MRT.app if you rely on Apple's security features at all.
– user187561 – 2018-07-10T01:38:02.963Also see Just what do XProtect and MRT protect your Mac from?.
– user187561 – 2018-07-10T02:06:57.047Is there away to limit its ram usage ? or is it the expense of using a Beta version – Farabi Abdelwahed – 2018-07-11T12:49:40.103