164

I am running top to monitor my server performance and 2 of my java processes show virtual memory of up to 800MB-1GB. Is that a bad thing?

What does virtual memory mean?

And oh btw, I have swap of 1GB and it shows 0% used. So I am confused.

Java process = 1 Tomcat server + my own java daemon Server = Ubuntu 9.10 (karmic)

Christopher Perrin
  • 4,741
  • 17
  • 32
kapso
  • 1,983
  • 4
  • 14
  • 7

7 Answers7

183

Virtual memory isn't even necessarily memory. For example, if a process memory-maps a large file, the file is actually stored on disk, but it still takes up "address space" in the process.

Address space (ie. virtual memory in the process list) doesn't cost anything; it's not real. What's real is the RSS (RES) column, which is resident memory. That's how much of your actual memory a process is occupying.

But even that isn't the whole answer. If a process calls fork(), it splits into two parts, and both of them initially share all their RSS. So even if RSS was initially 1 GB, the result after forking would be two processes, each with an RSS of 1 GB, but you'd still only be using 1 GB of memory.

Confused yet? Here's what you really need to know: use the free command and check the results before and after starting your program (on the +/- buffers/cache line). That difference is how much new memory your newly-started program used.

apenwarr
  • 2,012
  • 1
  • 11
  • 11
  • 3
    "Check results before and after starting your program", alternatively, use USS (Unique Set Size) as returned by `smem`. – Hubert Kario Dec 31 '12 at 10:35
  • So is there a tool that gives the true amount of memory being used, tools that are not third party. – CMCDragonkai Jul 21 '14 at 08:23
  • @CMCDragonkai Yes, free. – deviantfan Jun 02 '15 at 09:57
  • 1
    If I start a java process via `java -Xmx16g RunLong`, which would reserve 16Gb memory for the java process, then in `VIRT` of top, it seems the 16Gb is counted. In this case, what is the type of this 16Gb memory, is that mapped memory or .. ? – Eric Mar 11 '16 at 05:15
  • 1
    @EricWang, that memory isn't mapped. It's just a request to OS to reserve memory which may be needed later (perhaps never). (At least) on Linux, this is an mmap call with MAP_NORESERVE and PROT_NONE flags (see os::pd_reserve_memory calls anon_mmap: https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/35ddd5d8df9a6266fb82926d20487323a0d1c360/src/hotspot/os/linux/os_linux.cpp#L3350); no memory is actually allocated by default - that is only done later at the point when the program really needs the memory to satisfy objects allocation requests. – Juraj Martinka May 27 '19 at 20:07
  • @EricWang try to run you program with `-XX:+AlwaysPreTouch` argument – razor Mar 04 '20 at 14:07
35

I found this explanation from Mugurel Sumanariu very clear:

VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card’s RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment.

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library.

SHR indicates how much of the VIRT size is actually sharable (memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES.

Dawid Laszuk
  • 103
  • 2
Franck Dernoncourt
  • 940
  • 1
  • 12
  • 28
  • 3
    I would perhaps only re-word "VIRT represents how much memory the program is able to access at the present moment." to something like "VIRT represents the size of the program's entire addressable space at the present moment." OK, that could still use polish. But the point is, "how much memory" can still give the impression we're discussing RAM, when VIRT has _nothing_ to do with RAM space. In fact, large programs will often have a VIRT size that's several MULTIPLEs of the total system RAM size — because that VIRT is almost entirely file-backed address regions (AKA "disk not RAM"). – FeRD Mar 22 '19 at 02:26
30

From the top(1) man page:

o: VIRT -- Virtual Image (kb)

The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out and pages that have been mapped but not used.

See `OVERVIEW, Linux Memory Types' for additional details.

(Prior versions of the documentation stated "VIRT = SWAP + RES." Where RES means RESident, or physicical memory used.)

Actually that's not correct (anymore). When it says "swap," that also includes files that the program has mapped into its address space, which may or may not actually be consuming real RAM yet. This memory is file-backed but isn't really swap.

VIRT also includes pages that have been allocated but not used for anything yet. Any page in this state is mapped to the kernel Zero Page (brilliant concept--you should look it up) so it shows up in VIRT but doesn't actually consume any memory.

amphetamachine
  • 832
  • 1
  • 8
  • 14
  • 2
    well thats interesting, so is VIRT = SWAP + RES, how come my SWAP usage is zero, while virtual memory for the 2 java processes is close to a 1GB?? – kapso May 04 '10 at 18:20
  • basically Top shows.... Swap: 1048568k total, 0k used, 1048568k free, 505728k cached – kapso May 04 '10 at 18:21
  • 16
    @user42159 This answer is WRONG! There's NO 'VIRT = SWAP + RES' in man top! `-m : VIRT/USED toggle Reports USED (sum of process rss and swap total count) instead of VIRT`. It's a pity that I can't downvote this answer. – duleshi Nov 04 '15 at 01:53
  • 3
    This answer is incorrect. USED = Res+Swap Size (from top Fields Management, accessed by pressing f key when in top. Also from top man page). – Jason S May 10 '17 at 23:34
5

VIRT column in the ps/top output is almost irrelevant to measure memory usage. Don't worry about it. Apache heavy load VIRT vs RES memory

https://stackoverflow.com/questions/561245/virtual-memory-usage-from-java-under-linux-too-much-memory-used

Community
  • 1
leonbloy
  • 2,028
  • 17
  • 23
  • Thanks, I got worried and confused, because while swap usage was 0%, virtual memory column is very high. And also I have only 1.7GB of the total 2.7GB physical memory used, while virtual memory is high? – kapso May 04 '10 at 15:56
4

VIRtual column of the top, refers to the super-space (super consumption space) of the process, which the process might not be actually taking at the run time. There is another column RESident, which refers to the actual physical memory/space allocated by the process, at the runtime.

The reason for difference, between the two, can be understood by the example: if the process is using certain library, then the library size, will also aid to the virtual-size. however, since only a part of the library would be used (i.e. some methods in use), so that will aid in resident-size.

Refer for More Info

parasrish
  • 141
  • 4
  • the [link](http://mugurel.sumanariu.ro/linux/the-difference-among-virt-res-and-shr-in-top-output/) provided in the answer points to irrelevant site – devp May 03 '22 at 10:46
  • archived link updated.thx @UdeshRanjan ! – parasrish May 04 '22 at 10:48
1

"VIRT" just address space, RES is the "real" memory, but the "SHR" (=shared) amount of "RES" is the part of RES that is shared with other processes. So for most processes, I believe that by subtracting SHR from RES gives you the amount of memory that is really attributable to this particular process.

Max
  • 111
  • 3
0

Linux supports virtual memory, that is, using a disk as an extension of RAM so that the effective size of usable memory grows correspondingly. The kernel will write the contents of a currently unused block of memory to the hard disk so that the memory can be used for another purpose. When the original contents are needed again, they are read back into memory. This is all made completely transparent to the user; programs running under Linux only see the larger amount of memory available and don't notice that parts of them reside on the disk from time to time. Of course, reading and writing the hard disk is slower (on the order of a thousand times slower) than using real memory, so the programs don't run as fast. The part of the hard disk that is used as virtual memory is called the swap space.

Linux can use either a normal file in the filesystem or a separate partition for swap space. A swap partition is faster, but it is easier to change the size of a swap file (there's no need to repartition the whole hard disk, and possibly install everything from scratch). When you know how much swap space you need, you should go for a swap partition, but if you are uncertain, you can use a swap file first, use the system for a while so that you can get a feel for how much swap you need, and then make a swap partition when you're confident about its size.

You should also know that Linux allows one to use several swap partitions and/or swap files at the same time. This means that if you only occasionally need an unusual amount of swap space, you can set up an extra swap file at such times, instead of keeping the whole amount allocated all the time.

A note on operating system terminology: computer science usually distinguishes between swapping (writing the whole process out to swap space) and paging (writing only fixed size parts, usually a few kilobytes, at a time). Paging is usually more efficient, and that's what Linux does, but traditional Linux terminology talks about swapping anyway.

Source: http://www.faqs.org/docs/linux_admin/x1752.html

user42198
  • 55
  • 1
  • 3
    This answer spreads the misconception that virtual memory is the same as swapping or paging. Using disk as an extension of RAM predates virtual memory. And there are lots of systems (such as most SoHo routers) that have virtual memory but do not use disk as an extension of RAM. (And this is not the answer to the OP's question either, since he's not using any swap.) – David Schwartz Jan 27 '16 at 18:34