5

Official Oracle docs say that for a machine with more than 16GiB of RAM we need to allocate 16GiB of swap.

Our servers are RHEL 7 and have 256GiB of RAM.

DBAs do not want to see the system swap, so they want us to monitor the 16GiB of swap very aggressively.

I suggested that we double the RAM to 512GiB (the expense is approved), and disable the swap. However, this is against Oracle recommendation of having 16GiB of swap, even though we double the RAM.

Honestly, I don't get it how having 3% of swap makes any sense, or why if I am adding more RAM than we had swap, we have to keep the swap.

So, are there any good arguments I can use to justify running Oracle without swap?

P.S. The only reason I mention the doubling of RAM is to demonstrate the ridiculousness of the argument I am having a hard time arguing. What I am really looking for is arguments to justify disabling of the swap.

chutz
  • 7,569
  • 1
  • 28
  • 57
  • How much RAM does you application consume? – Orphans Nov 25 '16 at 09:05
  • You write "DBAs do not want to see the system swap" I read "DBAs want to waste RAM for useless content". – kubanczyk Nov 25 '16 at 11:50
  • @Orphans The "application" is Oracle server. It consumes as much as it is configured to use. And again, I am not the DBA, I am just arguing the case to not have swap on the system using common sense that everyone, hopefully, should understand. – chutz Nov 27 '16 at 23:24
  • I've updated my answer with citations from a variety of folks. – chicks Jan 03 '17 at 16:51
  • As an aside, the "amount of RAM" thing does relate to Oracle's inner workings (in terms of how it allocates buffers, temp tables, and so on). You'll need to buckle down with Oracle memory allocation specifics and talk directly to Oracle support to get your configuration right, or study up online, to determine if your intended memory configuration will work as expected. – T. B. Jan 03 '17 at 19:55
  • 2
    The idea that Oracle Database does not support using more RAM is, quite frankly, ridiculous on its face. Anyone who said that needs to provide documentation of just what exactly they're talking about. – Michael Hampton Jan 03 '17 at 20:05

4 Answers4

8

Disabling swap is a good idea if

  • your software can deal with out-of-memory conditions gracefully or limits itself to avoid OOM situations
  • having consistent performance is critical (when your system is swapping latency will increase which can be bad enough to make it effectively useless for many applications)

This sort of thing often happens with databases. I see it more with noSQL databases, but relational databases can suffer the same challenge.

There is nothing in the OS that requires swap to be there. Linux deals with this pretty gracefully by killing the last process that asked for memory. You don't want to get to that point so make sure you tune Oracle to use only ~90% of memory so there's some left over for the system daemons and margin for error. "Free" memory also gets used for buffering disk I/O which is a huge performance win so trying to consume more of memory by the database itself will eventually slow overall system performance enough to be counter-productive.

Even with systems that have a fraction of the memory from the question if the application is a database or a cache or similar system I'd default to no swap at this point.

Authorities

So that you're not just relying on my word for it:

Cassandra

Datastax explains for Cassandra:

You must disable swap entirely. Failure to do so can severely lower performance. Because Cassandra has multiple replicas and transparent failover, it is preferable for a replica to be killed immediately when memory is low rather than go into swap. This allows traffic to be immediately redirected to a functioning replica instead of continuing to hit the replica that has high latency due to swapping. If your system has a lot of DRAM, swapping still lowers performance significantly because the OS swaps out executable code so that more DRAM is available for caching disks.

riak

Basho explains for Riak that you should:

Ideally, you should disable swap to ensure that Riak’s process pages are not swapped. Disabling swap will allow Riak to crash in situations where it runs out of memory. This will leave a crash dump file, named erl_crash.dump, in the /var/log/riak directory which can be used to determine the cause of the memory usage.

mysql

Percona is sitting on the fence and provides useful caveats for both sides of the question. MariaDB disagrees with disabling swap:

While some disable swap altogether, and you certainly want to avoid any database processes from using it, it can be prudent to leave some swap space to at least allow the kernel to fall over gracefully should a spike occur. Having emergency swap available at least allows you some scope to kill any runaway processes.

ServerFault

A well received answer here includes:

I personally find a swappy system worse than a crashed system. A crashed system would trigger a standby backup server to take over much sooner. And in an active-active (or load balanced setup) a crashed system would be taken out of rotation much sooner. A win for the no-swap system again.

That answer has 22 upvotes today and is 4 years old. You can also see some other answers there extolling the value of swap, but there's no indication they're running databases. They don't have as many upvotes either. :)

squid

While they don't overtly recommend disabling swap the squid guys say:

Squid tends to be a bit of a memory hog. It uses memory for many different things, some of which are easier to control than others. Memory usage is important because if the Squid process size exceeds your system's RAM capacity, some chunks of the process must be temporarily swapped to disk. Swapping can also happen if you have other memory-hungry applications running on the same system. Swapping causes Squid's performance to degrade very quickly.

That's what you don't want to happen to your database.

redis

While redis officially recommends swap the users don't buy it:

First disable swap - Redis and swap don't mix easily and this can certainly cause slowness.

hadoop

As seen in this answer with the most votes on hortonworks community:

For slave/worker/data hosts which only have distributed services you can likely disable swap. With distributed services it's preferred to let the process/host be killed rather than swap. The killing of that process or host shouldn't affect cluster availability. Said another way: you want to "fail fast" not to "slowly degrade.

[....]

For masters, swap is also often disabled though it's not a set rule from Hortonworks and I assume there will be some discussion/disagreement. Masters can be treated somewhat like you'd treat masters in other, non-Hadoop, environments.

The fear with disabling swap on masters is that an OOM (out of memory) event could affect cluster availability. But that will still happen even with swap configured, it just will take slightly longer. Good administrator/operator practices would be to monitor RAM availability, then fix any issues before running out of memory. Thus maintaining availability without affecting performance. No swap is needed then.

I like this because it is talking about a Java app, but it reaches a lot of the same conclusions mentioned above about databases. Also, it mentions monitoring which is very helpful in tuning high performance applications. If you don't have numbers to compare everything is based on feelings which are harder to compare. Make graphs for every measurable metric - application-level latency and throughput down to CPU, disk, memory, and network graphs. Those provide the bulk of the real data you have to make decisions on.

chicks
  • 3,639
  • 10
  • 26
  • 36
  • 1
    Invalid generalization. All these arguments repeat over and over: "to ensure your OS never swaps area XYZ out, disable swap entirely". In reality it's a tradeoff. You are trading some RAM and you get something else in return. So it's an economic decision. – kubanczyk Jan 03 '17 at 16:59
  • 2
    I make it clear this is specifically for databases, caches and similar systems. This is not a recommendation for other systems. So who is generalizing? – chicks Jan 03 '17 at 17:01
  • 2
    You recommend for databases generally, when *this* database vendor (Oracle) says differently. They recommend to have some swap, presumably because they have unused areas of memory. That's the use of swap. To save RAM for real work and put useless crap on swap instead. – kubanczyk Jan 03 '17 at 17:13
  • 3
    I've never found Oracle to be that authoritative about their own database or Linux which is why I cited many other sources. If you like Oracle's answer go with it. – chicks Jan 03 '17 at 17:16
  • 3
    Anyone who has worked with heterogenous systems and architecture knows, for a fact, that Swap Is Not Your Friend (tm). ONLY in the case of monolithic systems does it create a stability advantage which could be argued outweighs concomitant performance losses, otherwise, in this day of replicas and failover protocols, swap is a tremendous detriment to performance. – T. B. Jan 03 '17 at 19:40
  • 1
    Thanks T. B. I totally agree. There's a lot of FUD and cargo cult beliefs to weed through to get the news out to people. – chicks Jan 03 '17 at 19:42
0

Alex on Linux has an interesting read on this subject: "Swap vs. no swap" http://www.alexonlinux.com/swap-vs-no-swap

Bottom line is that, without swap:

  • Your system will be less stable.
  • Disk access speed in your system will be slower compared to a system that has swap partition. Moreover, disk access speed will drop in the course of time.
bvdli
  • 19
  • 2
  • 2
    Alex is talking about desktop (specifically laptop), not server. The considerations are totally different. – T. B. Jan 03 '17 at 19:46
0

Why not keep a reasonable amount of swap for unused pages, and modify the vfs cache pressure to modify swapiness to only swap on the case of OOM condition. Also you could pin the oracle processes to main memory to ensure they do not ever touch disk. This satisfies the database not being impacted by hitting slow IO systems, and also allows you to free garbage from main memory to be used by the database and buffers and cache. It's the best of both worlds.

  • I understand that there are lots of things we can do about how swap is used, but my question is very specific about that I want to know what is wrong with running without a swap. – chutz Oct 15 '18 at 06:55
-1

This topic pops up frequently. Swap is just an extension of RAM, so let's buy more RAM, right? Wrong. Setup with 16 GiB swap and 512 GiB RAM makes perfect economical sense. Let me explain.

If you know the main software well, you know how much "stupid" memory it takes quite precisely. What "stupid" memory? Various code and data that initially shows in RAM, but will be never ever critically needed again. That is, the performance as seen by user will never suffer because this stuff is not readily available in memory.

Instead of fixing the software, you can just give it that amount of swap but not more than that amount. Yes, let it use 100% of swap. That's the point. Don't increase the swap, or you risk that some critical stuff accidentally ends up there. Document it, so people don't freak out seeing 100% swap usage. In case of Oracle that amount is 16 GiB and I can say from my experience it will get used, even on a 700 GiB box, and you will not experience swpin impacting the performance.

In effect you get 16 GiB of RAM to do real work and benefit your users. As of 2017, it reduces your organization's cost by approx $50. If your server has 256 GiB RAM, you configure swap and save $50. If your server has 10 TiB RAM, you configure swap and save... $50. See? Still the same.

Presently, it's always safe to have zero swap. It simply costs you that small $50, that's all.

If your organization is not capable of dealing with 100% used swap (for example a separate monitoring team, etc.) don't do it. If you make anyone think over this issue, you've already wasted the $50 of their time.

Some vendors really have zero wasted memory. And some vendors don't feel confident enough to estimate the amount of "stupid" allocations, so they say "zero swap" to avoid unknown problems just to save a bunch of your dollars. That's OK too! I would just trust the vendor on this, they support the installations, they know their stuff.

kubanczyk
  • 13,502
  • 5
  • 40
  • 55
  • 6
    Well... this isn't true in my experience. Especially with Oracle, but also with many other vendors, they typically don't have a clear idea of how their product is being used in the field, and so are largely clueless on how to configure the underlying OS/Hardware. As far back as 1997 I was routinely ignoring Oracle's recommendation on Solaris/SunOS because their strategies didn't fit our workload, nor our typical OS install and hardware. – T. B. Jan 03 '17 at 19:49
  • 1
    My question is not about the economical feasibility of swap. I specifically mentioned that the expense to get double the RAM is approved to avoid talking about cost. Just like if I want to spend money on an all-SSD machine, I shouldn't need to keep a slow spinning, but more economical hard disk around for things that don't really need an SSD. – chutz Oct 15 '18 at 06:58