What's in my ZFS ARC and L2ARC caches?

Question

I have a simple ZFS setup at home, four disks mirrored, 8GB ram and 120GB Intel X25-M SSD for L2ARC. I ran zpool add poolname cache cXtXdX then generated load and eagerly tracked the warming process by running zpool iostat -v mypool. A few hours later, the L2ARC is full, but what exactly has been cached?

Can I peek in and see what's in my ARC/L2ARC? I understand that certain blocks would map to multiple filesystems (due to snapshots, cloning or dedup) but I'd still like to sample the statistics like these:

Type of block cached: metadata. iSCSI, data. etc.
Age: How long has each block been in the L2ARC
Which files: directories that have some of their contents cached
Filesystem breakdown: GB cached per filesystem
Location of non-volatile copies of blocks (which drive/vdev)

I'd imagine zdb and/or dtrace would be the tools de jour, but don't really know where to start.

There is a excellent [blog entry](http://blogs.sun.com/brendan/entry/test) by Brendan Gregg that answers some of your points. — user9517, Jan 30 '11 at 09:17
The blog entry by Brendan Gregg has moved: [ZFS L2ARC](https://blogs.oracle.com/brendan/entry/test) — Marco, Jul 13 '13 at 15:30

score 7 · Answer 1 · edited Sep 03 '12 at 06:32

7

I would probably argue that you should care not about what's in the cache, but whether the most effective things have been cached. Ie, is the ARC cache being used to its maximum efficiency?

I run a small-scale (9TB) ZFS implementation at work and I find Ben Rockwood's tool arc_summary.pl, and Sun's arcstat.pl, both detailed in this cuddletech blog post, to be immensely useful.

I know it doesn't answer your question directly about what exactly is in the ARC cache, but these should give you some solid data about how your cache is being used, and whether you're in need of more memory/SSD space.

edited Sep 03 '12 at 06:32

user9517

114,104
20
206
289

answered Jan 30 '11 at 12:43

growse

7,830
11
72
114

1

None of the links work anymore. However, they seem to be available at https://github.com/benr/solaris_tools/blob/master/arc_summary.pl and https://github.com/mharsch/arcstat/blob/master/arcstat.pl, respectively. – Marcel Waldvogel Dec 08 '20 at 16:04

score 2 · Accepted Answer · edited Nov 13 '12 at 17:41

2

Here's a script that's been updated to reflect the data in L2ARC.

See: http://blog.harschsystems.com/2010/09/08/arcstat-pl-updated-for-l2arc-statistics/

Source available here: https://github.com/mharsch/arcstat

edited Nov 13 '12 at 17:41

notpeter

3,505
1
24
44

answered Jan 07 '12 at 00:18

ewwhite

194,921
91
434
799

What's in my ZFS ARC and L2ARC caches?

2 Answers2

Linked