6

Could you please help me to understand 'direct-io-mode' in GlusterFS.

I'm getting better read results with direct I/O mode disabled - is it mean that in disabled mode it's writing/reading from cache - is it a system cache or GlusterFS cache? What's the recommended method? I'm using 4 nodes with distributed-replicated volume for web serving:

Write: dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
echo 3 > /proc/sys/vm/drop_caches
Read: dd if=tempfile of=/dev/null bs=1M count=1024
Read-Read: dd if=tempfile of=/dev/null bs=1M count=1024

Write           Read            Re-Read
67.5 MB/s       94.9 MB/s       94.6 MB/s   direct-io-mode=enabled
65.8 MB/s       230 MB/s        226 MB/s    direct-io-mode=disabled
HTF
  • 3,050
  • 14
  • 49
  • 78

2 Answers2

6

after a long search i've found this article. it's not glusterfs specific, but it may help to understand better: in short, direct i/o mode disable the file buffer implemented by the os for that mount point.

after reading it i've come to this conclusion:

Enabling or disabling may give you better transfer rate depending on which application is using your glusterfs volume: if you are using applications or databases that handle by themselves caches you are going to have some boost enabling it ( because of the missing overhead of a second buffer ). if you are just doing file accesses probably disabling it will increase your performances because of less network access due to local file cache.

Valerio Minetti
  • 333
  • 2
  • 7
-1

You enable direct-io-mode by"mount -t glusterfs XXX:/testvol -o direct-io-mode=enable mountpoint"??If so, the file was not cached in GlusterFS client kernel,in other words,the cache is system cache.But,the file was cached in GlusterFS server kernel.

user6481589
  • 1
  • 1
    Welcome to serverfault! Please take the guided tour. You have a question about how the direct-io-mode was enabled; it should be in a comment to the question, not an answer. I don't see that the rest of your post is an answer to the question. To answer questions usefully, it would be better to concentrate on recent unanswered questions than three-year-old questions with an accepted answer. – Law29 Jun 19 '16 at 14:33