Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [deduplication]

94 questions
2
votes
1 answer

ZFS deduplication

This is my first post on this website so please be kind with me :) On Centos 7, using ZFS deduplication in the past on a storage pool, I moved all file-systems that dedup was on to another storage pool. But, doing a "zpool status -D " I can see that…
Nicolas-V13
  • 21
  • 2
1
vote
0 answers

Is Path-based Unoptimization Possible?

I have a deduplicated volume in Windows Server 2016 that some software is having problems dealing with. Specifically, Matlab 2015 and older couldn't read deduplicated files, and while updating to a current version allowed for the files to be read,…
Louis Waweru
  • 695
  • 9
  • 26
1
vote
0 answers

Deduplication Savings differs a lot between same two volumes

I run this robocopy command to copy all data from the existing volume to my new volume which will replace the old one in the near future: robocopy I:\ E:\ /E /COPYALL /PURGE /MIR /B /R:1 /W:1 /XD "Recycler" "Recycled" "$Recycle.bin" "System Volume…
donduc_jr
  • 73
  • 5
1
vote
1 answer

VHDX drive space being consumed

I have a host running Server 2012 R2, and a file server VM also running 2012 R2. The VM has 3 drives, OS.vhdx, file-1.vhdx and file2.vhdx. Dedupe is running inside the VM on file-1.vhdx and file2.vhdx. I have checked via file shares and no-one is…
morleyc
  • 1,120
  • 13
  • 45
  • 86
1
vote
0 answers

Deduplication assistance

We have consolidated a couple of drives and NAS systems of a former colleague before the disks failed (some actually showed signs of degradation). Now, we know that the colleague has done "RAID by hand" aka "copying stuff between the disks"... and…
Marco Schuster
  • 53
  • 7
1
vote
0 answers

How to calculate dedupe ratio per folder

Is there a way to determine the zfs dedup ratio on a per folder basis? I realize the potential difficulties in computing this as every block would need to be analyzed, but can such a method be scripted?
Dan Buhler
  • 456
  • 4
  • 9
1
vote
1 answer

Windows deduplication doesn't yield usable space

Dedup size picture Hi all, When I setup Windows depuplication, I get great savings. See the attached picture. 1.28 TB deduped down to 23 GB. However, the space savings are not real. Lets say that volume is 2TB. According to the screenshot, there…
1Geoff
  • 75
  • 8
1
vote
1 answer

How to find duplicate files against a reference directory structure in Linux

There are a couple of duplicate file finders for Linux listed e.g. here. I have already tried fdupes and fslint. However, from what I have seen, these will find all duplicates of the selected directory-structures/search paths and thus also…
packoman
  • 165
  • 1
  • 2
  • 8
1
vote
0 answers

Clarifying sizing of dedup table on ZFS when only some datasets are deduped

I am trying to be clear about how zfs dedup handles the case where some (but not all) datasets in a pool are deduped, from a dedup table/RAM impact perspective. I found this quote from the FreeBSD mailing list in 2012: "Note that only file systems…
Stilez
  • 664
  • 6
  • 14
1
vote
0 answers

Better to deduplicate inside VHDX on client or outside VHDX on host?

I have a 16TB array that I'm hosting in a virtual environment. It contains video and photos from our marketing department. The data resides on the Hyper-V host itself as a VHDX file on a 8-drive SATA array. Our file server is virtualized and running…
Mike Capps
  • 41
  • 2
1
vote
1 answer

18.1 TB Drive Shows .75 TB Free but only 4.66 TB Deduped on Windows Server 2012 R2

I have a Dell PE R520 with 18.1 TB drive (built on RAID 6, disk internal to server). I setup Microsoft's Deduplication functionality on this drive and began moving large amounts of material onto it. Now it shows that it has .75 TB free out of the…
David Mackey
  • 677
  • 2
  • 13
  • 28
1
vote
0 answers

Error 0x80070519 reading file: revision level is unknown

I have mounted a dedup volume in Windows Server 2012 R2. When I try to copy a file from the volume to another I get the error 0x80070519 revision level is unknown The volume seems to be healthy otherwise: PS C:\Users\Administrator>…
Ronnie Overby
  • 681
  • 2
  • 12
  • 24
1
vote
0 answers

Stop Windows Backup from making Deduplication Exclusion for backup drive

With Windows Server Backup after every backup, the server puts an exclusion in the Deduplication list for the backup folder. Is there a way to stop this behavior? If not, is there a way to get Powershell to remove the exclusion? Edit: The Powershell…
Kayot
  • 251
  • 1
  • 2
  • 7
1
vote
2 answers

Is there a way to get the cmdlet for data deduplication when running an Unoptimization job?

I deactivated data deduplication on one of my drives because it doesn't work well with the backup system. I then started an unoptimization job in Administrative PowerShell. After that I realized I didn't put the priority up high so the job runs…
Gopoi
  • 547
  • 5
  • 21
1
vote
0 answers

Is it safe to delete chunks.chk file while using SDFS?

I setup a temporary storage NAS using various hard drives and random components lying about my workshop. It is running CentOS 7, with a software RAID of 8 500 GB SATA drives connected as spare drives through cheap fake-RAID controllers. I then used…
0xhughes
  • 141
  • 5
1 2 3
4
5 6 7