In addition to Moshe's answer, I'm going to provide an example with LUKS since some people seem unconvinced. Also, see here for why overwriting may not be 100% effective (although it certainly helps).
Example
Make a sparse file, create a filesystem, and mount it:
$ truncate -s 100G /tmp/device
$ mkfs.ext4 /tmp/device
$ sudo mount /tmp/device /mnt
$ sudo chown user:user -R /mnt
Make a few confidential files:
$ echo "super secret data" > /mnt/secret
$ echo "super secret data" > /mnt/confidential
$ echo "super secret data" > /mnt/top-secret
Get inodes for files:
$ ls -li /mnt
total 28
13 -rw-rw-r-- 1 user user 18 Nov 10 11:34 confidential
11 drwx------ 2 user user 16384 Nov 10 11:33 lost+found
12 -rw-rw-r-- 1 user user 18 Nov 10 11:34 secret
14 -rw-rw-r-- 1 user user 18 Nov 10 11:34 top-secret
Ensure files are written to disk, then get extents for inodes:
$ sync /mnt/*
$ debugfs -R "stat <12>" /tmp/device
...
EXTENTS:
(0):33793
$ debugfs -R "stat <13>" /tmp/device
...
EXTENTS:
(0):33794
$ debugfs -R "stat <14>" /tmp/device
...
EXTENTS:
(0):33795
Check those blocks to make sure the data is there:
$ dd if=/tmp/device bs=4096 skip=33793 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 1.9034e-05 s, 215 MB/s
$ dd if=/tmp/device bs=4096 skip=33794 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 1.888e-05 s, 217 MB/s
$ dd if=/tmp/device bs=4096 skip=33795 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 7.1178e-05 s, 57.5 MB/s
Remove the files:
$ rm /mnt/secret
$ rm /mnt/confidential
$ rm /mnt/top-secret
$ ls -l /mnt
total 16
drwx------ 2 user user 16384 Nov 12 17:34 lost+found
Format the device using LUKS, then create a new filesystem:
$ sudo umount /mnt
$ sudo cryptsetup luksFormat /tmp/device
WARNING!
========
This will overwrite data on /tmp/device irrevocably.
Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
$ sudo cryptsetup luksOpen /tmp/device encrypted_device
Enter passphrase for /tmp/device:
$ sudo mkfs.ext4 /dev/mapper/encrypted_device
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 26213888 4k blocks and 6553600 inodes
Filesystem UUID: 279e6c3b-a183-4a94-b06e-78db1665b2a0
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Now we have a new filesystem:
$ sudo mount /dev/mapper/encrypted_device /mnt
$ sudo ls -lR /mnt
/mnt:
total 16
drwx------ 2 root root 16384 Nov 10 11:37 lost+found
/mnt/lost+found:
total 0
But is our secret data still there?
$ dd if=/tmp/device bs=4096 skip=33793 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 1.8944e-05 s, 216 MB/s
$ dd if=/tmp/device bs=4096 skip=33794 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 2.2056e-05 s, 186 MB/s
$ dd if=/tmp/device bs=4096 skip=33795 count=1
super secret data
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 8.7082e-05 s, 47.0 MB/s
Conclusion
Unless you wipe the disk, it's probable that at least some of the old data remains there unencrypted.