Generally speaking, most but not all of the previously unencrypted data will be recoverable if the encryption and formatting process does not involve overwriting every single byte on the underlying storage device or issuing a (properly implemented) ATA Secure Erase command. In the case of non-self-encrypting rotational hard disks, the time cost of a Secure Erase is similar to that of overwriting the entirety of the drive through software; in the case of SSDs, or self-encrypting rotational HDDs, the time cost for a Secure Erase may very well be trivial: just discard the encryption key or possibly clear the block mapping tables; the data would still be there and readable in the latter case, but there is no way to figure out which part goes where because it consists of randomly scattered blocks. Also note that many SSDs are self-encrypting under the hood no matter what you do.
A modern rotational HDD can sustain around 100 MB/s in purely sequential write throughput. This means that a 4 TB drive will take about 4 TB * 1,000,000 MB/TB * 100 MB/s = 4,000,000 MB / 100 MB/s ~ 40,000 seconds (about 11 hours) to complete a single full overwrite pass (overwriting large swatches of storage to wipe it is one of the few reasonably common truly sequential workloads). The sustainable write speed scales roughly with rotational speed, so a 10k rpm or 15k rpm server drive will be able to sustain a higher write rate than a 5400 rpm laptop drive. Time to completion obviously scales directly with storage capacity and achievable write throughput. SSDs are much faster but still have finite write speeds, and for large SSDs, the time needed just may be sufficient to somewhat accurately judge what happened.
Assuming you are using a rotational drive and that the formatting process took significantly less than 2½-3 hours per TB of storage, it is reasonable to expect (likely significant amounts of) data remanence.
According to Microsoft KB941961 Change in the behavior of the format command in Windows Vista and later versions, in Windows Vista and newer (that currently means Windows Vista, Windows 7, Windows 8 and Windows 10, and very likely the corresponding server releases as well) by default formatting does do a single full overwrite pass of the partition being formatted; the old, pre-Vista behavior of simply writing new file system metadata is now referred to as the quick format mode. (Hat tip to TTT for pointing this out.) However, I am not sure whether this is really relevant in the case of using TrueCrypt, so would prefer to err on the side of caution. The time the formatting took (for a rotational drive, did the formatting take around one second per 100 MB of storage?) can be used to infer whether an overwrite has likely been performed.
In the situation described, it seems relatively likely that common off-the-shelf data recovery software will be able to recover significant amounts of previously stored, unencrypted data. Some of it will have been overwritten and thus not be recoverable, but much may be relatively easily recoverable using just software solutions.
To protect against this for data that has been stored unencrypted, you need to do at least one full overwrite pass over the whole device that will hold the encrypted data (the container backing device, in LUKS parlace) before you start using it for that purpose. It's your choice whether to overwrite with just a fixed pattern, or random data; remember that a fixed pattern encrypted with a throwaway key and a good encryption algorithm becomes random data. Overwriting with a fixed unencrypted pattern may reveal metadata such as amount of data stored and its location on disk (which can be used to infer e.g. file system, which in turn in principle can provide known plaintexts).
Unless you are a nation-state actor, a single overwrite pass is almost certainly enough with modern drives, and if you are a nation-state actor, you would have access to better people to ask for advice than random strangers on the Internet.