Selecting (and sizing) the correct file system is more important than anything else, not only for security but for tons of other reasons people do not usually recognize. Without a file system all processing would go to null.
Very well put response by Olli, and the OP is much dated, but file systems are my pet peeve I could not stay away. superuser.com is not something I visited before, I am not an admin, but I signed up and I am going to visit more.
Things changed a lot since 2011, but even back then I formatted USB cards FAT, and used USB drives to carry 4Gb+ files around. The reason of course was compatibility not security (so much for S in SD, but I use passwords on my 7z's), and I never really carried anything bigger than a CD ISO, they were mostly for SQL scripts and daily-hourly diffs of already encrypted database snapshots squeezed to near death by 7-Zip.
These days I wear any SD out faster than anyone I know. I have a USB stick in some production machines at my employer for hourly automated backup, formatted FAT. I keep an eye on them every day though and - you guessed - back them up religiously by hand (they are off-line secured building ITAR stuff). SSD leveled some of the playing field but I still do not trust them as much as regular HD, and SD is worse than optical. They go bad in an instant and the loss is total.
Any file system which invites the host OS to write randomly to it (NTFS, Recycle Bin) is bad news for an SD. Also, unmounting it helps a lot, no OS is going try accessing unmounted storage, so any file system will do as long as the SD includes a script to unmout itself (one of the standard files on every SD on mine).
Reading an SD is still slow today, so I would recommend something like disk dump (dd) to grab the entire image when mirroring instead of file-by file. dd also let you know when there is something wrong, so your file manager won't go kaboom.
Of course, if your primary purpose is to extend the life of some penny-stock you are going about your business the wrong way. I do what I do not for extending the life of an SD but to keep it from going bad when I am not watching, and there is the difference.
I avoid ext4 or any journaling FS on SD because I do not care when they go bad writing to them, but it sure hurts when a day or so later I cannot read them!
@Olli Is NILFS2 a good choice for a SSD drive ? – SebMa – 2018-11-28T23:50:56.243
6Thanks, this is very informative! However, the "big red note" from the UBIFS website says: "One thing people have to understand when dealing with UBIFS is that UBIFS is very different to any traditional file system - it does not work on top of block devices (like hard drives, MMC/SD cards, USB flash drives, SSDs, etc). UBIFS was designed to work on top of raw flash, which has nothing to do with block devices. This is why UBIFS does not work on MMC cards and the like - they look like block devices to the outside world because they implement FTL (Flash Translation Layer) support in hardware." – gspr – 2011-02-20T13:21:12.927
3Nice answer, plus there is F2FS from Samsung, also very promising system, quite new tho. – lzap – 2013-06-23T15:00:56.040
16@gspr is correct: SD has a flash translation layer, and JFFS2, YAFFS2, LOGFS and UBIFS are all designed for unmanaged flash. The options for SD are traditional block device filesystems, like ext2 / ext3 / ext4. – Robert Calhoun – 2014-01-21T16:12:32.867