No, it's not possible to execute out of all flash memory. For example, say you have a serial flash chip that requires a number of instructions just to read a single byte out of that flash chip. How could you execute code from that flash chip?
Pretty much every PC has a flash chip on every RAM stick containing the RAM's specifications. You sure can't execute the code in that flash chip, since it's not mapped into memory and can only be accessed by complex sequences of code talking to the memory controller.
Or consider a CPU that has separate executable and data address spaces. A flash chip that was mapped into data address space wouldn't be executable either.
The details are very device-specific. But it's not uncommon at all for a device to have flash that is not executable.
Think about an SSD or USB flash drive. You can't execute code out of those without copying it to RAM first. (And embedded systems often don't have enough RAM to hold a copy.)
Also, some CPUs have performance requirements on memory that is executable. Some CPUs don't have the ability to insert wait states on program fetches but do on data fetch. So executable flash must be very fast. It's not unusual to use cheaper flash for data. (The DS80C320, for example, can tolerate slower data memory but executable memory must run at its full speed.)
1Could we have a bit more context? Does this have anything to do with an embedded system? Or is it referring to PC secondary storage (SSD/flash drive)? Or something else? – Bob – 2012-10-26T17:48:52.090
@Bob - embedded system, edited the post with this point. – Mike – 2012-10-26T18:31:28.250