Memory Hierarchy

In a perfect world, a computer would have a processor and a very large amount of very fast memory, as fast as the processor. But the problem is, high performance parts are ungodly expensive and nobody can afford a computer unless compromises are made. Cue the memory hierarchy, which is different "levels" of memory that have different performance rates, but all serve a specific purpose. A lot of research lately has been done to reduce the amount of levels or make accessing the slower one less painful.

Memory has five levels of speed and performance, and if what the processor wants isn't in one of these, it moves to the next level. These levels are as follows from fastest to slowest:

Registers

Typical access time: One clock cycle.

Registers are typically Static RAM in the processor that hold a single word of data. A word is usually the largest number of bits the processor can handle at once (but this is a very loose definition). The two most important registers that are found in all processors are the program counter and the status word. The program counter holds where the next instruction is. The status word holds the current state of the processor and/or the special results of an operation. The status word is needed for decision making. One that is important, though not exactly required, is the accumulator, which stores an immediate result from a math operation. Another one that's found on a lot of processors is the stack pointer, or where in RAM the stack is. Other registers are typically general purpose or special use.

Typically complex instruction set computers have around a dozen or so registers since they're geared more for accessing main memory. Reduced instruction set computers have a lot more registers.

Cache

Typical access time: Tens to hundreds of clock cycles.

Cache is also typically found in the processor, though high levels can be kept outside. This memory can be measured in kilobytes up to a few megabytes, depending on the level. Level 1 cache is usually reserved for data and instructions (kept separate for security/stability reasons). Level 2, 3, and beyond are progressively slower memory that hold pertinent data. Whatever this data may be depends on the algorithm used, thus it's possible for data or an instruction for a currently running program to no longer exist, then the processor has to go to the next level up until it hits Main Memory.

Main Memory

Typical access time: hundreds of clock cycles.

What most non-computer savvy people call RAM. Main memory is relatively fast and holds most of the data and instructions that are needed by the currently running programs. Main memory can also be used to hold parts of a program that aren't running right now, but may be needed later. This is called caching, and as soon as the operating system finds out you want to run the program, it will execute the program directly, appearing as if the program loaded much faster.

Secondary Memory

Typical access time: Millions of clock cycles.

Secondary memory is where data can be permanently stored. Typically this is a hard drive. As you can see, there's a huge gap between main and secondary memory. The hope is that solid state drives can close this gap, and so far in some configurations, have closed the access time gap far better than traditional hard drives. But the SS Ds are still much to slow to be comparable to main ram for general purpose activities of ram. Only High end SS Ds in raid are actually comparable to ram in PCs. However it's nice to now that SS Ds are much faster then previous soild-state cartridges in Consoles such as the N64 or GBA.

Removable memory

Typical access time: Tens of millions of clock cycles

Data that's intended to move around resides on removable memory. Floppies, CDs, thumb disks, all of these are removable memory. The biggest drawback of course is that they're horrendously slow, except for external hard drives using eSATA (although USB 2.0 has respectable speed).

Example

A way to comprehend all of this more easily is to pretend you have an open notes physics exam. You also decided to bring a cheat sheet.

• A register would hold the current formula you're using right now. You'll promptly toss this out of your head when you're done.
• Cache would be holding the formulas you memorized. Occasionally you'll have a brain fart.
• Main memory is the cheat sheet you have. It holds all the pertinent information you could use. But you might forget something minor or you're not quite sure how it worked...
• Secondary memory is your notes. This may have a condensed version of what's in your book, sort of like how installing a program doesn't exactly copy everything to the hard drive.
• Removable memory would be your books. Since you couldn't take your books, you could lack the necessary "program" to solve your problem.