The concept of a Stack is fairly simple. Push adds to the top, Pop takes off the top. It's basically a LIFO (Last In, First Out) queue (an actual Queue would be FIFO - First In, First Out; Push adds to the bottom, Pop takes off the top.).
For the stack concept applied to executing a program, typically termed a call stack, Wikipedia has a
decent article. For specifics of the CLR execution stack, and the way a finally block is attached to it (as well as a lot more), Joe Duffy's
Professional .NET Framework 2.0 is a pretty good, easy reading intro.