There are 2 different 4 gB memory limits on 32 bit Windows client operating systems (Server OS's and 64 bit OS's don't count here).

They are:
- the 4gB virtual address space limit
- the 4gB physical memory (RAM) limit.

The 4gB Virtual Address Space Limit

Let's get the 4gB virtual address space out of the way first (as the physical memory one will take more time).
Each process in Windows is assigned a 4gB virtual address space.  
So, if you've got 50 processes running, you've got 200gB of virtual address space available.
This is a lot more address space than there is RAM on most systems!

In the 4gB address space, 2 gB is reserved for kernel stuff (OS, drivers, etc) and 2 gB is reserved for the user stuff (application files, data, etc)
It doesn't have to use all of this space, but it's there if it's needed.
The space allocation can be changed to 1 gB for the kernel stuff and 3 gB for the user stuff - but can only happen if 2 things are true:
- Windows has to be running with the switch for this enabled (\PAE or \3GB) 
and 
- the application that's being used has to be aware of these switches and able to make use of them.

The 4gB Physical Memory (RAM) Limit

32 bit client versions of Windows Operating systems are limited to 4 gB of RAM by design.  This happened way back in the olden days when no one expected there to ever be a need for that much RAM.  Future developments of these 32 bit client OS's have retained this limit in order to maintain backwards compatibility with BIOS's and drivers that can't deal with addresses above 4gB.  As such, 32 bit Windows client OS's will ignore memory above the 4gB limit.

Now, add to this the fact that the BIOS and some drivers will reserve address space for themselves in the first 4gB of RAM.  Note that this is address space, and not actual physical memory.  Link that discusses this (thanks to archie123):  http://support.microsoft.com/kb/929605/en-us

So, if you have 4 gB of RAM installed, and a driver reserves 1/2 gB for itself, Windows will only show 3.5 gB of RAM being available.  This is because the remaining address space for your 4 gB resides above the 4gB limit that 32 bit Windows client OS's recognize.  So Windows will ignore the 1/2 gB that's above the 4 gB limit.

Nope, there's no easy way around this.  The easiest thing would be to have the drivers reserve address space above the 4gB limit - but that causes drivers to crash and the system to become unstable (and there's no memory for it there anyways!).  The most reasonable way to manage this is to figure out what's reserving the address space, and use different hardware that isn't as aggressive in the way that it reserves address space.  And that's a lot of work, and a lot of expense just for a few more mB of RAM being available!

How Does Virtual Memory Relate to Physical Memory (RAM)?
It doesn't relate, but the Windows Memory Manager is responsible for keeping track of what's where and when it's there.
There are complicated rules for what gets to stay in physical memory, and what can be swapped out (hence the term "swap file").
Certain things have to stay in physical memory, while other things can be swapped out - or can be discarded since they can be read directly from disk.

The basic concept here is that memory is kept in "pages" (hence the term "page file").  And the Windows Memory Manager keeps track of what pages are where, and moves them around as needed.  In general, those pages that are needed for computing operations are kept in physical memory, while those that aren't needed right now are swapped out to the pagefile (this is an extreme simplificaton of what happens) - or are discarded.  Those that can't be "paged out" (not allowed to be swapped out) are located in the non-paged pool, and those that can be "paged out" are located in the paged pool.

Most of this info is available in the "Pushing the Limits of Windows" blog series by Mark Russinovich.
Here's the links to the first 3 articles:
1 - Physical Memory: http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx
2 - Virtual Memory:  http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx
3 - Paged and Nonpaged Pool: http://blogs.technet.com/markrussinovich/archive/2009/03/26/3211216.aspx