I'll mention something I think is important to know as well. Operating systems that are designed for 32-bit programs/execution, on Windows for example, are given a maximum of 4GB of total addressable memory to use unless PAE is enabled. However, given that the core OS services and components also NEED this finite physical memory themselves, the operating system explicitly limits these applications to about half of this (~2GB). This isn't a lot of memory to use for taxing applications, especially when several intensive resource-hungry processes are running simultaneously.
With a 64-bit OS you have accessible memory that is exponentially greater (into the exabytes!) that is potentially usable by the operating system. For Windows 7 Ultimate and Windows 7 Enterprise 64 the maximum RAM limit is up to 192GB. These OS editions are considered desktop editions. The 64-bit SERVER editions of windows can ALWAYS and will always address higher amounts of physical memory because that's what they're designed to be used for... high workload resource-hungry servers. Windows Server 2008 R2, for instance, can use up to 2TB of RAM (Data Center and Enterprise editions). This is obviously far more than their Windows desktop edition OS counterparts. The role of the operating system (server/workstation vs. desktop edition) is just as if not more important than the 32 vs. 64-bit debate when it comes down to using system memory over the traditional 32-bit 4GB barrier.
Physical Address Extensions support (in Windows) can allow a 32-bit operating system to address up to 128GB instead of the normally limited 4GB last I checked. This works through some clever tricks (remapping memory etc) to extend them to be able to use > 4GB when that RAM is installed and available. Other "solutions" like ReadyBoost that allow you to use Flash drives etc to increase the total amount of system memory is slower but a good technology nonetheless. With a higher amount of addressable RAM your applications will be able to use more physical memory, which is fast, with less swapping to hard disk (ie. paging, virtual memory) which is definitely slower. Applications use the pagefile / swap to provide "virtual memory" by reading and writing (i/o) to your hard disk(s) to allow the programs to use less physical memory when running. This is great for a RAM-limited system but if you've got many Gigabytes or even Terabytes (in Server edition roles) of actual physical memory then using the swap file for page-pooled virtual memory will degrade performance. Using non-page pooled physical memory will always allow higher performance because of the i/o overhead cost associated with virtual memory. Operating systems that are 64-bit can allow you to use so much RAM (if you've got it) that you don't need to rely on your swap/pagefile nearly as much and in some cases (high-end 64-bit Server OS's) you technically wouldn't even need to use it at all (Windows crash dump reports are written here so I'd never recommend disabling it in most cases).
^
Disabling the pagefile / swap virtual memory support is something I'd recommend only if you have a high or reasonably high capacity SSD (Solid State Drive), provided you also have enough RAM in your system. Paging virtual memory (pooled memory) on an SSD can and eventually will wear out the drive, reducing it's lifespan, sometimes considerably. I believe this is being worked out in newer models as SSD's are becoming more advanced and durable. As a general rule of thumb all that excess disk access can wear it out though. This shouldn't be much of a problem in high-end current and next generation SSD's, we hope.