Overclocking is the process of increasing the clock speed of a processor beyond its stock setting. It can increase a processor's performance, but at the cost of increased power usage, heat generation, and the possibility of system instability or physical damage. Power users who enjoy tinkering with their computers often overclock their CPUs (and sometimes their GPUs and RAM) to get the most speed out of their hardware.
In order to overclock a processor, it must have an unlocked frequency multiplier (also called a clock ratio). Intel processors whose product numbers end in "K" are unlocked, as are most AMD processors. It also requires a motherboard whose chipset allows you to modify frequency multipliers and voltage settings; you can change these settings in the BIOS / UEFI, or within third-party software available for Windows.
Overclocking involves managing three variables — the clock speed frequency multiplier, voltage, and heat. The first step is to adjust the processor's frequency multiplier, which multiplies the motherboard's base clock (typically set at 100 MHz) to get the processor's effective clock speed. For example, a 3.5 GHz processor starts with a frequency multiplier of 35, and by adjusting that multiplier to 36, then 37, then 38, you can incrementally increase its clock speed. At some point, this will cause system instability or boot failure; you can then roll back the frequency multiplier to the highest stable setting.
You can reach higher clock speeds by increasing the voltage supplied to the processor. However, increasing the voltage also increases the heat the processor produces, which needs to dissipate before it overheats the computer. It often requires more robust cooling than the stock heat sink and fan packaged with a processor. Most overclockers upgrade to a third-party HSF or install a liquid cooling system to manage a processor's heat. Increasing the voltage too far will cause permanent damage to the processor (which is not covered by manufacturer warranties), so exercise caution when overclocking.
NOTE: Modern processors often include features that automatically increase their clock speed when additional power is needed, then lower it back to the default after the need has passed. This feature, called Turbo Boost on Intel processors and Precision Boost on AMD processors, provides some of the benefits of overclocking automatically without the extra configuration and risk.