Beeps when pressing some keys

I keep getting beeps when pressing some buttons at some times. For example, I just got a beep when I pressed the backspace key on an already empty text box, but that doesn’t always happen. It also happens on some other keys.

Does anyone know what is causing this and how I can disable it?

Edit: It also happens when my volume is set to 0%.

So the sound comes from a beeper, not speaker. Use module_blacklist=pcspkr kernel parameter.