Is it worth installing one of those optimized kernels out there?

This is the way. I was very recently hit by a linux kernel bug:

Having the zen, LTS, and mainline kernels installed made it fairly effortless to narrow my buggy behavior down. I’m able to stay up-to-date with the LTS kernel until the fix lands.

For my particular case, scripting, lots of videos and music, console emulation, and some older games, zen provides just a little boost. It’s tuned to be more favorable to foreground-centric ‘end user’ loads like I put on it, rather than ‘server loads’ where there are a ton of things that all need to be working in somewhat good harmony. (I pray I never again have to run a real db server on my personal machine.)

The ONLY downside, IMO, is that when doing anything that updates the kernels or drivers needed at boot, you’ve got to rebuild an new initial ramdisk image for each kernel. That can take a few minutes for each kernel.

YMMV very much. It’s relatively easy to install, try out, and see if they match your use case.