=> Your system is behaving exactly as expected! (See this post for a detailed explanation.)
As @dalto already stated, the unlock time for a key-slot is calculated for your specific hardware when setting a passphrase and defaults to 2 seconds (once the system can make use of hardware assisted decryption).
Grub can only do pure software decryption or AES-NI, not SSE-accelerated decryption. Thus your first grub unlock stage will probably take multiple times longer than your set iter-time; multiple in this context meaning it could take 10 or more times longer than the calculated default 2 seconds. So the encryption will probably add about 20sec to the boot time for most users.
If you absolutely can’t live with this you basically have three options:
-
Don’t use encryption at all
-
Change to a setup with an unencrypted /boot
(Downside is you’ll have somewhat reduced security because your kernels and intramfs are potentially accessible to everyone.) -
Lower the iteration number of your current passphrase (see the aforementioned post)
(Absolutely not recommended; the ~2 second (respectively ~20 in grub) default (on your system) was chosen by developers for a reason. Lowering the iter-time can seriously compromise security. Combine that with a subpar passphrase and your system may just have become brute-forcable without the need of high-end machinery.)