I made a lot of tests today to find the root cause of that problem.
In the end I am not sure if I have found the root cause, but I changed two things that made the situation much better:
- As mentioned from @ricklinux and @Amsterdam4572 I activated Host I/O Cache. This only increased the performance a little.
- I moved the .vdi-file temporarily to an other hardrive and back.
Both things together helped to increase the performance to a satisfying level. The first start of the VM is slower than every following.
Thank you everybody for your ideas to solve that problem!