Why is the "Move left/right a space" shortcut slow, when "Switch to Desktop X" is fast?

This drives me crazy.

I am using MacOS Sonoma 14.4 on M1 Max macbook.

When using shortcuts to move left/right a space, not only the animation is slow, but then the time to interactivity is also slow. Whereas if I jump to the same space using the "Switch to Desktop" shortcut, it is fast as expected.

To reproduce:

1. Open text editor on both space/desktop 1 and 2
2. Use the same shortcut as I show here, which you can set up in Keyboard shortcuts under mission control
3. from space 2, compare the speed of going left to space 1 and then try to type as soon as possible, and compare that with doing the same but using the "Switch to desktop 1" shortcut instead.

Please tell me I am not the only one! I want to find a solution for this.