Lag in performance mode when using FX

I'm having some issues in performance mode when using either the X-pad for FX or the beat pads. When Quantize is turned on for FX, there's about a .300ms delay - making the effect sound awful. Trans effect doesn't cut the sound at the right time for example. But in standalone mode, it doesn't do this. Everything works fine. 
I tried reducing my ASIO buffer size to reduce lag but that didn't help at all.
The PC my controller is connected to is more than powerful enough. It a desktop with a 9700k and 16gb ram with a 2070 super. It barely hits 25% with RB running. The Controller is connected to the PC directly with the supplied USB cable.
Is this normal behavior? I'm not sure what I should do. I obviously can't perform like this.