Android: Swipe event-listener causes scrollView not to fluidly scroll up / down.

I have a swipe event listener on my main window, this event has the following code:

if (e.direction === 'up' || e.direction === 'down') return true;
          if (e.direction === 'right' && $this.animated) $this.toggleSidebar.apply($this);
          else if (e.direction === 'left' && !$this.animated) $this.toggleSidebar.apply($this);

The up/down is to try and prevent this event from blocking the swiping gesture to fluidly scroll through a scrollView however no matter how I try to prevent it return; return true; return false; it completely blocks all of my scrollviews from fluidly sliding up and down with the swipe gesture.

Was wondering if anyone has a work-around or fix?