How you take the club back can directly influence your downswing. Try this drill with the goal to be just inside the ball line on the backswing so you can drop just outside the ball line on the downswing.

Changing Your Flat Backswing