Make sure you back off the front roller with the allen wrench if you have the Bottom Adapter.
If your receiver is out of spec it is best to get it into shape. Most modern receivers are well made but some low cost and billet lowers can be way out of spec.
Usually if fitment is required it is just a few thousandths off in a couple areas. Take your time with flat files. If you are unsure have a competent gunsmith perform the work.