I figured out something today I should have probably figured out some time ago.
Somehow I fixed it with scaling and other hacks.
When you add a button to a horizontal container, tell it to fill, and your background image looks squished and jacked up, there is an easy fix.
Here they are as they appear in the default configuration:
Now, they aren't supposed to look like that. On the right in the details pane, change the "Draw As" option to Image.
After changing the setting with any luck, you'll have some normal looking button images in your unreal engine games.
Hopefully, this helps someone else before they spend too much time trying to fix it.