Nokia Lumia 820 - Lines that are straight in real life appear bent in a stitched image on Panorama. What can I do?

In order to stitch the images they often have to be warped somewhat to allow objects to fit smoothly into the final image. This warping causes usually straight lines to bend.