During a recent podcast appearance, the frontman discussed wanting to avoid commenting on the public narrative surrounding his celebrity — including dating Swift last year — through his music