The Cake twins are not the product of infidelity. Carrot just sucks at keeping track of his in-laws.
Scootaloo is adopted, and is being raised by a single earth pony stallion. He does his best, and he really does love her, but the race and gender difference sometimes makes it hard for them to see eye-to-eye sometimes, and he has no idea what to do about her flight problems but worries about it a lot. Before that, she bounced around between foster homes, some of which turned out better than others. She has a photo of her biological parents, but her only real memory of them is when they gave her up, so she feels like they betrayed and abandoned her and mostly uses the photo as a dart board.
Lyra and her EqG counterpart are both expressly forbidden from traveling between worlds.