Discord is a sugar daddy boomer

@Background Pony #27BE
Knowing Discord and his love of chocolate (Chocolate Rain and Chocolate Bars) they probably are…

@Darjetii  
Discord might find anything but coins. Even stuff that wasn’t in your couch cushions to begins with.

@mathprofbrony  
I kinda like your second idea! As long as he doesn’t start digging through my couch cushions for lost coins! :P

@Hopeful_Ink_Hoof  

 
Celestia did say at one point that she has use for his magic so he could get an allowance for doing magical chores for the kingdom and, as you said, its unlikely that he has any real expenses so the allowance likely builds up over time
 
Plus I don’t think the show staff wanted to depict him stealing

Maybe he just knows where some natural buried gold is, and is teleporting it in. If it’s stamped with his own face instead of pretending to be an Equestrian coin, then he basically paid them with raw gold that he mined using his own magic. As long as it wasn’t part of somepony else’s claim, he could have a totally legitimate source.
 
Alternative 2: He acquires gold bits that are honestly lost, as in misplaced by their owners – a very random source of income! In this scenario he would be employing legal salvage rights.

@laofuzi  
And then she gets the blame for it.

He totally stole that from the Canterlot bank

Gold isn’t worth much in Equestria. After all, ponines didn’t even react when AJ found a treasure filled with gold. Or those are bits, in which case it’s a bit much.

Hopefully they are not chocolate coins

@Hopeful_Ink_Hoof  
Well, either he’s creating them ex-nihilo, he has a bank account or dragon horde or allowance or something normal and boring and undercutting the much of the plot of that episode, OR he could just be transporting them directly from the Royal Vaults without asking, on the assumption that “Celestia ought to be doing that and he’s totally entitled, no really” Or maybe he just collects them randomly from every vault, stash, and horde in Equestria, just to keep things “fair” ;)

@Hopeful_Ink_Hoof  
I wish I could get paid for not doing things.

Are we sure that he simply conjured them magically though? I mean, Princess Celestia could give him a regular payment as an allowance, stipend, or incentive for not causing kingdom-wide chaos. Then he likely doesn’t have any bills, seeing as most of the time, he can get what he wants with a snap of his fingers and lives an a realm of his own. With that last one, I doubt that any one would try and force him to pay property taxes. So, for all we know, he could have an account somewhere where bits just build up that he never uses, and since he has no real interest or use for money, he just drops a large amount of them on the rare occasion he actually decides to buy something.