@Background Pony Number 17
 

 
Heh, that reminds me of the dragon in the Discworld novel Guards! Guards!
 
The basic idea is that you have two species of dragons on the Discworld: the swamp dragons (which are more or less what you would need to have a “realistic” fire-breathing dragon, essentially an extremely complex biological chemical plant that’s very unstable and prone to exploding) and the nearly mythical noble dragons, which are the huge, armored and terrifying things in myths.
 
A noble dragon shows up in the novel, and it’s pointed out throughout how impossible the animal is – how can it possibly fly will that armor on while barely flapping its wings? How can it leap off of a tower and turn a fall at terminal velocity into an effortless upward climb? How can it breathe fire hot enough to torch a building in one go without burning its own lips off? How can a house-sized reptile possibly survive on a diet of one virgin every other month?
 
To make a long story short, it turns out that noble dragons got around the limits of biology and physics by evolving to eat magic – to sustain their impossible biology, they need to have access to a steady source of magic to essentially allow themselves to bypass the laws of physics. The dragon in the novel did this by siphoning off magic from the local wizards’ university, but if Equestrian gems are essentially crystallized magic like you say, then I can imagine Equestrian dragons having developed along similar lines.

@Frustration in Excelsis  
Yeah, I’ve had similar notions about the origins of gems in Equestria. My take on it is that it’s magic they’re feeding on. Magic is everywhere in Equestria, so it’s probably flowing through the Earth as well. When the Earth magic becomes concentrated in an area, the base rock transmutes into precious gemstone.
 
This also explains why dragons like to eat gems–they can absorb and “digest” the magic bound up in them.

@Background Pony Number 17
 
The way I like to headcanon this is that while gems are not necessarily organic in and of themselves, they grow in a very organic way.
 
Essentially, as I see it, Equestrian gems grow through naturally occurring magical process that cause the gradual accumulation of minerals in earth and rocks, binding molecule to molecule and microscopic crystal to microscopic crystal to eventually cause the growth of large, pure, well-formed and faceted gemstones. The result is that you get pockets and deposits of gems scattered more or less everywhere growing in a way much more similar to a plant’s than a real-life stone’s, “gathering” material from the soil or the rock matrix and slowly adding it to themselves. It’s not a fully organic process – gems don’t metabolize or reproduce, for instance – but it’s similar.
 
The bit where they stick out of the cave wall like mushrooms is a bit more difficult to explain – perhaps it’s a combination of loss of surrounding material to the crystallization process and the increasingly large gem being “squeezed” out into the open.
 
The fact that gems seem to be much more plentiful in some places than in others I ascribe to the fact that to grow large and to be plentiful, gems require both a substantial amount of raw material (perhaps renewed by upwellings of magma and/or mineral-rich water) and of the right kinds of naturally occurring magic to promote the growth. When these factors combine, you get hotspots like this cave here.
 
The convenient thing with this sort of system is that as long as the favorable conditions last, gem deposits can never become truly tapped out. Once you’ve harvested an area dry, you wait however long it takes – and it could take a while – for more gem “seeds” to form in the rock and grow, until one day a new crop of of gems is ready for harvesting.
 
On a slight tangent, I think that one of the biggest factors in favor of gems being something that grows naturally in Equestria is the existence of a plentiful dragon species. Considering how big dragons get, a single dragon must regularly go through stupendous quantities of gems to keep themselves going. The amounts needed to continuously feed an entire species day in and day out must be absolutely enormous – and just going by the number of migrating adults in “Dragon Quest”, there are many dragons around. If gems were not renewable, the ancestors of the dragons should have tapped out almost every deposit in existence a long time ago: if it’s something they habitually feed on, and logically should have been feeding on in their prehistory and thus presumably before rock farming was developed, the most logical explanation is simply that the gems they eat renew themselves.

The gems are literally sprouting out of the cave walls like mushrooms. Further evidence for my headcanon that gems in Equestria are organic.