Very small actually.
Him grabbing that supercarrier in his backstory (which ends up destroyed during Halo Wars 2 by the way) was a pretty big deal and more or less made up the bulk of his “armada”.
To quote an in-game log itself:
I don’t think so, sir. Throughout the war he was always about raiding, about building up his army. He never struck directly against the Covenant unless there was something to gain from it. Never took territory, never made demands or speeches. He kept as low as he could, a small target. Now his targets have become more focused, less hit and runs than surgical strikes. And he’s still recruiting. Even some of the Lekgolo are joining him now as well as more ex-Covenant who don’t want to stop fighting. The Covenant was never the target. They were just convenient suppliers. He’s just getting started.
The threat of the Banished was never their size or their strength. Even though pound for pound the Banished were superior to the Covenant, in a straight up fight the Covenant could just Zapp Brannigan the numbers until even Atriox is killed. The main threat of the Banished was that the Covenant would never know where any of them was until Atriox was already raiding another one of their supply depots, and by then he’s not only bolstering his resources but also converting more troops.