The problem with the auctioneer.lua file filling up is impossible to avoid when operating on multiple realms. To me the most important thing is the custom prices I use for selling things. Can you please make an option to have auctioneer only store custom prices and not all the sell prices of everyone? In that way I can keep my custom prices and use them on multiple realms without worrying about the file running full. Alternatively, make an option allowing the user to select storage on a per character level instead of on account level.
Well, if that's the main thing you care about you could simply stop scanning, then it'll only use your set prices and you won't be risking things...
I don't think we'd add something like this, mainly because working around the problem in any way at all isn't really going to help in the long term. As the game is expanded and more items are added this is just going to get consistently worse and we'd rather not make things even more complicated to support and explain in the long term.
The old Auctioneer will eventually die completely due to this problem, probably when Wrath of the Lich King introduces another massive amount of items and a bunch of returning players. At that point there will be zero choice but to move to advanced anyway. And, yes, we could do a lot of work and change the way Classic stores things so the problem doesn't come up and... Well, actually, we _did_ do that and Advanced is the result. I'm not being facetious. This exact problem was the main thing that led to Advanced being made the way it is in the first place. In the long term it _is_ the solution and we won't be doing other stopgap measures in Classic when we've already solved the problem in our opinion.
I know it's a pain to get used to something new and I also know there are things in Advanced that still need work and tweaking to be more comfortable to people, but we're getting there. The implementation of the Search interface for Advanced was one of the big outstanding issues. It's there now. It still needs work, but if more people use it and report problems here, the quicker it'll move towards perfection.
Everyone will have to move to Advanced eventually, I'd suggest that in your situation you are pretty much at that point now.
Sorry I can't tell you what you want to hear, but even if you found some way around the problem now it'd still come up in a few months with Lich King...