Hmm, I'll have to think about this one a bit.
First off, as for the lack of leaping strides of innovation, you havbe to remember, the game playing public is a fickle crowd, with many varied tastes.
While some things may appeal to you or this whole community, that can't be taken as "this is what every gamer wants." And being able to cater to those needs/wants is a difficult task in itself, the most any game can really hope for is to please the audience it's aiming for. With the MMO market that can be RPers, PowerGamers, Achievement Monkeys, PVP nuts, etc.
Some games try and incoporate as much as they can to cover all the bases, but when you look at each game you start to see it's largest selling point, something it does better than any other factor (within its own game that is). Also, taking bigger risks may yield bigger rewards (as the saying goes) it can also yield bigger downfalls. There's alot of money involved in the development of these games, so a company can stand to either make a lot of money for being daring or get a nice mention in an article somewhere with "at least they took a chance." which they can read as they're selling off all the server equipment and watching closed accounts whistle by as they have to shut-down the game.
Put yourself in their shoes, you'e the one responsible for the financial security of the company, so no only do their books, but your job, rests on the ability to make the correct decisions on green-lighting a project. You have two developers come up to you, both equal in skill, but with two wildly different ideas for MMO's. One is very, very innovative (Option A). Incorporating completley new ideas and game designs. The other is a more standard approach (Option B), maybe a fewminor innovations, but a lot of design that is already in use elsewhere, jsut with some minor tweaks so it's not an exact duplicate. Now you take both ideas, and do some market sampling. Option A has definately sparked some interest for all it's innovations, but the numbers show it to be more a more niche or cult hit than a mainstream hit. Option B, while hearing some grumblings of "Oh...more of THAT.." still pulls much higher numbers. Looking at strictly numbers (completley off the top of my head) Option A looks to have about a 43% of being a hit and financially successful, Option B has a slightly better 65% chance. Which do you choose?
I, for one, give kudos to comapnies that strive for different things in their games. It's the only way games advance, but it's also a huge risk, and alot of these companies have to look at the hear and now. For smaller comapnies with a "Nothing to lose and everything to gain", it may be a different story. For companies on the brink, it's much more pronounced. For larger mega-firms, it could be very well that they would take the felgercarb shoot and roll on the little innovative guy, as long as they feel they can afford and recoup the loss should it fail.
As for the "grindfests", well to me there are two resons for that. One, a lot of Asian developed games tend to play heavily into that role, it's just an aspect of their game design. Does it make a game inherently bad? To you maybe, but that's a matter of taste/prefference, not any indicationt hat the game is bad. A lot of western gamers like instant gratification, or at least much more rapid gratification, not all, but a majority, look at the numbers of western players on WoW vs. FFXI for example.
Now it also stands to reason that everygame is going to have some amount of "grind" in it. It's not exactly cheap for these companies to use, maintain and upgrade all those servers, plus the manpower involved with maintaining an MMO, so they need to do what they can to maintain player involvement, if that means installing some "grindfest" methodology, then that's what they have to do. Remember, one of the first goals of any company is to continue to make profit, whether that's good or bad is to be left to philosophers.
It's easy to call a company a "money grubbing -----", but that's the other side of the coin as to what they're there for. One side is to provide an enjoyable, quality product for their consumers, the other is to make a profit off of those same consumers.
As for the belief out there on so many of the gaming forums same companies never listening to their consumers? That's an ignorant and gross overstatement. For one, you have to consider the amount of input that they would have to deal with if they were to simply open the floodgates. But for the most part, you will find that there are ways in which the player-base does have input ont he games they play. Open or in-game surveys would be interesting, but not the only means. If a game has official forums, such as WoW, a lot of player input is taken into consideration. But they also have to be able to gauge how well it will be received by the entire population of their game, bot just by the 15 who posted about it. Outside of forums, there are almost always e-mails or even street addys that can be written to, get the support of the games community behind you. Discuss it on the most popular forums, where you're sure to get a lot of people's attention and therefore their voice and start a letter writing campaign.
The change of the Dragoon's two-hour ability had been something that had been on the mind of the FFXI dev's for a long time, but not until they began to hear more and more from the player community was it something that was implemented. And look at the upcoming Burning Crusade, ther was enough feedback from the player base that Blizzard is capping many of the instances at 25-man. This of course also works within their game design and doesn't destroy balance.
Probably one of the best phrases I can think of is: "Don't complain if you don't intend to do something about it." I see a lot of game forums with players complaining about this or that, but not doing anything outside of that to effect a change. THese companies want the player-bases business, they need it, and therefore strive to provide you with an enjoyable, quality experience. But that can't be expected to know what's on everyones mind.
As for the quality f the content, once again, that's personal prefference, not a definitive fact. I, for one amongst many others, find the plot-line of FFXI to be very enjoyable and the use of it fun. On the same token, using Fox's example, I could say that Dirge of Cerberus and every FF game after 9 are vapid wastes of time, but once again that would be my personal opinion, not making the games bad or junk, just not in my realm of tastes.
Which is one of the reasons I don't read game reviews unless they tap into specific technical issues (major bugs, glitches, etc.)
Anyways, just my .02¢