Literary Agent Janet Reid has a great post here on the reasons she turned down fifty queries, along with stats on how many of the queries fell into each no-thanks category. Aside from the fact that this is an excellent chance to get into an agent's head and see queries from her POV, it also serves as a quality assurance checklist for the query-writing author.
Let's run through the list of her reasons (quoted from her post), and I'll give you some ideas on how to tell if your query will get bounced for the same:
Thinly disguised novel that's really a memoir
Many things can trigger a novel-wrapped memoir, and writers usually go through at least one of them: a dysfunctional family, an unhappy childhood, a wretched adolescence, a lousy marriage, loss of a loved one, a career meltdown and/or a terrible divorce. So if the prominent features of your story and/or your protagonist bear a striking resemblance to you/your life, you've likely gone memoir in your novel.
There is nothing wrong with writing a memoir; I don't particularly care for them but there is a market, mainly celebrity-driven, for them. That said, a memoir is not a novel. Presenting it as a novel probably feels the same way it did to use code in your high school diary so that if your little brother snitched it from your room and picked the tiny lock he still wasn't be able to figure out who Chipmunk, Sassafrass and Mr. Whoopsie were. Don't assume the agent or the reader has to be treated like your little brother.
Recitation of events
Could your query pull double duty as a chapter summary or a synopsis of your novel? If so, you're event-reciting, not pitching. This type of play-by-play query often comes from an unwillingness or an inability on the part of the writer to condense a story into a pitch. A pitch is not a breakdown or a summary of the parts; it's a brief presentation of the whole.
Example: I could tell you I have a front end, a custom modified engine, tires, a chassis, a metallic flake scarlet paint job, a steering wheel, high-grade fuel, leather-upholstered seats, brakes, headlights, a trunk, a license plate and a rearview mirror. Or I could tell you I have a little red sportscar so fast that no cop can catch me. Now, which do you think is more interesting? Which better relates the concept of the object?
Interesting concept/query so over written it boded ill for novel
You can spot a story that suffers from over-writing (or over-workshopped writing, or over-critiqued writing) from a mile away, because by the tenth or eleventh paragraph the reader become losts in the writer's strange, complicated and mostly unidentifiable Novel LaLaLand. Same goes for the query.
I've seen a couple of these. Generally the writing is mysterious, beautiful to look at, and mainly incomprehensible; the writer tosses around enigmatic-sounding phrases like coalesced chiaroscuro conflicted characterization or denuded dichotomous delineation departure to show how well they spent a hundred grand of their parents' money on that MFA in the signature block.
No one in NY offers you a contract because no one can understand your novel, your concept, or you. Simplify, use common language, let your concept shine without loading it down with sparkles, and stop trying to outword Umberto Eco. Over-writing won't sell your book, and I think it annoys Bert.
Spent so much time telling me what the novel isn't, didn't tell me what it was:
The pitch is about your book, not about everyone else's books. An agents business is to know what's been published; they will be able to tell, right off the bat, what your novel isn't. They don't need a checklist provided upfront by you.
Measuring the ewww-factor of your query is pretty simple: you should be able to read your query letter out loud in a crowded public place and get the people around you to listen. If instead they would throw up, run away, or have you arrested, you need to consider toning it down. Way down.
I think the author is seriously deranged
A long rant or a short manifesto presented as a novel query might make an agent think this. Common inclusions in a query that are warning signs: obvious ax-grinding, bewailing the state of the Presidency/the country/religion/sex/Publishing, insulting a majority or a minority, a laundry list of heart-breaking personal problems, emotional blackmail, the state of your finances, etc.
If you are pissed off at the world, that's okay, we all get to that place at some time or another. But a query is not a soapbox or a therapy session, so don't try to make it one. (P.S. Don't tell Janet that we're all a little deranged.)
sadly out of touch about publishing time lines
Janet explained this one better than I think I can, but here's my shot: because books take so long to get into print, you need to think ahead. If you're writing something that was popular five or ten years ago, and there are no signs of a resurgence in that popularity, you're going to come off sounding dated and out of touch. Make sure your novel has enough wow factor to be considered contemporary two years from now.
Sadly out of touch self-help book
I think this is the same as being out of touch about publishing time lines, just with a specific corner of the market. But the self-help market is like any other genre, in that it's just as subject to trends and categories and such. If I were going to write any kind of self-help, the first thing I'd do would be to extensively research the market, see what's out there and what people want to read.
If you have to convince a friend in the biz to front your query, you're saying "I know the secret handshake." The problem is, there is no secret handshake.
blatant ripoff of another popular book or movie
Knockoffs are always being published, so I don't agree with this 100%, but I know a lot of writers go for writing clones instead of knockoffs. The difference between original and ripoff novels is pretty obvious, but if you're still not sure, look at it this way: Since my SF series predates James Cameron's Avatar by ten years, and has nothing to do with his movie, I am allowed to keep putting tall, exotic blue people in my books. If you want the same privilege, you'd better have a more compelling reason than you adored Avatar so much that you saw it fifty or sixty times.
Don't want to read this/cliche characters and plot
This likely comes from too many writing workshops and not enough original thinking, and the writing organizations out there who harp so much on homogenization and conformity. You can actually be brain-washed into believing that everyone wants to read Yet Another Novel about a cynical P.I., a secret baby, or a vampire brotherhood. Selling that to an agent, on the other hand, is a little tougher.
Writing is not about gaining the approval of one's peers, satisfying the Thought Police, or in any other way placating the masses. It's not something you do to fit in or make yourself look good. Writing is about setting the reader's imagination on fire; how are you going to do that by serving up something that barely registers as lukewarm in an already glutted market?
Nothing compelling or enticing about the novel
Your ability to competently put together one hundred thousand words that are spelled correctly, obey the rules of grammar and make sense does not entitle you to a publishing contract. There has to be something in all those pages that is going to provide serious competition for TV, movies, the internet, video games, sex, food and now texting. If there's not, it likely won't sell.
Nothing fresh or new with usual elements of a novel
Some people say nothing is fresh, nothing is new. And some of the time when I'm reading yet another blatant knockoff, I think they're right. But the one factor in the writing/publishing equation that is always fresh and new is you. There is only one of you. So what do you bring (or can you bring) to the novel table that no one else has?
No one is exempt from this, not even published writers, and I think it has several causes, like the writer's state of mind, poor editing, poor planning, stagnation, phoning it in, rushing, or just not being interested enough to present your best work. And nothing shrieks "I don't care" like bad writing.
I feel for people who have no formal writing education to speak of; I am one of you, but that didn't stop me from employing my public library card and improving my understanding of fiction, writing, and the ten thousand other things I had to learn to become a pro. And I'm sorry, but with all the free resources at hand on the internet to help you improve, polish and otherwise bring your writing up to professional level, there's simply no excuse for bad writing.
Don't want to read this
Not every agent is going to be right for you. Sometimes it's not personal or even a reflection on your work; it's a matter of an agent or editor who is a bad match for you. Remember that these people have to sell us, and if they don't feel passionate about our work, they aren't going to get us a dime. So in a sense this kind of rejection is good because it avoids a partnership that isn't going to work.
No idea what the novel is about
Writing is a journey. Whether you choose to use a map to take yours, or simply wander down this road or that to see what you find, by the time the journey ends you should be able to tell someone about it. If you don't understand, you don't remember it or you're not sure why you took the journey in the first place, this is not a time to try to sell it. Ditto for those of you who find it impossible to condense an entire journey into a couple of concise sentences. Rambling on and on in a disconnected fashion isn't going to interest anyone in taking the same journey. You have to relate what makes it worth the trip.
Querying is not a skill most of us are born with; it takes thought, practice, decision-making, concept spinning, presentation, showmanship and a lot more thought and practice. Every novel is different, so every query is new territory. The biggest favor you can do yourself is not to send out the first draft of a query you wrote in a few minutes. Set it aside. Think about it. Querying is not typing a business letter. As Janet wrote, it is HARD. And for my two cents, I think it's an art all on its own.
Look at examples of different query letters that are posted online. Read the thoughts about queries from other writers who have been able to sell a lot of books (and before you ask, mine are here and here, as well as these posts: Novel IV: Pitch, Ten Things About Novel Proposals, Queries and Synopses, and Query Nation.)
Another thing you can do is role-play with another writer or someone suitable in your family or circle of friends (and pick someone who won't kiss your backside because they love you) and have them play agent and read it. Get their reactions and think about what they tell you.
Look for rough spots in the query -- look for any place you are not writing like a professional -- and edit. Revise and rewrite. Keep at it until you have achieved the absolute best version of your query. Also, if you get any specific editorial feedback from an agent or editor who's read your query, think about that, too. They don't often have time to give us meaningful feedback, but occasionally you'll get a comment that does flag a real problem that you need to address.
In a sense, a query is like an agent all on its own: it serves as your pitchman, your sales force, your PR rep. You'd never send some smelly, lazy, loud-mouthed slob in a wrinkled suit to sell your book; don't make the mistake of mailing off the query version of that.