I believe that whether or not you write a form of speculative fiction, you will have to engage in some degree of world building to make your story believable. You might have to create a fictional town. Or disguise a real one (Gotham City, anyone?). Otherwise, you might simply bend the real world rules a little in to make them fit the purposes of the story.
Because of this, I'm going to address two types of world building. Spec fic and non spec fic.
BUT remember, you're looking for a feel for the place/time so that you can write a piece of fiction. You're not writing a text book. So if you've written blocks and blocks of information with minute details of everything, you might have to cut back. It's sort of similar to what I said about using senses. Characters aren't going to list the histories/descriptions/cultural impact of every single thing the see and experience. Rather, we the readers want to feel everything through the character. Show the impact of certain things. Show what they mean. Don't list them and go on and on about minute details.
Special bonus for historical fiction writers: Anachronisms are incredibly annoying, so make damned sure that the things used/referred to by characters existed/happen in the time of your story. NOTHING annoys me more than reading a western where badass gunslingers use the colt peacemaker three years before it existed. And yes. I know when it did or didn't exist. Other people will too. Keep your dates straight. If you absolutely must bend the dates to suit the story, please remember to make note of it in a foreword or something. That way, you show that you're not an idiot, and (possibly more importantly) that you don't think the reader's an idiot.
Spec fic, on the other hand, sets world building on a whole new level. More often than not, the world of your fantasy/steampunk/sci fi/urban fantasy/dystopian/horror/etc. etc. story will be foreign to your readers. And if your readers can't place themselves in the world of your story, you already lost the battle.
When it comes to my spec fic stories, I try to know more than what goes into the book. Note: MORE. Not everything. Every single thing doesn't have to go in. Important things go in. And not always in a clearly outlined way. Let's say that amongst other things, your world randomly loses gravity. I wouldn't suggest that you necessarily go into the depths of why, unless it's important. The same for the cultures that you create. Remember, most spec fic characters already live in the world that you've created. So they won't be explaining things to themselves or others. At least not all the time. There's a fine balance between enlightening the readers and boring them with too much detail. Make sure that you stay on that line.
Taking the world rules a little further.... Natural laws should exist as natural LAWS. Same with the rules of your magic system. Or your cultural norms, rules and regulations. DO NOT UNDER ANY CIRCUMSTANCES let your characters break any of the above without them being aware of the fact, without an explanation, and without potentially huge (and hugely negative) repercussions. Especially, don't let them do it to save the day. If you do, you're undermining the credibility of your own story. These rules should be the frame that keeps everything in your story structured and believable. You can't ignore them for convenience sake. It will make your story collapse like a house of cards. If the world rules create a problem for the story, you have two options: either rewrite the rules (and revise the whole story to fit them) or go look for a solution that fits and even comes out of the rules. See my P post for more info on that.
On a lighter note, having a fantasy world helps to set the mood of the story, if you use your world right. You have the joy of creating something special and unique. It's one of the few forms of pure creation. So have a blast!
Look Out for These:
1) In both: Over-telling on the world/time, boring the reader and making everything seem unrealistic. Under-informing the reader, making them wonder how things work.
2) Non-Spec fic: Anachronisms, not knowing enough to get the feel of the time/place right.
3) Spec fic: World rules that are broken.
What do you love/hate about world building in your genre?