Tooth brush is just as (if not more) important indoors, you use it to clean chalk off of holds. As chalk builds up it reduces the all important friction that is required to hang onto holds. You should get a brush for indoors and get in the habit of cleaning holds in between trying problems (I use it as a way of stopping myself from jumping straight back onto a problem I'm working on so I have a proper rest). It means they stay cleaner for longer for all (unfortunately a lot of people seem to think holds clean themselves and just complain about how chalky places are as opposed to being pro-active and doing their bit).
Putting it on a stick means you can reach holds you can't from the ground.
A chalk bucket is just a large chalk bag with a flat bottom that means it won't roll away.
Guidebooks are useful as they get you to the areas/problems you might want to do.
Bouldering mats are useful as they protect the ground from erosion (which occurs when lots of people fall/stand in the same area). They also help prevent you from hurting yourself, be that your feet/ankles (believe me you don't want to fracture your calcanium, I speak from experience), back or any other part of your body when you inadvertently part company with the rock you're climbing.
Shoe cleaning rags mean your shoes are clean before you step onto the rock, this in turn means you'll get the maximum amount of friction between your shoe and the rock, it also means you won't leave dirt on holds and other people will be able to climb them.
Beanies keep your head warm when its cold.
Shoe wise, don't be taken by looks, try lots on and go with what feels snug (but not massively uncomfortable) and good to stand on (decent shops should have little kickboard sections for you to at least try edging on, smearing is more difficult to judge on indoor holds). Search the board, there are quite a few threads on shoe advice, they all say the same thing though (i.e. go with whats comfy). As you're just starting out you won't need anything massively technical (or appreciate the advantage that they can give).