Okay, so I've got the bike running beatifully after some troubles with idle, clutch, etc. 

Now, as I'm riding to work (daily this week) I'm getting small backfires when I come off the throttle to down shift in traffic, etc.  Nothing big, just some little noticable backfires.  Carb has been cleaned and rebuilt, rubber looks good, jets look good.  No vaccum leaks=no high idle.  Oil just changed.  starts on first crank.  runs strong.

What should I be looking for to fix this?

rich jetting

More info needed... stock or after market exhaust? stock or after market airfilter? stock or rejetted?

All bikes will backfire on decel as unburnt fuel ignites in the exhaust. Free flowing exhausts & air filters with larger jets will increase the back fires.

What Dgyver said + exhaust leaks around the head/header area will cause it and any leaks where the exhaust tubing is fitted together.

thank you for the notes.  all stock.  stock jets, stock exhaust, air, etc.  bo leaks, checked on that.  Didn't do it as much tonight.  I took it out on the highway and ran it strong and hard for a while.  Man do I need better front springs.

I'm going to rejet over New Years.  Should I change or stick with stock.  I'm not a crazy insane hair on fire rider, i do like strong accel, but I want to maintain some gas milleage.  Recommendations?

Does the sound come from the exhaust or air breather?

Mine give a little popping sound from the breather, but not very often.
Usually only on cool damp mornings.

Not really sure what it means though.

I'd suspect a little richness in the jetting and/or a leaky exhaust. Those headers do loosen up from time to time. Also the plug wires can just plain be a pita.

Mine backfires on monday, wednesday and saturday mornings. IF it's moist out. Otherwise, only on downshifts, and only just to make me wonder.

If it only does it occasionally, suspect gremlins. If it's consistant, suspect exhaust leaks and /or jetting.


IMHO

If it's on deceleration I believe it's actually a LEAN condition rather than a RICH one. 

http://www.mikuni.com/tg_backfires_in_exhaust.html

I had a backfiring / afterfiring (depending on who you talk to) on my VFR with a aftermarket Fuel Injection upgrade, it had to do with the PAIR valve.  It would run RICH but on deceleration those RICH fumes weren't combusted so the PAIR valve would open up and add more air to LEAN out the mixture to help it ignite for emissions purposes.  Personally I didn't like the backfiring sound so I removed the PAIR valve (no more added air to the exhaust) and voila no more backfiring!  If it's running too RICH those fumes most likely won't ignite in the exhaust (sort of like if you had a match and put it on a table then covered it with a glass jar you would see it suffer and the go out), I know it seems backwards but that's been my best understanding of it.  If you also want to tune your bike yourself, a great way to do it is with a Colortune http://www.gadgetjq.com/ctune.htm 

It definetly makes adjusting your air mixture screws easier and you can see how the engine is firing when you turn them.  Good luck!