What can I do to keep my baby's hands warm at night when he's sleeping?

My 5 month old baby sleeps through the night beautifully and we recently transitioned from him being swaddled at night to letting his hands and arms be free. While he sleeps in a sleep sack and fleece pj's, his hands (and arms sometimes) are ice cold when we all wake up. We have tried to turn the temperature up in the house, but that hasn't seemed to help. When he was swaddled we didn't have this problem because his hands and arms were tucked in close to his body. I don't really want to use gloves because they seem to annoy him because he likes to suck on his fingers to soothe himself. Any suggestions?