What type/brand of cat food are you feeding?
Does he finish his entire bowl of kibble everyday? If he does, then yes, you are feeding too little. If there's always some left over, then it should be fine, but it does depend on what food you are feeding him.
Some hedgies just develope a taste for their litters, and you have to take them off it, immediately, before it gets to the point where they can get an impaction. Impactions are always scary...