A causal feedback map, taking sequences of measurements and producing sequences of controls, is denoted as finite-set if its range, for any arbitrary finite time interval, is a finite set. Bit-rate constrained or digital control are particular cases of finite-set feedback, where the range of the feedback map has an upper-bound on the rate of exponential growth. In this paper, we show that the finite gain (FG) l_p stabilization of a discrete-time, linear and time-invariant unstable plant is impossible by finite-set feedback. In addition, we show that, under finite-set feedback, weaker (local) versions of FG l_p stability are also impossible. These facts are not obvious, since recent results have shown that input to state stabilization (ISS) is viable by bit-rate constrained control. In view of such existing work, this paper has two conclusions: (1) in spite of ISS stability being attainable under finite-set feedback, small changes in the amplitude of the external excitation may cause, in relative terms, a large increase in the amplitude of the state (2) FG l_p stabilization requires logarithmic precision around zero. Since our conclusions hold with no assumption on the feedback structure, they cannot be derived from existing results. We adopt an information theoretic viewpoint, which also brings new insights into the problem of stabilization.