We can't use Fermat's little theorem directly, but we can calculate the residues mod 5 and mod 7. By Fermat's little theorem,

.

Also, the powers of 2 form the sequence 1,2,4,1,2,4,1,2,4... (mod 7). 20 is congruent to 2 (mod 3) so

Out of the numbers 1,2,3,...,35, we can easily show that 11 is the only number that is both 1 mod 5 and 4 mod 7. Therefore the answer is 11.