The finite case is an application of Sylow's theorem. Look up the part of the theorem which tells you how many Sylow p-subgroups there are and work out which bit is of interest here. Then think about why there can't just be 2 Sylow p-subgroups.

For the infinite case you need to remember that, well, there is no infinite case! Essentially, you were right with what you said; you are working with Sylow subgroups, which are only applicable to finite groups!