Here's a riddle. Suppose you have formed a new political party and you want to assess the voter sentiment with respect to electing your party. Your party has to get at least 2% of total votes to enter. How many randomly selected respondents would you need to sample to draw the correct conclusion with 95% confidence?
The answer depends on the true proportion in the population, your required significance level and the statistical properties of the binomial distribution.
The figure above shows the minimum respondents you would need to sample, as a function of the true population proportion. If the true proportion is between, say, 3.5% and 4.0%, you would need to sample around 1000 respondents, before a statistician using a one-sided 95% confidence band would announce your party a viable proposition.
You can go about this in a dynamical fashion. Sample 500 respondents. If the mean proportion is below ~4.7%, then sample another 500. If the new calculated mean stays above ~3.7% you are fine.