the complete group consists of x persons. Thus the expenses for the whole group is E(group)=(200+400x)x=200x + 400x^2
Do the same for the costs:
C(group)= (900 - 100x)x = 900x - 100x^2
The profit is the difference between expenses and costs:
P(group) = C(group) - E(group) = 900x - 100x^2 - (200x + 400x^2)= 700x - 500x^2
As for the mathematical writing look at the Latex-tutorial.