The force at D is in the opposite direction, so you should have:
M1 = r x F = (-3i + 2j) x (-400cosx i - 400sinx j) = (1200sinx + 800cosx) k
and so the total moment is
Meq = (1200sinx + 800cosx - 400cosx) k = 400(3sinx + cosx) k
(and you'll write instead of x in the solution, of course)
Part (d) should be easy - just substitute for x.
For (b), you can take the derivative and set it to zero. (I don't see how to do it without calculus - if you haven't had calculus, post again)
For (c), notice that if x is between 180 and 270 degrees, the forces will both be in the opposite direction, so the moment will have the opposite sign. So it must have passed through zero somewhere - just set Meq to zero and solve for x.
Post again if you're still having trouble.