You need to find the maximum in so what you need to calculate is the value of x for which

So solving for you get so the only critical point occurs for this value. Now determine if it is a maximum or minimum you need to look at the next derivative, i.e. you will find the sign is negative and therefore x=10 is a global maximum.

The rate of peak efficiency is just

One may assume that at time the subject has not memorised anything so

This can be used to eliminate the constant. ( ).

The only thing you have done wrong is to have the wrong sign for the marginal cost. As this is a cost term it should be negative. So start with

and you should get the correct answer.

Hope this helps.