What should I try for the particular integral inorder to solve these problems where D means d/dx.
$\displaystyle 1.(D^2-4D+4)y=x^3e^{2x}$
$\displaystyle 2.\frac{d^2y}{dx^2}+4y=sin^2x$
Are you familiar with the so-called "Annihilator" approach?? To see how you do it this way, look at post #7 in my differential equations tutorial [also look at post #6 and #8 for two other techniques].
Going along with this, we solve the homogeneous solution first:
You should end up with $\displaystyle y_c=c_1e^{2x}+c_2xe^{2x}$
Now for the particular solution, solve the non homogeneous equation.
This will lead us to finding the annihilator of $\displaystyle x^3e^{2x}$, which is $\displaystyle \left(D-2\right)^4$
The DE becomes $\displaystyle \left(D-2\right)^2\left(D-2\right)^4=0\implies (r-2)^6=0\implies r=2$ with multiplicity six. Thus, the particular solution with only consist of r repeating 4 times, since 2 were used in the homogeneous solution.
Thus, $\displaystyle y_p=Ax^2e^{2x}+Bx^3e^{2x}+Cx^4e^{2x}+Dx^5e^{2x}$
Now substitute $\displaystyle y_p$ into the original DE and find the coefficients...
If you still do the annihilator approach, $\displaystyle \frac{d^2y}{dx^2}+4y=\sin^2x\implies \left(D^2+4\right)y=\tfrac{1}{2}-\tfrac{1}{2}\cos (2x)$$\displaystyle 2.\frac{d^2y}{dx^2}+4y=sin^2x$
Try to do something similar to what I have done above.
I hope this helps.
--Chris