Im solving:

y''-2y'+y= (2e^x)(cosx)

The complimentary function after solving the auxiliary equation is:

(A+Bx)e^x

How should I start finding out the particular integral?

by substituting

y= k(e^x)(cosx) ( where only k is a constant)

or y= k(e^p.x)(cos q.x) ( where k, p and q are constants)