I'm lost in this question.
Find the smallest angle of triangle ABC in which a:b:c = √2 : √3+1 :2
The smallest angle will be opposite the smallest side, which side do you think is the longest?
After you know this apply the cosine rule to find the angle.