WLOG, we can suppose that i>j. Expand everything in
Then yes you have to take the double expectation for each term.
Just note that according to the martingale property for any , it would be useful to condition by the X with the lowest index.
For example, .
For , note that with the martingale property,
So that .