So there are two cases,
1. Both the man and wife are present.
In that case, two members are already fixed, so you need to select the remaining two, one man from 4 and one woman from 5 , so the number of combinations =20
2. Neither are on the committee.
Here you need to select two men out of 4 and two women out of 5, so no. of possibilities = 6(10) = 60
So total number of ways = 60+20 = 80