Yes, and that's exactly what happens here! Of course, by "row times column" you mean the "dot product" where you are treating row and column as vectors. 1st row times first column is the number in the "first row first column" of the product matrix. Continue like that. "Multiply first row by second column" gives the number in "first row second column", "first row by third column" gives the number in "first row third column" and "first row by fourth column" gives the number in "first row fourth column".

That gives you the four numbers in the first row. Now do the same, multiplying the second row by each of the four columns to get the four numbers in the second row of the product matrix, and multiply the third row by each of the four columns to get the four numbers in the third, last row of the product matrix.

The product matrix will have, of course, 3 rows and 4 columns.