Exercise Show that for a 3 by 3 matrix, all rows are pivot ⇔ all columns are pivot? What about general n by n matrices?
Answer There are three rows in a 3 by 3 matrix. If all rows are pivot, then there are three pivots. Since these three pivots are in different columns, and total number of columns is three, we see that all these columns must be pivot columns. The converse is also true for the similar reason (simply exchange the words "row" and "column" in the argument).
The same proof also works in general. Therefore for any n by n matrix, we also have
all rows are pivot ⇔ all columns are pivot