What structures could not be

James Ladyman has recently proposed a view according to which all that exists on the level of microphysics are structures “all the way down”. By means of a comparative reading of structuralism in philosophy of mathematics as proposed by Stewart Shapiro, I shall present what I believe structures could not be. I shall argue that, if Ladyman is indeed proposing something as strong as suggested here, then he is committed to solving problems that proponents of structuralism in philosophy of mathematics such as Shapiro are trying to solve. Attempting to do so, however, brings out a tacit tension in Ladyman's position. I shall argue that the upshot of this is that the ontological import that Ladyman attributes to structures is rather epistemological import properly understood.