That's a good question, and I did not find information about that. However I can speculate:
Skidoo as a town likely became irrelevant before the mill did. As the scale of mining decreased, the mill was still used but a town couldn't be supported. As the town shrank, buildings and materials were likely scavenged for a variety of uses - buildings elsewhere, fuel, etc.
I also imagine that the owners of the mill protected from scavenging after it closed as the materials there were of some value.
Finally, since a fair amount of Depression-era mining occurred in Death Valley, there was likely a lot of ongoing scavenging for materials.
Why the mill was never dismantled I don't know. They usually are as the materials are valuable. This is why I was so excited to see the stamps intact, you don't see that often. The government would even seize these things for WW2 iron.