Garments mean many things to Mormons. One of the implicit symbols in garments is the rite of passage into adulthood. Mormons generally wear garments for the first time when they are about to go on a mission, or about to be married. Both mark significant moments in young Mormon's lives, and show with a physical object that they are now "one of us" and a full adult.
This rite of passage should be a marker of a personal and maturing relationship between the person and God, and some faithful Mormons would define it as such. This is comparable to many Muslim women's experience who, at some point in their youth or young adulthood, decide to don the veil to demonstrate their desire to follow God's will. Ideally, she is free to adopt the veil or not, and in many places, she is. But what if she doesn't veil? In some ways, she is, in effect, remaining in childhood. In some places, only little girls don't veil; women do. In a symbolic sense, she is refusing a rite of passage that would mark her as not only religious, but also as a mature adult. Hence, non-veiling Muslim women, Christians or Western tourists in a highly Muslim area may be implicitly considered non-adults, regardless of their age.
I suggest the same may be true with garments. Unlike the veil, the garments are meant to be hidden and private, and, like the veil, the spiritual act of wearing them is supposed to be intensely personal. However, there are plenty of opportunities when the fact of wearing garments becomes public, and therefore, a point on which to judge, among other things, Mormon's adulthood status.
While not as publicly visible as a veil, the garments are something of a public phenomenon among insiders, especially in Utah. Failing the garment-line test might simply mark someone as young. For example, young male BYU students who didn't demonstrate neckline “smiles” or knee-lines were dubbed pre-missionaries, or pre-mies for short. It was not lost on us that the nickname also referred to pre-mature infants. The pre-mies were either fair game for dating (for women who didn't want to find a marriage partner yet), or off-limits (for women who wanted to date only men who were of marriageable age and status). Pre-mies were not quite men.
Within the Mormon paradigm, failing the garment-line test, especially among adults out of the college years, also means something more than youthfulness. The only reason a Mormon 30-year old wouldn’t wear garments is lack of faithfulness to the church. Either she didn’t get her act together enough to go to the temple (a demonstration of immaturity), or, worse, she took her garments off (gasp!).
In Mormon-dom, children and youth don’t wear garments; adults do. Interpreted from a faithful Mormon viewpoint, taking off the garments can be considered a return to youth and immaturity. I am not arguing that it is the only interpretation. Indeed, the other connotations are undoubtedly stronger: unrighteousness, faithlessness, apostasy, covenant breaking, etc. I do suggest that because the act of donning the garments is a rite of passage into adulthood, the equally symbolic and significant act of removing the garments is rejecting that passage into adulthood.
This does not mean that people who remove their garments feel that they themselves are returning to youth, but that faithful Mormons may (subconsciously?) view them as doing so. For their part, ex-Mormons might feel they are rejecting the particular Mormon version of the rite of passage into adulthood. For them (for us), the act of removing garments marks our own version of a rite of passage into a new adulthood. One in which we can demand a more personal claim on our personal lives. We are, after all, talking about underwear.