First, let's take a look at the most popular definition of veganism:
“Veganism is a philosophy and way of living which seeks to exclude—as far as is possible and practicable—all forms of exploitation of, and cruelty to, animals for food, clothing or any other purpose.”
Nobody can be or is expected to be a perfect vegan. That is unless we grew our own food, were capable of avoiding any unintentional killing of insects, and could account for every way in which we could cause animal suffering. But does this mean we shouldn’t be 99% vegan? In other words, does this mean we shouldn't try our best to avoid harm to animals? Absolutely not.
In reality, the meat, dairy, egg, wool, fur, and leather industries alone account for most of all animal abuse. In our modern world, it is impossible to exist without coming into contact with some kind of animal-derived ingredient or product. So the fastest and most practical way to end animal abuse is by refusing to support the main creators and producers of animal abuse.
The primary reason we find animal by-products in such a wide range of every day products is due to the scale of industries based on exploiting animals. These industries produce so much waste (ligaments, bones, brains, intestines, etc), that it makes economic sense to use it elsewhere. Decreasing the production of animal products, by avoiding the main industries, would make the use of these waste products impractical and unprofitable.
Nobody is expected to be perfect, but ultimately veganism is about doing the best you can and causing as little harm towards animals as possible.