Yeah the archive feature can seem annoying in your situation, but is really helpful in others (if for example someone by mistake deletes an asset, via archives you can retrieve it back, like what you did).
For versioned assets in a package, to actually delete an asset, you would archive it first, then go to the Administration tab -> Archives, select the archived assets and click on the Delete button. Then they are gone for good.