Do you ever delete tests?

Yes. There are basically two fundamental reasons for deleting tests:

  1. The test is syntactically poor: it doesn’t compile, it’s commented out, it has been refactored into nonsense, or it has been used for training by a developer who just started learning about unit testing.
  2. The test is redundant or outgrown. Sometimes tests are written from one angle, but then something changes and suddenly there’s a better approach, method, scope, or API boundary which makes the old test look poor and duplicated.
Book References
Read more about this topic in Developer Testing: Building Quality into Software:

  • Chapter 17: Working with Test Code, pages 241-243