Using the X button does not delete the data block (IPO, Action, Material, whatever), it just unlinks it from the selected data block (Object, Armature, whatever). The “X’d” data block will persist in the file as long as it has any users, including Fake Users. To fully delete a data block of this type, you need to make sure it has no users (usually indicated by a small circle next to the data block name in the drop-down listing), then save the file, close it, and reopen it. AFAIK only data blocks with no users are deleted, and only when the file is closed.
Unlinking a mess of IPOs one at a time is more work than needed, I think. For any selected Armature, an Action is a data block made up of a number of IPOs (also data blocks) that apply to various bones in the Armature, plus (if any) those for the Armature as an Object. In the Action Editor, these are represented as keys rather than curves, but those keys correspond to the IPO keys. The Action Editor just provides a different level of editing capability for the IPO data blocks. So to unlink a set of IPOs from an Armature all at once, you should only have to unlink the Action associated with them. That’s how it’s worked in my experience.
Keep in mind that if an Action has a “presence” in the NLA Editor (as a strip, for example), it cannot be deleted because the NLA Editor is a user of the Action. Plus, if you select that strip in the NLA, it will automatically re-link that strip’s Action with the Armature in the Action Editor, even it it has been previously unlinked. That’s a bit of a “gotcha” but it makes sense if you think about the data block hierarchy involved – in order for the higher level data block (the NLA strip) to be valid all the lower-level connections it represents (such as Action to Armature) also have to be valid, so it re-establishes them. At least that’s how I think of the situation.
As a result, if I’m trying to completely delete any IPO, I make sure all NLA references are removed first, then unlink the Action from the Armature, and make sure it has no Fake User status, and make sure the little circle mark shows in the Action Editor choices listing. If the last check fails, I look for other users much as you described, by using the Outliner.
IPOs are a little different in that they can be incorporated into one or many Actions, so the list of possible users for each IPO is a lot larger.