Keyframe a custom property to an action

You can keyframe visibility of a bone constraint to a specific action. In my case I have a custom property that toggles the the visibility of a number of constraints, and I would like to keyframe that property. However when keyframing custom properties the key does not get added to the action itself but to the timeline.

Is there a way to key custom properties to specific actions?