Wait on someone else for "official" word on this, but as I understand it:
Penalties can never cause a card to penalize you (2f follower gets -5f, it is 0f). However, if you stack a bunch of penalties and bonuses, they all count in finding the final number. 2f follower, give it -5f. Then +4f... it's now 1f. So the negative values are there, but they only matter for "possible future bonuses."
In addition, for "paying costs" - you can't go negative to pay a cost - you need to have it before you can pay a cost with it. If you try, I expect the action will fail (though I'm iffy on the whole "announce, pay costs, resolve" business - I've never had an issue where that mattered much).