Unfortunately yes I've done this, you don't even have to retire them, simply setting them as offline and saving will be enough to screw you over.
Until you take your ps3 back online, they will remain available for others to use, and you'll even be able to see their credits increase through the website. You'll never be able to cash them in though.