I asked this before on ARM, but I think I might have phrased it wrong.
The Sibyl System is made of the brains of those whom it can't judge correctly. This kinda reminds me of the problem of "infinite regress", e.g. "if a god created the universe, who created that god?", as there would have needed to be a first brain when the system was first implemented. At the moment, the best explanation I have for that was that the brains of random psychopaths were collected, forming the original Sibyl System.