Because their founder (Marlinspike) is probably under a National Security Letter, maybe it's just that, maybe he's done some crimes they're also holding over him. If you look at his behavior it's that of someone very paranoid that they're going to be found out to be cooperating with the feds and get hit with charges for not upholding the bargain, someone straddling one or two big lies that have to be maintained to keep their life going. Very controlling of things they should be open about if they care about privacy as they claim. But exactly the behavior of someone under an NSL who's terrified of getting hit with charges for that and maybe other things but who is expected to front and run a purported privacy first messenger. The secrecy, the refusal to allow others to operate their own servers, the antagonism towards federation, the long periods without publishing source code updates.
This doesn't necessarily mean that signal message content is compromised, the NSA primarily scrapes metadata and would most care about knowing who is talking to who and to put real names to those people and building graphs of networks of people. Other things like what times they talk can be inferred from upstream taps on signals servers without their knowledge or cooperation via traffic observation and correlation especially when paired with the fourteen eyes global intercept network. With a phone number it's also a lot easier to pinpoint an exact device to hack using a cooperating (or hacked) telecom. Phone numbers can also be correlated to triangulated positions of devices, see who in a leftist protest network was A) heavily sending messages and B) attended that protest and left last and begin to infer things about structure and particular relationships.
And those saying it has to do with spam prevention, that's kind of nonsense. First I still get the occasional spam, second a phone number that can receive a confirmation text is something all these criminal organizations have access to which the average person doesn't. Third it's possible to prevent spam just by looking for people (especially new accounts under 120 days old) sending very small amounts of messages (1-3) to a very large amount of other users especially in a short amount of time. Third there's no reason to keep the phone number tied to the account, a confirmation text could be required with a promise to delete the phone number immediately after (would still be technically useful to the NSA though less useful for keeping track of people changing numbers or using a burner for this who might be higher value targets).