Musk/Trump are not saving huge amounts of money - there's a lot of analysis around showing that they are saving rather small amounts of money. Rather they are, above all, trying to take money away from people they really don't like, even if it is relatively small beer in the overall business of money-saving.
As the Economist pointed out, if they are to save anything like the sums of money they are talking about, £500bn to £2,500bn, they will need to start cutting not just staff and small stuff, but really big stuff. Staff costs in government employees are only about 4% of federal spending. One of the biggest things that Trump hasn't said is protected is Medicaid. That's the federally part-funded medical assistance for the poor and infirm, as opposed to Medicare for the old, which Trump says is protected. Since Medicaid is part state-funded, what cuts occur on the ground depend on the attitude of the individual states to the federal funding cuts. The Economist considers a maximally feasible cut to Medicaid to be not quite £100bn/yr, which is still a long way to go to cuts of the sizes mentioned. And, depending on the attitude of the states, could potentially remove insurance from up to 1/3 of Medicaid customers, though in practice will probably not be quite that much.
A lot of spending is congressionally directed, and those directions are passed as US law. So those payments are legally difficult to stop. One Trump appointee was recently spotted in public trying to justify stopping congress-directed FEMA payments to New York State on the innovative grounds that "Congress shouldn't have done that". Though when he ended up in court over that attempted cut, he had the sense not to mention that, but instead alleged, without any good source, that some of the money was leaking to organised crime. He failed to impress the judge with this. And doubtless the incident related to federal funds going to New York State, precisely because it was New York State. (Source:
youtube video by Legal Eagle, 23mins)