I would guess it is either the ECU or the button if you have already changed the solenoid.
I say this because if it actually starts and runs when you jump the solenoid, the key has authenticated and the reed on the post has contact. Once this is satisfied, the start button power circuit is live for the button to be pressed. That button just puts 12V to the Solenoid, which then closes the starter circuit.
If the button works to stop the machine, I would have to think the start button works, but you could pull it out and check for 12V on the power side of it. If there is none with the Key on and the regular beeps, the ECU is not enabling the switch.
If you jump the solenoid without the key, the ECU does not light up power to the injectors so the machine will turn over but not start.