Hello!
Yes, for TLS you only need one cert per agent, and the CA pub cert used to signed them.
Regarding manager, you could use only the common CA and not different individual cert for the agents, but they will show in the UI as untrusted.
Please take into account manager is able to create its own CA, and sign all needed certificates.
Do you want to elaborate more on your case?