Public Projects enable you to allow anyone to see your source code or your work items without having to login. Test Plans and Artefacts aren’t visible at the moment to the anonymous users who aren’t login. This may change as this feature is in preview at the moment.
One of the key benefits in using a Public Project over a Private one is you can have 10 build agents hosted by Microsoft. These can run parallel Build or Releases which, which can last up to 360 minutes (6 hours) per one. There is no time limit on how many minutes you use in the month. Compared to a Free offering of private projects you only get one parallel agent that can run up to 60 minutes for a total of 1,800 minutes (30 hours) a month.
So if you are working on a Open Source Project or trying to learn how to configure Parallel builds Public is a good choice for the projects.
How to disable for my Organisation
The last thing you would want is a Organisation which needs to be locked down to have a public project created. So to stop that from happening head over to the Organisation Security Policies.
Switch the Allow public projects to be Off.