In the forum permissions in the Admincp, change the permissions to no view for usergroups you don't want accessing it.
Go to your admin CP, then forums and posts.
Choose the forum, then click 'edit' in the dropdown menu. After that, scroll down, and select which usergroups you would like to view the forum. Just drag the 'view' to either allow or disallow.