In the general channel for a classroom team there is a folder called, "Class Materials". This folder is setup automatically for all classroom Teams. It is great because you can put files there that students can download and view, but not edit.
HOWEVER, it turns out that you can make any folder (or file) in any channel of any team read only for students.
Navigate to the container of the folder you want to change access to.
Click on "view in sharepoint"
Click the triple dots by the item you want to change
Select "manage access"
Click the down caret, "˅" to the right of '[teamname] Members"