Yarmulkes, Mitras, Veils, Turbans, Hijabs, Dastars, Chadors, Habit/Nuns headscarf, Sheitels, Biretts, Tagelmusts, Shtreimels, Amish hats and bonnets, to name a few religious head coverings.
With the Religious Freedom Restoration Act, the Waterbury School would have to demonstrate a narrow and compelling reason for why your child cannot wear a Yamaka or a Hijab (or other) in school. It is highly unlikely any Waterbury school will be able to demonstrate such a reason. As such, the prohibition will be unlawful under Connecticut's General Statute section 52-571b.
Advise your school that your child wears the Yamaka or Hijab or other headdress as required by your faith. Request the accommodation (or exception to the school uniform rule), in writing. An email is fine.
Your school, (the District), would be very foolish to deny it. Besides having the visual appearance in the media of discriminating against a specific religious group, and rejecting diversity in a diverse city, a challenge to their denial of your religious freedoms, will likely be granted in your favor.