There are only 2 subjects from which children may be withdrawn on religious grounds, according to the Department for Education (DfE):

• Religious education (RE)
• Some aspects of sex and relationship education (SRE)

https://www.gov.uk/national-curriculum/key-stage-1-and-2

This applies to maintained schools and academies.

Pupils may raise spontaneous questions on religious matters in subjects other than RE, and that issues related to religion come up in other subjects such as history or citizenship.

However, the right of withdrawal does not extend to these areas of the curriculum nor P.E/swimming.