While PE is a mandatory subject in schools, the recommended amount of two hours a week is not enforced.
YST chief executive Ali Oliver said: "Our children are moving less, feeling unhappier, and losing access to the transformative power of PE, contributing to stagnant physical activity levels.
"The fall in PE hours is sadly an exacerbation of a longer-term trend and should be a wake-up call to society, from policymakers to schools and parents.
"Unless we take action to reverse these damaging trends and increase activity levels to improve wellbeing, we risk failing a generation."
In a statement, the government said: "These figures highlight the government's dire inheritance, but we're determined to break down barriers to accessing PE and school sports for young people through our Plan for Change, helping to improve their mental and physical wellbeing.
"We are working across the government and with our partners including Youth Sport Trust and Sport England to boost participation and have already invested £100m to upgrade sports facilities and launched a programme to improve access to sports for pupils with special education needs and disabilities.
"Our ongoing curriculum and assessment review seeks to deliver a broader curriculum, so that children do not miss out on subjects including PE and sport."