Calculated Field (?) in Sub Total?

Hey all -

I have a pivot table that contains calculated fields where it will determine a score based on values in C - PV, PV 1, PV 2, PV 3. The calculations I have so far seem to be working as I'd like. However, what I'd like to be able to do under each calculated field for each agent is calculate the average score for each agent under each score field based on the numbers above it. I set the field setting for the agent to Average and it only averages 1 part of the table where I need it to do both pieces. I can't figure that one out.

I've attached a sample workbook with data and the pivot i'm using and have highlighted the calculated fields I added in yellow, and where I'd like to see the average score under each field in green. Any help or insight would be much appreciated.