AVERAGEIFS with condition NOT to include cells with letter

I am trying to Average columns in a spreadsheet (small portion attached, I HOPE) where I want to indicate that there is no information available by using an "X" character. I don't want to leave these cells blank, because then I keep looking at it thinking I need to find some information. I think I need to use the AVERAGEIFS formula, but can't figure out the syntax needed. If I use
=AVERAGEIFS(N$6:N$32,$C$6:$C$32,"=2",N$6:N$32,"<>")
I get #DIV/0 in cell N37, probably because cells N25, 26 and 27 have an X in them.

If I use
=AVERAGEIFS(N$6:N$32,$C$6:$C$32,"=2",N$6:N$32,"<>*x*")
I also get #DIV/0 in cell N37.

I want cell N37 to average any numerical values in N6:N32 ONLY if C6:C32 equals 2. The equation works for the ages that don't have any Xs in the required range N6:N32, but I'd really like to get rid of the #DIV/0 problem.

I apologize if I have not made myself clear.

The AVERAGEIFS function would not include cells with letters in, as they would be text values with no numerical value, so you don't really need the second criteria.

Hope this helps.

Pete

Pete