Can't sort Column properly

I have inputted a countif formula that is returning a numeric value in the column of cells.
When I filter this list and sort, I don't know what's happening.
The sorting from smallest to largest is:
2
3
94
1
1
3
3
2
4
etc.

I have selected this column and formatted it to be a number without zeros, but still no difference.
Please advise on how to correct this.