How to stop adding automatic reference to cell when I click on it after I wrote a formula

When I'm writing a formula in a cell then click on another cell it adds a reference to that cell. How do I stop this from happening?
For example--when I have copied the contents of a cell, and I am trying to simply paste it into another cell, it will instead try to add the contents of that cell to the first cell (=$A$9 or something like that will appear in the second cell while $A$9 is added to the formula in the first cell) and I will have to press Esc, which sometimes works (other times it goes back to selecting the first cell), and then paste it.
When I google this most of the results are about the unrelated question of keeping absolute values when you paste into another cell. This is what I am talking about: how-to-get-excel-out-of-select-cell-mode superuser