Can you use arrow keys to edit a range when selecting data in a chart?

When I'm selecting a data range for a series for a chart in excel, I sometimes misclick between digits. So we'll say my original range is B2:B200. I want to make this range B2:B300. I accidentally click between the two zeroes instead of the 3 and the zero (Oddly enough, this is very easy to duplicate since it seems excel likes to place your cursor a few pixels to the left of what it actually is when you first edit a series). You would think I would be able to just hit the left arrow key since I've already selected the values. Instead, I hit the left arrow key and suddenly it instead changes it to B2:B2 + A200. I understand that excel thinks I'm using the arrow keys to select the data that I want for the chart, but this messes up the entire range and I honestly can't think of anyone who would find any practical use from this situation. It messes up the formula so bad that if you accidentally hit enter, you basically have to start from scratch so after you do this a handful of times you just learn to hit escape as soon as you see it happen. Is this unavoidable?

Thanks for the explanation! I'm assuming I can't make this default into Edit mode instead of point mode when I click it?