userform button is depressed and unable to get it to go back to normal.

I have a userform that opens when you click "add new item" and "close" when you click close, however if you do that then click a second time on the "add new item" button the "close" button is depressed in and looks odd, is this just a glitch on my computer, or did I write something wrong.

You've inserted Toggle Buttons rather than CommandButtons.