Tweak to SUMIFS to exclude a letter in another cell

Hello,

In cell M1 I have a single letter than can change, currently it is the letter P

I need to tweak this formula to say SUMIFS anything but the single letter contained within M1

I have tried adding <> but cant seem to get it to work.

=SUMIFS(Sheet2!$E:$E,Sheet2!$B:$B,Sheet1!$J$1,Sheet2!$O:$O,Sheet1!$M$1,Sheet2!$D:$D,Sheet1!$F5,Sheet2!N:N,"E")

Thanks BVG

How about
=SUMIFS(Sheet2!$E:$E,Sheet2!$B:$B,Sheet1!$J$1,Sheet2!$O:$O,"<>"&Sheet1!$M$1,Sheet2!$D:$D,Sheet1!$F5,Sheet2!N:N,"E")

If the formula is on sheet1 then you should get rid of the sheet1 references & just use the cell address