How to delete rows or columns when you get not enough memory

I have been given a 40mb xlsb file. I noticed on a worksheet that it 1048576 rows of which only about 150 are being used. Also it goes out to column XFD. I haved cleared contents tried saving etc. If I try to delete a row or column I get an error not enough memory. I know that all these blank rows and columns are blowing up the file size. How do I get rid of them?

Any help would be greatly appreciated.

Make sure there isn't something hidden down there, save the file, close it and open it again.

I have cleared contents and tried to clear formats but it won't let me clear formats, says not enough memory.