Fast Excel open, Slow Workbook open

I need more information what os are you running?
if windows there are a few possibilities but the main ones are likely the excel file extensions are not correctly configured in explorer causing a lag when trying to open them uninstalling excel and reinstalling should correct this problem.
or you could have spyware, most spyware scans files you try to open for sensitive information.
you could have a virus scan program set up for active monitoring meaning the virus scan will scan all files when being opened for malicious code.
malware bytes has an active monitoring setting when used can delay certain file types.
and because excel includes vbs scripts most virus scan programs are set to scan them by default

Sorry I didn't include enough info. I'm on a Windows 7 machine, running IE 11.

I'm in a work environment, so really don't want to try to uninstall/reinstall Excel if I can help it. We're in State government, so have a pretty proactive IT group, so I doubt spyware. We use LANDesk Antivirus, but I have no options to configure it.

UPDATE: I looked at the file associations, and all Excel file types have Excel as the default program to open them.

its not if its configured for excel its how its configured windows sends a command to the excel program when opening files that tells excel how to handle the file some times the commands get messed up by other programs but it sounds like it may be the antivirus you use, and as far as reinstalling you may be able to see if there is an option to repair the program.
try this instead of opening the file try dragging the icon for the file into the icon for excel, does it still open slowly?

and is it every excel file or just the one?

it something that happens some times its a registry thing you wont see the corruption but its there I have always said that windows is a virus it follows all the guidelines of a virus but one, its not small, and windows tends to screw its self up.

if this solved your problem please don't forget to add reputation.