Conditionally formatting Chart Labels?

I have a relatively simple chart in a worksheet that contains the number of parts created (y-axis) that's graphed by date. On the same chart, I'm also graphing a percentage (scrap/parts created).

The percentage bar is labeled directly on the chart above the bar itself (see attached picture), and I'm hoping to change the color of that label but only if the % is above 4% (thus the 8% on the bar should be one color and the 3% should be another)

Is this something that can be done in Excel itself? Or do I need to code something in VBA to accomplish this?

Alternatively, the bar itself could change colors, but that doesn't seem like a simpler solution...