Add data not in original range to pivot table

I have a pivot table that I am trying to add extra data to. I used a pivot table to sort the data, and now I want to add another column in the pivot bar chart. When I right click on the graph and try to "Select data source" everything is grayed out. Is there a workaround to this so I could add another selection to the pivot table? There is not a way to include the data in the original source, so that is not an option for me.

Thanks!