How to shoot with sunglasses on?

I just bought my first proper camera, an OM-5 Mark II, and I absolutely love it. But I ran into an issue on a two week trip to Albania.

It was very sunny, so I wore prescription sunglasses and used the viewfinder. The problem is that I cannot accurately see colours that way. My prescription is around minus ten, so going without glasses is not an option, also I can't wear contacts.

I bought this camera mainly for travel and vacations, which are exactly the situations where I will almost always be wearing sunglasses. Right now I ended up shooting everything on auto, which feels like it defeats the purpose of getting a more advanced camera.

Has anyone dealt with something similar or found a good workaround? I am starting to wonder if this kind of photography setup is actually going to work for me.