I was wondering if anyone knows why when I turn on the headlight switch knob to the right to flip the headlights, they won’t work and lights won’t turn on, but when I try to do the overheads, they turn on no problem (pop up and pop down).

Any insight on how to fix this would be great. Thank you all!

It is very difficult to pin point the issue from your description that could be both mechanically or electronically.
Mechanically means it could be simple as worn out outer switch knob (plastic) without the metal insert...etc.
Electronically could be faulty switch, shorted ...etc.

My suggestion for the start will be try with other switch and see if symptoms remains the same or not.....good luck.

PS This issue started recently or it's been acting since you are owning your car?

I just got the car yesterday so I noticed it. I heard about a RTR relay in the fuse box, the switch itself, and a 10A fuse in the kicker panel that could be the issue. Or it could be the relay under the dash as well.

So not sure....

Also, please excuse my ignorance, but was is the difference between the dimmer switch and the headlight switch? Where is the dimmer switch located? Is it this thing?

https://rover.ebay.com/rover/0/0/0?mpre ... 2948952343

As far as I remember (my last AE86 was like over 7yrs ago....sorry), dimmer switch is built in the headlight switch / outer ring portion.
...or am I misunderstanding the meaning of "dimmer" ? Please advise.

I am not sure how many AE86 you went through but, I highly suggest you to acquire FSM that you won't regret.