Linux Mint still has some scaling problems compared with other distributions, that can affect things like that.
I am not sure how to solve it for the general file explorer windows; however, with Firefox you can remove Linux Mint's titlebars and use Firefox' native ones.
Can you use the native buttons of Firefox, to see if those are bigger?
To do this, right click on the space between the address bar and the extensions (if any):
Then select Customize Toolbar in the menu that pops up, and there untick the check box on the lower left corner.
Once done, click Done and see if the new buttons have a reasonable size.