Thumbnails in Explorer
With Cobalt Photo installed, Explorer shows Photoshop, Krita, SVG, JPEG XL and RAW files as pictures, not icons. It is free, like the viewer.
Thumbnails and the preview pane
Explorer shows thumbnails only of the pictures Windows reads itself; for the rest it shows the app's icon. Cobalt Photo gives it the pictures too: Photoshop (PSD, PSB), Krita, OpenRaster, TGA, SVG, macOS icons, JPEG XL and the RAW files of cameras get a thumbnail like any photo, at every size of Explorer's view.
The preview pane (Alt+P in Explorer) shows the same files, fitted to the pane, and grows with it.
Both can be turned off in Settings → File association: Thumbnails in Explorer and Preview pane in Explorer.
For Photoshop's files, step by step: How to view PSD files without Photoshop.