It doesn't appear to be possible with the current Windows 11 taskbar.

The best I've got is pressing Win+E to open Explorer. That opens "Home" by default; you can pin "This PC" to the quick launch list.

OK. Found this: https://www.windowscentral.com/software-...-11-2022-update

You can change the default from "Home" to "This PC" in the options page. So you can pin Explorer itself.

(I originally looked in Options, but due to the high DPI on my screen (and my old eyes), I literally didn't see the drop-down)