A file window is not just a list. It reads metadata, generates previews, checks icons registered by other programs, and waits for every location it has been asked to show.
Network and removable locations are the biggest risk. A disconnected drive letter or an unreachable server makes the window wait for a timeout that can last a minute.
Thumbnail generation causes the second pattern: folders with thousands of photos or videos freeze while previews are built, then behave normally afterwards.
Third-party context-menu entries are the quiet cause. Every installed program that adds a right-click option runs code inside the file window, and one badly behaved entry hangs the lot.