Wat Phon Chai, the temple behind the big white gate, plays a major role in the Phi Ta Khon festivities, so it's an appropriate home for this museum, also known as the Dansai Folk Museum. It has a collection of costumes worn during the celebrations and a display showing how the masks are made. There are few English captions, but the masks are great to see nonetheless.
From the Dan Sai bus stop on the main highway, walk up the side road towards Dan Sai and the museum is on your right after 200m.