Rechargeable sodium-ion batteries have received growing attention as promising alternatives to lithium-ion batteries for grid energy storage devices because of the abundance of Na resources as well as its high theoretical capacity of 1165 mAh g^-1, and low electrochemical potential (-2.71 V vs. standard hydrogen electrode). However the safety issues still remain as the most critical challenges for its wide application. The rise of solid state electrolyte (SSE) with high Na-ion conductivity has brought about new opportunities for the development of sodium metal batteries. The recently reported Na-ion superior conductor Na₃SbS₄ (NSS) with a room temperature ionic conductivity of 1.2 mS cm^-1 represents the highest Na-ion conductivity. However, NSS is unstable with Na metal and will decompose to Na₂S upon cycling confirmed with synchrotron X-ray diffraction and XPS studies. The formation of Na₂S leads to the increase of interfacial resistance and deteriorates the battery performance. To stabilize the interface between Na and electrolyte, we report a multilayer cellulose-supported PEO polymer(CPEO)/NSS electrolyte design which combines the high ionic conductivity of NSS and the high stability of CPEO layer, thus resulting in a stable electrochemical process of Na stripping/plating (Figure b). Symmetric cell with CPEO/NSS multilayer electrolyte demonstrate a stable areal specific resistance (ASR) of 2000 Ω cm² for 130 cycles, while the battery with pristine NSS electrolyte shows an increasing ASR from 80 Ω cm² to 4300 Ω cm². The superior interfacial stability of multilayer electrolyte would be promising for designing all-solid-state sodium batteries.