The cube-like Fe3O4@SiO2@Ag (FSA) nanocomposites with great SERS activity have been successfully synthesized by a layer-by-layer procedure in this paper. The cube-like Fe3O4@SiO2 core–shell structures were prepared via a new route and Ag nanoparticles were introduced onto their surface through a one-pot hydrothermal reaction. By controlling the reaction time, the coverage rate of Ag on the FSA surface could be tuned, and then a series of FSA composites were obtained. The SERS properties of these FSA composites were investigated using p-aminothiophenol (p-ATP) as the probe molecule. It was found that the FSA composites synthesized with a reaction time of 6 h showed the best SERS performance, and the detection limit for p-ATP could reach 1 × 10−7 M. For practical application, the FSA composites were also used to detect thiram, one of the dithiocarbamate fungicides that has been widely used as a pesticide in agriculture. The detection limit is as low as 1 × 10−6 M (0.24 ppm), lower than the maximal residue limit of 7 ppm in fruit prescribed by the US Environmental Protection Agency. The resulting substrate with high SERS activity, stability and strong magnetic responsivity makes the FSA composite a perfect choice for practical SERS detection applications.