The fourth generation of synchrotron radiation sources, commonly referred to as the Free Electron Laser (FEL), provides an intense source of brilliant X-ray beams enabling the investigation of matter at the atomic scale with unprecedented time resolution. These sources require the use of conventional linear accelerators providing high electron beam performance. The achievement of chirped pulse amplification allowing lasers to be operated at the Terawatt range, opened the way for the Laser Plasma Acceleration (LPA) technique where high energy electron bunches with high current can be produced within a very short centimeter-scale distance. Such an advanced acceleration concept is of great interest to be qualified by an FEL application for compact X-ray light sources. We explore in this paper what the LPA specificities imply on the design of the undulator, part of the gain medium. First, the LPA concept and state-of-art are presented showing the different operation regimes and what electron beam parameters are likely to be achieved. The LPA scaling laws are discussed afterwards to better understand what laser or plasma parameters have to be adjusted in order to improve electron beam quality. The FEL is secondly discussed starting with the spontaneous emission, followed by the different FEL configurations, the electron beam transport to the undulator and finally the scaling laws and correction terms in the high gain case. Then, the different types of compact undulators that can be implemented for an LPA based FEL application are analyzed. Finally, examples of relevant experiments are reported by describing the transport beamline, presenting the spontaneous emission characteristics achieved so far and the future prospects