In general, motion of orbital objects is following the Kepler's law, and an orbit determination system in ground stations has the algorithm to analyze the Kepler's motion. A tethered satellite system (TSS) is the representative future space system, and does not follow the conventional space dynamics. Therefore, new methods for the orbit determination of TSS are necessary. Reviewing the previous researches, the following three problems are considered in this paper. At first, the sensitivity that the length of tether and the librational motion influence observations, which are range, range rate, and direction, is researched. Secondly, the new filtering algorithm is proposed for the orbit determination of a TSS. Thirdly, the initial orbit determination of a TSS is proposed. After the considerations, it was made clear that observations of range and range rate can reflect the motion of a TSS, on the other hand, observations of direction are not effective for the detection of a TSS motion. Moreover, the new algorithm for the orbit determination of a TSS showed the excellent performance. When the motion of the center-of-mass is already known, methods of the initial determination of the length of tether, librational angle, and angular velocity were introduced, and the performance was proved