Principal curves are smooth, self‑consistent curves that pass through the center of a data distribution and are widely used for dimensionality reduction and feature extraction. However, their traditional definitions are ill‑posed and lead to practical difficulties in algorithm design. This dissertation introduces a novel framework that redefines principal curves, surfaces, and manifolds using the gradient and Hessian of the probability density function, providing a clear geometric interpretation and a unified view of clustering, principal curves, and manifold learning. The approach supports multiple density estimation methods and leads to practical algorithms, which are demonstrated across diverse applications including image processing, signal analysis, and manifold learning.