While a system state in quantum mechanics can be described by its state vector $\ket{\psi}$, it can also be described by a so called density operator $\rho$.
We define the density operator to be a positive operator $\rho$ with a trace equal to one ($\tr{\rho} = 1$).
This follows (verbatim) the Postulates as written in \cite{Nielsen.Chuang:2010}.
1. Associated to any isolated physical system is a complex vector space with inner product (that is, a Hilbert space) known as the state space of the system. The system is completely described by its density operator, which is a positive operator $\rho$ with trace one, acting on the state space of the system. If a quantum system is in the state $\rho_i$ with probability $p_i$, then the density operator for the system is $\sum_i{p_i \rho_i}$.
2. The evolution of a closed quantum system is described by a unitary transformation. That is, the state $\rho$ of the system at time $t_1$ is related to the state $\rho'$ of the system at time $t_2$ by a unitary operator $U$ which depends only on the times $t_1$ and $t_2$, $$\rho' = U \rho U^\dagger$$.
3.
4.
For multiple pure states it is defined as $$ \rho = \sum_i p_i{\ket{\psi_i}\bra{\psi_i}} $$.